一 安装calico网络插件
官方文档 https://docs.projectcalico.org/v3.0/getting-started/kubernetes/installation/hosted/hosted
1.1 下载yaml文件
使用V3版本，下载对应的yaml文件
在master1 节点上
mkdir /opt/yaml
cd /opt/yaml
wget https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/rbac.yaml
wget https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/hosted/calico.yaml
wget https://docs.projectcalico.org/v3.7/manifests/calico-etcd.yaml
calico.yaml和rbac.yaml。其中rbac.yaml 是权限管理yaml，这里面不需要改任何东西，是（rbac）授权文件。
calico.yaml 中比较重要的是etcd tls 文件配置。要配置到
/calico-secrets/ 着个目录下，配置到其他目录下是不生效的
根据yaml文件创建对应的rbac权限
kubectl apply -f rbac.yaml
查看创建的权限
kubectl get ClusterRole
NAME AGE
admin 15d
calico-kube-controllers 5s
calico-node 5s
1.1 创建ETCD tls文件，在三台master节点上
mkdir /calico-secrets/
cp /etc/kubernetes/ssl/ca.pem /calico-secrets/etcd-ca
cp /etc/kubernetes/ssl/kubernetes.pem /calico-secrets/etcd-cert
cp /etc/kubernetes/ssl/kubernetes-key.pem /calico-secrets/etcd-key
1.2 calico etcd 相关证书做base64。如果没有配置会报错
data:
# Populate the following files with etcd TLS configuration if desired, but leave blank if
# not using TLS for etcd.
# This self-hosted install expects three files with the following names. The values
# should be base64 encoded strings of the entire contents of each file.
# etcd-key: (cat /calico-secrets/etcd-key | base64 | tr -d '\n')
# etcd-cert: (cat /calico-secrets/etcd-cert | base64 | tr -d '\n')
# etcd-ca: (cat /calico-secrets/etcd-ca | base64 | tr -d '\n')
1.3 更改calico.yaml配置文件
cat calico.yaml
# Calico Version v3.1.1
# https://docs.projectcalico.org/v3.1/releases#v3.1.1
# This manifest includes the following component versions:
# calico/node:v3.1.1
# calico/cni:v3.1.1
# calico/kube-controllers:v3.1.1
# This ConfigMap is used to configure a self-hosted Calico installation.
kind: ConfigMap
apiVersion: v1
metadata:
name: calico-config
namespace: kube-system
data:
# Configure this with the location of your etcd cluster.
etcd_endpoints: "https://192.168.70.21:2379,https://10.7.1.45:2379,https://10.7.4.17:2379"
# Configure the Calico backend to use.
calico_backend: "bird"
# The CNI network configuration to install on each node.
cni_network_config: |-
{
"name": "k8s-pod-network",
"cniVersion": "0.3.0",
"plugins": [
{
"type": "calico",
"etcd_endpoints": "__ETCD_ENDPOINTS__",
"etcd_key_file": "__ETCD_KEY_FILE__",
"etcd_cert_file": "__ETCD_CERT_FILE__",
"etcd_ca_cert_file": "__ETCD_CA_CERT_FILE__",
"log_level": "info",
"mtu": 1500,
"ipam": {
"type": "calico-ipam"
},
"policy": {
"type": "k8s"
},
"kubernetes": {
"kubeconfig": "__KUBECONFIG_FILEPATH__"
}
},
{
"type": "portmap",
"snat": true,
"capabilities": {"portMappings": true}
}
]
}
# If you're using TLS enabled etcd uncomment the following.
# You must also populate the Secret below with these files.
etcd_ca: "/calico-secrets/etcd-ca" # "/calico-secrets/etcd-ca"
etcd_cert: "/calico-secrets/etcd-cert" # "/calico-secrets/etcd-cert"
etcd_key: "/calico-secrets/etcd-key" # "/calico-secrets/etcd-key"
---
# The following contains k8s Secrets for use with a TLS enabled etcd cluster.
# For information on populating Secrets, see http://kubernetes.io/docs/user-guide/secrets/
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: calico-etcd-secrets
namespace: kube-system
data:
# Populate the following files with etcd TLS configuration if desired, but leave blank if
# not using TLS for etcd.
# This self-hosted install expects three files with the following names. The values
# should be base64 encoded strings of the entire contents of each file.
# etcd-key: (cat /etc/kubernetes/ssl/etcd-key.pem | base64 | tr -d '\n')
# etcd-cert: (cat /etc/kubernetes/ssl/etcd.pem | base64 | tr -d '\n')
# etcd-ca: (cat /etc/kubernetes/ssl/ca.pem | base64 | tr -d '\n')
etcd-key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBdDUydUdhRHdhRHBxNktMejFUby9EckhQbUNNTUUvNlVQblZiT0Rna0h1QktVMlpDClBKNjh1Uk11SjBPRjRNRS9PSTJEU25PZVBjTklSREM5U1lkMUJXa2ZLZjNtNExMRWFoUmhscXkrSXlQeEpwT2wKaHJPTHFjR2hBdXRCNlpKcTRhN3ZXeXZMT3N1OHhkTnIxN1R1R0ZzczlBZ0RISFdGaEJXbGkzZjZaUXM0NWdvTgpIS0p6R2FpY29jTFlFZlRoR0xVMk5qZ1N0K3c3TGZSS05HNktqMm4zSkNHcFJ0SExybjI2SHIzc3BnazI1RzBHCm5LZEJQOXRnekJESXQzaHN4amkzU0g3UlgzQkJYMDEwc3BKRWlubXFOMmpxd1R0WExvYVYrejlzQVNjcTZUTzUKOW5PMVM2d1JXck5Cb3VPbExqNGxZQ3R4UDJxL21TYWdlMS82Q1FJREFRQUJBb0lCQUJVTExrUGpyUW5WQUNHdgpZMUNodjA1WTFKajBFMU5MVHdRbGdSSGRaVTVnbUVRY3FTUzNjSnVwZGhXcjZIRU0xTWtQZHVlV2ZxRkhlMFhOCmJEUThJUTVBc1FDb0I4amkxQjN3UGdyOXUwRGR3cXFRSzExWHdIN3hJUm5yaW45UmQ0eHI5eVMwelRMczNMS3oKZTEyYWI5czh2OWNyLytkSUZNaEJqdHFBSXFObnh2TlpTdVJqNngzdDJRZmJKTVNhamtTZlNVK01tSVZuZzJNYgpsTFJkeFRtUEtpcE5oRlA0STlPdlRQNlRiR1lUcUlNRUIxelpIVzVNdXBJaVl6UFArVVk1QWFFdmNtYTc3SlExClhOQjZpZ1VvT1RtdndMbVptR0wrdU9wZHpYM3FCR1RVcmN4TXZIWFFHMG04VnBUWVVhbjNaZmFCditoQ2pMMHMKRkt4WXNuRUNnWUVBeGVSdnlNRDBmL3hCWWxwOVFFNlAzREJiaEJ6c2M4a2pScUY4b2lBTW1MNmRNbG9qeEVlbQpnanJIM2tWeWtiUU1DYlJJbjdLRU5EaFdreEN6NWtYZE5XcEFYTHVrVU8xVWJEa3lxVDB4NE5WQ2M3dERyV09aClBKNkc5SjdvbWJjUWRYRUtjaW1nNDFxKzB1dXg2blcvVWxRMytMaFMxZ3RMNVhqeXFaNnJ4eTBDZ1lFQTdZZ1cKVW5xOFlqdFVZWVMzMmw3bjUxRTB0N0d2YWhxRFF3NDVTbG9yU2ZHa21BV0wxSVNXNVIzWm40YUdsODBUSStUNApVY0RIRE9tNWQ0RjA1dDRuNEJDcm94dHI2bFltTjZVclc4WFYwdDB2NSswSHZsdnp4SldNNktqMzFnT2UvMmE5Ci82NHl6cFVOaEhJNHlJZ0M4RTZXUi81a3ZpdzdUOHA5VVNKa1I4MENnWUF4VE5ySWNjRzN6TmxoTGxmNzJGYWcKclc5dk4yRjA2d3dkd1lkeVFIZkM2ZUh4dEdFcXVncnljTUJ3VUc3ZGU3UndDbmo1U0JrRmhXK1VEZG96cVA0VwpHNVZUUlBBSHRVaU4rYVgwWWFsMmNNcXdXRnZnNjJvMm51dlNMM0NWTXZVbnZQUzBRa2l4dTE3RTN6RWd6eHJOCjRPdGN6RmJldWt1N0xHbVA5bGFuTlFLQmdRREx4eFJmYml4dm1iYXFjdEhINWVQTjd6enEydVZDZ2J4YXFyazcKc0YreThEdmtyOTFtVk5tZmRoeGRYMnZZajNiOE5GTlV6NjMweGc3M0YzalRGNFBlRm04cFh6MFZyUkxjYWpibwpaTEVzVmRYamd0WkIyV0VvSmFUZGRSZ3A0ckZsS3grVldqejhQYU43SFFMYUJ0VitrKzBabG9XWElSdEJPTmFKCjVKZGpnUUtCZ0gvdlZxR1JHTGw0azJ1Z2xNY1hnSGhaZlJnZW1POGtMQyt5Zk9LOXcrTE9Lc0FPMnBsNlRFRmQKU0taQVZXcFREOGN5QVFCNGcvYmJJRjBrTEpzNFJMK0IzeVphTFdseGR4U1dHSVNSQU80SEMwaXFWZElHOEpjVwovV3dTZVFFeVNHUUJSWnFFcTRZVFN0SEdwbGVsSTF0dmNJUE5KdzZaeUwzb2RrQUJtdSthCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==
etcd-cert: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVuekNDQTRlZ0F3SUJBZ0lVRkxxOWwzY3A2cWpLMWpuT3kxZTlvMzE5Rkwwd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1pURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByCmRXSmxjbTVsZEdWek1DQVhEVEU0TURReU9EQTJORFV3TUZvWUR6SXhNVGd3TkRBME1EWTBOVEF3V2pCbE1Rc3cKQ1FZRFZRUUdFd0pEVGpFUU1BNEdBMVVFQ0JNSFFtVnBTbWx1WnpFUU1BNEdBMVVFQnhNSFFtVnBTbWx1WnpFTQpNQW9HQTFVRUNoTURhemh6TVE4d0RRWURWUVFMRXdaVGVYTjBaVzB4RXpBUkJnTlZCQU1UQ210MVltVnlibVYwClpYTXdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFDM25hNFpvUEJvT21yb292UFYKT2o4T3NjK1lJd3dUL3BRK2RWczRPQ1FlNEVwVFprSThucnk1RXk0blE0WGd3VDg0allOS2M1NDl3MGhFTUwxSgpoM1VGYVI4cC9lYmdzc1JxRkdHV3JMNGpJL0VtazZXR3M0dXB3YUVDNjBIcGttcmhydTliSzhzNnk3ekYwMnZYCnRPNFlXeXowQ0FNY2RZV0VGYVdMZC9wbEN6am1DZzBjb25NWnFKeWh3dGdSOU9FWXRUWTJPQkszN0RzdDlFbzAKYm9xUGFmY2tJYWxHMGN1dWZib2V2ZXltQ1Ria2JRYWNwMEUvMjJETUVNaTNlR3pHT0xkSWZ0RmZjRUZmVFhTeQpra1NLZWFvM2FPckJPMWN1aHBYN1Ayd0JKeXJwTTduMmM3VkxyQkZhczBHaTQ2VXVQaVZnSzNFL2FyK1pKcUI3Clgvb0pBZ01CQUFHamdnRkRNSUlCUHpBT0JnTlZIUThCQWY4RUJBTUNCYUF3SFFZRFZSMGxCQll3RkFZSUt3WUIKQlFVSEF3RUdDQ3NHQVFVRkJ3TUNNQXdHQTFVZEV3RUIvd1FDTUFBd0hRWURWUjBPQkJZRUZCQWVOemhhbE1HcwptUEdBZzFSV3BPTTdqeW4rTUI4R0ExVWRJd1FZTUJhQUZEZ2J4NU0vVVBna3UxZG9TQmRGeFVYRjhGS1BNSUcvCkJnTlZIUkVFZ2Jjd2diU0NDbXQxWW1WeWJtVjBaWE9DRW10MVltVnlibVYwWlhNdVpHVm1ZWFZzZElJV2EzVmkKWlhKdVpYUmxjeTVrWldaaGRXeDBMbk4yWTRJZWEzVmlaWEp1WlhSbGN5NWtaV1poZFd4MExuTjJZeTVqYkhWegpkR1Z5Z2lScmRXSmxjbTVsZEdWekxtUmxabUYxYkhRdWMzWmpMbU5zZFhOMFpYSXViRzlqWVd5SEJIOEFBQUdICkJNQ29UQXFIQk1Db1RBdUhCTUNvVEE2SEJNQ29UUHFIQkFvSEFTMkhCTUNvUmhXSEJBb0hCQkdIQkFyK0FBRXcKRFFZSktvWklodmNOQVFFTEJRQURnZ0VCQUFhN0crR1FkQVRnTmU1RHRXWDgwRytjZ2tLRzBuTHErUUdiNkJBdQo5UE9qTWVYMERKSzZPWGVUVE5Yb0J6Q1BUTHY4YkhtdFhxN3I4S0x5aGo5RnBFelRBL3lkMkpQQlBvdUh1S1VLCmlZOUFLeEVvTkRGNVI5V2laYWRyMUthaFNLU2sxSGh3WDFMOWh3b0JJYTREYWlBUWhmRXdpd2Vpc2NpRGJBcmYKMW1qRk5XbEUwUFB0bWVIUVFUK3JhRE4yUHlZdTRGVlFraGVpMUdFQnFrMHRXTFB2YjNaZyt4Rk1kTkRWNHZ5NgpSWjlGejFVajVlamFzb1dRS1dqRHJLM0t5ZWp1WVNCR1FxOUZqLzA0eDN2L2JHK05DOWJEelJselJNdDlsazNJCjhOM29aSXNid0szM1JkVDU2WUtKRXB1RGcwbkdYeDViQ3JKZ2RSdTc5aTBtUTNBPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
etcd-ca: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR3RENDQXFpZ0F3SUJBZ0lVTENJM1UyUDVZaEpaSHlCYnpCYThvWjlWQzg4d0RRWUpLb1pJaHZjTkFRRUwKQlFBd1pURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByCmRXSmxjbTVsZEdWek1DQVhEVEU0TURReU5EQTNNekF3TUZvWUR6SXhNVGd3TXpNeE1EY3pNREF3V2pCbE1Rc3cKQ1FZRFZRUUdFd0pEVGpFUU1BNEdBMVVFQ0JNSFFtVnBTbWx1WnpFUU1BNEdBMVVFQnhNSFFtVnBTbWx1WnpFTQpNQW9HQTFVRUNoTURhemh6TVE4d0RRWURWUVFMRXdaVGVYTjBaVzB4RXpBUkJnTlZCQU1UQ210MVltVnlibVYwClpYTXdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFERGJkclU0UXNycGpVSzJFQ3UKaTJWYUxVN0RqM3hxWVYvdUpvei9CbnppRHJwRkxzSEdYZWJPUWNYV1AxTHdXK1ZUOWJWWlBJTWdrT2Z3YUVObQpMN1pkc2I0cHdVY0RtemxkVkJ1dmJJVGh1cG9MTmoyTGhhWlpqWjJUK01UVDI3TXhKWTJycGE3Qlc2U1hUSm9vCmFNZzR0OTZlZFpSRngyVlN0dEFLOFJxL1M3RGdCSDQ0eVdlUmphYjJaU2FKK01mbUxOYW5pVTIxZkwrWGlQRWQKWVBTN3piOENXMFU5NjJvY0lTU0tMSzhQdlhnb3h5ZGRYSlNwTG9XajlmdTY0a2tHMFZQSUN2QUVOUDROd0V5TApyTTdFQ3ZLTEFGZUNIWHd4V2kyTWU0QklPbnZueHNnbW5EczhWWXBDM2x5d1RwK2ZrY2V5RUI0VGptODlEaWViClJBVVBBZ01CQUFHalpqQmtNQTRHQTFVZER3RUIvd1FFQXdJQkJqQVNCZ05WSFJNQkFmOEVDREFHQVFIL0FnRUMKTUIwR0ExVWREZ1FXQkJRNEc4ZVRQMUQ0Skx0WGFFZ1hSY1ZGeGZCU2p6QWZCZ05WSFNNRUdEQVdnQlE0RzhlVApQMUQ0Skx0WGFFZ1hSY1ZGeGZCU2p6QU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFKSXRPczVxaDBHSWxJcFM0Ckk4M2hVS0ZJZFpoUjF5RlQyd2RzWU5Yd1VGTituY1BYSjBIcThyTlNSRDJHa2VyT0xkZ0VOa3djZ01Idmo0TksKcllDdzVXZVNRUHA1eWRRSDlZMkQwK0p5ZE1DK2xVc2ZYLzBzSUhTWG1pN0FSb3RuaG50Q1VRa0MrU08vYnFQMwpKcWFXLzNDa0FpdUNhbnAzbE9UaWI4TEhNYklQWFhHT1J6djJacVlPN3B6SHhOYVBuSmVDRE9kT1NGckptTnBpCm5FQjB6TDFjb0EyV3Z0dUhwejFVay9PNzhKVWhGMWpFL0l3L3UxWDNvMFV5dGxscFp1RytMSXJQWHZ0SFF3WXcKcVNhNW1CY1JIQXdiazJyRVFLL29ZdUZ1aHoxOXgyVVF6ZUpQdDZ0bHVkNnZEUWhDeFpNL3VKSWRwUlVwK3EycgpxRW4wVlE9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
---
# This manifest installs the calico/node container, as well
# as the Calico CNI plugins and network config on
# each master and worker node in a Kubernetes cluster.
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
name: calico-node
namespace: kube-system
labels:
k8s-app: calico-node
spec:
selector:
matchLabels:
k8s-app: calico-node
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
template:
metadata:
labels:
k8s-app: calico-node
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
spec:
hostNetwork: true
tolerations:
# Make sure calico/node gets scheduled on all nodes.
- effect: NoSchedule
operator: Exists
# Mark the pod as a critical add-on for rescheduling.
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
operator: Exists
serviceAccountName: calico-node
# Minimize downtime during a rolling upgrade or deletion; tell Kubernetes to do a "force
# deletion": https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods.
terminationGracePeriodSeconds: 0
containers:
# Runs calico/node container on each Kubernetes node. This
# container programs network policy and routes on each
# host.
- name: calico-node
image: quay.io/calico/node:v3.1.1
env:
# The location of the Calico etcd cluster.
- name: ETCD_ENDPOINTS
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_endpoints
# Choose the backend to use.
- name: CALICO_NETWORKING_BACKEND
valueFrom:
configMapKeyRef:
name: calico-config
key: calico_backend
# Cluster type to identify the deployment type
- name: CLUSTER_TYPE
value: "k8s,bgp"
# Disable file logging so `kubectl logs` works.
- name: CALICO_DISABLE_FILE_LOGGING
value: "true"
# Set noderef for node controller.
- name: CALICO_K8S_NODE_REF
valueFrom:
fieldRef:
fieldPath: spec.nodeName
# Set Felix endpoint to host default action to ACCEPT.
- name: FELIX_DEFAULTENDPOINTTOHOSTACTION
value: "ACCEPT"
# The default IPv4 pool to create on startup if none exists. Pod IPs will be
# chosen from this range. Changing this value after installation will have
# no effect. This should fall within `--cluster-cidr`.
- name: CALICO_IPV4POOL_CIDR
value: "172.20.0.0/16"
- name: CALICO_IPV4POOL_IPIP
value: "Always"
# Disable IPv6 on Kubernetes.
- name: FELIX_IPV6SUPPORT
value: "false"
# Set Felix logging to "info"
- name: FELIX_LOGSEVERITYSCREEN
value: "info"
# Set MTU for tunnel device used if ipip is enabled
- name: FELIX_IPINIPMTU
value: "1440"
# Location of the CA certificate for etcd.
- name: ETCD_CA_CERT_FILE
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_ca
# Location of the client key for etcd.
- name: ETCD_KEY_FILE
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_key
# Location of the client certificate for etcd.
- name: ETCD_CERT_FILE
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_cert
# Auto-detect the BGP IP address.
- name: IP
value: "autodetect"
- name: FELIX_HEALTHENABLED
value: "true"
securityContext:
privileged: true
resources:
requests:
cpu: 250m
livenessProbe:
httpGet:
path: /liveness
port: 9099
periodSeconds: 10
initialDelaySeconds: 10
failureThreshold: 6
readinessProbe:
httpGet:
path: /readiness
port: 9099
periodSeconds: 10
volumeMounts:
- mountPath: /lib/modules
name: lib-modules
readOnly: true
- mountPath: /var/run/calico
name: var-run-calico
readOnly: false
- mountPath: /var/lib/calico
name: var-lib-calico
readOnly: false
- mountPath: /calico-secrets
name: etcd-certs
# This container installs the Calico CNI binaries
# and CNI network config file on each node.
- name: install-cni
image: quay.io/calico/cni:v3.1.1
command: ["/install-cni.sh"]
env:
# Name of the CNI config file to create.
- name: CNI_CONF_NAME
value: "10-calico.conflist"
# The location of the Calico etcd cluster.
- name: ETCD_ENDPOINTS
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_endpoints
# The CNI network config to install on each node.
- name: CNI_NETWORK_CONFIG
valueFrom:
configMapKeyRef:
name: calico-config
key: cni_network_config
volumeMounts:
- mountPath: /host/opt/cni/bin
name: cni-bin-dir
- mountPath: /host/etc/cni/net.d
name: cni-net-dir
- mountPath: /calico-secrets
name: etcd-certs
volumes:
# Used by calico/node.
- name: lib-modules
hostPath:
path: /lib/modules
- name: var-run-calico
hostPath:
path: /var/run/calico
- name: var-lib-calico
hostPath:
path: /var/lib/calico
# Used to install CNI.
- name: cni-bin-dir
hostPath:
path: /opt/cni/bin
- name: cni-net-dir
hostPath:
path: /etc/cni/net.d
# Mount in the etcd TLS secrets with mode 400.
# See https://kubernetes.io/docs/concepts/configuration/secret/
- name: etcd-certs
secret:
secretName: calico-etcd-secrets
defaultMode: 0400
---
# This manifest deploys the Calico Kubernetes controllers.
# See https://github.com/projectcalico/kube-controllers
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: calico-kube-controllers
namespace: kube-system
labels:
k8s-app: calico-kube-controllers
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
spec:
# The controllers can only have a single active instance.
replicas: 1
strategy:
type: Recreate
template:
metadata:
name: calico-kube-controllers
namespace: kube-system
labels:
k8s-app: calico-kube-controllers
spec:
# The controllers must run in the host network namespace so that
# it isn't governed by policy that would prevent it from working.
hostNetwork: true
tolerations:
# Mark the pod as a critical add-on for rescheduling.
- key: CriticalAddonsOnly
operator: Exists
- key: node-role.kubernetes.io/master
effect: NoSchedule
serviceAccountName: calico-kube-controllers
containers:
- name: calico-kube-controllers
image: quay.io/calico/kube-controllers:v3.1.1
env:
# The location of the Calico etcd cluster.
- name: ETCD_ENDPOINTS
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_endpoints
# Location of the CA certificate for etcd.
- name: ETCD_CA_CERT_FILE
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_ca
# Location of the client key for etcd.
- name: ETCD_KEY_FILE
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_key
# Location of the client certificate for etcd.
- name: ETCD_CERT_FILE
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_cert
# Choose which controllers to run.
- name: ENABLED_CONTROLLERS
value: policy,profile,workloadendpoint,node
volumeMounts:
# Mount in the etcd TLS secrets.
- mountPath: /calico-secrets
name: etcd-certs
volumes:
# Mount in the etcd TLS secrets with mode 400.
# See https://kubernetes.io/docs/concepts/configuration/secret/
- name: etcd-certs
secret:
secretName: calico-etcd-secrets
defaultMode: 0400
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: calico-kube-controllers
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: calico-node
namespace: kube-system
1.4 下载依赖的国外镜像
gcr上镜像获取请参考这篇文章
http://blog.csdn.net/qq_27028561/article/details/79064414
k8s.gcr.io/pause-amd64:3.1
使用 Calico 后需要修改 kubelet 配置增加 CNI 设置(--network-plugin=cni)，修改后配置如下：
vim /etc/kubernetes/kubelet
###
## kubernetes kubelet (minion) config
#
## The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
#KUBELET_ADDRESS="--address=192.168.243.184"
#
## The port for the info server to serve on
#KUBELET_PORT="--port=10250"
#
## You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=192.168.76.30"
#
## location of the api-server
## COMMENT THIS ON KUBERNETES 1.8+
#KUBELET_API_SERVER="--api-servers=http://172.20.0.113:8080"
#
## pod infrastructure container
####KUBELET_POD_INFRA_CONTAINER="docker.io/oudi/pod-infrastructure"
#
## Add your own!
#--cgroup-driver=systemd --require-kubeconfig
#--kubeconfig=/etc/kubernetes/kubelet.kubeconfig
#KUBELET_ARGS="--cluster-dns=10.254.0.2 --bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --config=/etc/kubernetes/kubelet.config --cert-dir=/etc/kubernetes/ssl --cluster-domain=cluster.local --hairpin-mode promiscuous-bridge --serialize-image-pulls=false"
KUBELET_ARGS="--network-plugin=cni --bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --config=/etc/kubernetes/kubelet.config --cert-dir=/etc/kubernetes/ssl"
重启
systemctl restart kubelet
1.5 启动calico网络
kubectl apply -f calico.yaml
1.6 验证
kubectl get all -n kube-system
1.7 安装calicoctl验证
下载对应版本yaml文件
wget https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/hosted/calicoctl.yaml
wge https://docs.projectcalico.org/v3.7/manifests/calicoctl-etcd.yaml
更改配置
vim calicoctl.yaml
# Calico Version v3.1.1
# https://docs.projectcalico.org/v3.1/releases#v3.1.1
# This manifest includes the following component versions:
# calico/ctl:v3.1.1
apiVersion: v1
kind: Pod
metadata:
name: calicoctl
namespace: kube-system
spec:
hostNetwork: true
containers:
- name: calicoctl
image: quay.io/calico/ctl:v3.1.1
command: ["/bin/sh", "-c", "while true; do sleep 3600; done"]
env:
- name: ETCD_ENDPOINTS
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_endpoints
# If you're using TLS enabled etcd uncomment the following.
# Location of the CA certificate for etcd.
- name: ETCD_CA_CERT_FILE
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_ca
# Location of the client key for etcd.
- name: ETCD_KEY_FILE
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_key
# Location of the client certificate for etcd.
- name: ETCD_CERT_FILE
valueFrom:
configMapKeyRef:
name: calico-config
key: etcd_cert
volumeMounts:
- mountPath: /calico-secrets
name: etcd-certs
volumes:
# If you're using TLS enabled etcd uncomment the following.
- name: etcd-certs
secret:
secretName: calico-etcd-secrets
在所有node节点 更改kubelet 配置（76.10，7.11,76.14)
vim /etc/kubernetes/kubelet
打开着个端口
KUBELET_PORT="--port=10250"
重启
systemctl restart kubelet
在所有node节点 添加iptables规则（76.10，7.11,76.14)
iptables -I INPUT -p tcp --dport 10250 -j ACCEPT
iptables -I OUTPUT -p tcp --sport 10250 -j ACCEPT
iptables-save
验证
kubectl exec -ti -n kube-system calicoctl -- /calicoctl get profiles -o wide
kubectl exec -ti -n kube-system calicoctl -- /calicoctl get nodes -o wide
NAME ASN IPV4 IPV6
node10 (unknown) 192.168.76.10/24
node11 (unknown) 192.168.76.11/24
node14 (unknown) 192.168.76.14/24
如果报权限不够。执行下面命令、绑定cluster-admin权限
kubectl create clusterrolebinding system:anonymous --clusterrole=cluster-admin --user=system:anonymous
遇到问题解决
kubectl create -f rbac.yaml calico.yaml 创建完后，只能看到一个节点，无法获取到节点NAME
NAME ASN IPV4 IPV6
localhost.localdomain (unknown) 192.168.76.10/24
解决办法，
停止所有k8s服务、master，node，etcd 删除etcd数据文件、删除/var/lib/calico/nodename文件，重启启动所有k8s服务
在所有node节点上，删除/var/lib/calico/nodename 着个文件
rm -fr /var/lib/calico/nodename