linux cpu占用率如何看
328
2022-09-11
2022使用二进制方式搭建k8s集群（亲测有效）
2022使用二进制方式搭建k8s集群（亲测有效）
使用二进制方式搭建k8s集群（亲测有效）
安装要求
在开始之前，部署Kubernetes集群机器需要满足以下几个条件：
一台或多台机器，操作系统CentOS7.x-86_64（个人选择CentOS7.2）硬件配置：2GB或更多RAM，2个CPU或更多CPU，硬盘30GB或更多集群中所有机器之间网络互通可以访问外网，需要拉取镜像，如果服务器不能上网，需要提前下载镜像并导入节点禁止swap分区
准备环境
（1）软件环境：软件|版本 ---|--- Kubernetes|1.19 （2）服务器规划：角色|IP|组件 ---|---|--- k8s-m1|192.168.1.147|kube-apiserver, kube-controller-manager, kube-scheduler, etcd k8s-n1|192.168.1.148|kubelet, kube-proxy, docker etcd
操作系统初始化配置
# 关闭防火墙 如果是minimal安装，默认没有装firewalld systemctl stop firewalld systemctl disable firewalld #关闭selinux sed -i 's/enforcing/disabled/' /etc/selinux/config #永久 setenforce 0 #临时 #关闭swap swapoff -a #临时 sed -ri 's/.*swap.*/#&/' /etc/fstab #永久 #根据规划设置主机名 hostnamectl set-hostname
部署etcd集群
etcd是一个分布式键值存储系统，Kubernetes使用etcd进行数据存储，所以先准备一个etcd数据库，为解决etcd单点故障，应采用集群方式部署，这里使用2台组建集群，当然，你也可以使用3台、5台组建集群，可容忍1台、2台机器故障。节点名称|IP ---|--- etcd-1|192.168.1.147 etcd-2|192.168.1.148 注：为了节省机器，这里与k8s节点机器复用，也可以独立于k8s集群之外部署，只要apiserver能连接就行。
4.1 准备cfssl证书生成工具
cfssl是一个开源的证书管理工具，使用json文件生成证书，相比openssl更方便使用。找任意一台服务器操作，这里用k8s-m1节点。 yum install wget -y wget wget wget chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64 mv cfssl_linux-amd64 /usr/local/bin/cfssl mv cfssljson_linux-amd64 /usr/local/bin/cfssljson mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo #确认文件移到正确的位置 $ ls -l /usr/local/bin/ total 18808 -rwxr-xr-x. 1 root root 10376657 Dec 7 02:36 cfssl -rwxr-xr-x. 1 root root 6595195 Dec 7 02:36 cfssl-certinfo -rwxr-xr-x. 1 root root 2277873 Dec 7 02:35 cfssljson
4.2 生成etcd证书
（1）自签证书颁发机构（CA）创建工作目录： mkdir -p ~/TLS/{etcd,k8s} cd ~/TLS/etcd 自签CA： cat > ca-config.json << EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "{ "expiry": "87600h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] } } } } EOF cat > ca-csr.json << EOF { "CN": "etcd CA", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "Beijing" } ] } EOF 生成证书： $ cfssl gencert -initca ca-csr.json | cfssljson -bare ca - 2022/01/31 20:33:58 [INFO] generating a new CA key and certificate from CSR 2022/01/31 20:33:58 [INFO] generate received request 2022/01/31 20:33:58 [INFO] received CSR 2022/01/31 20:33:58 [INFO] generating key: rsa-2048 2022/01/31 20:33:58 [INFO] encoded CSR $ ls *pem ca-key.pem ca.pem （2）使用自签CA签发etcd HTTPS证书创建证书申请文件： cat > server-csr.json << EOF { "CN": "etcd", "hosts": [ "192.168.1.147", "192.168.1.148" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "Beijing", "ST": "Beijing" } ] } EOF 注：上述文件hosts字段中IP为所有etcd及诶单的集群内部通信IP，一个都不能少！为了方便后期扩容可以多些几个预留的IP。生成证书： $ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server-csr.json | cfssljson -bare server 2022/01/31 20:34:22 [INFO] generate received request 2022/01/31 20:34:22 [INFO] received CSR 2022/01/31 20:34:22 [INFO] generating key: rsa-2048 2022/01/31 20:34:23 [INFO] encoded CSR 2022/01/31 20:34:23 [INFO] signed certificate with serial number 662529260630305134909583808009574907087019149943 2022/01/31 20:34:23 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for websites. For more information see the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum ( specifically, section 10.2.3 ("Information Requirements"). $ ls server*pem server-key.pem server.pem
4.3 从Github下载二进制文件
下载地址： cd ~ wget 部署etcd集群
以下在节点1（k8s-m1）上操作，为简化操作，待会儿将节点1生成的所有文件拷贝到节点2（k8s-n1）。（1）创建工作目录并解压二进制包 mkdir /opt/etcd/{bin,cfg,ssl} -p tar zxvf etcd-v3.4.9-linux-amd64.tar.gz mv etcd-v3.4.9-linux-amd64/{etcd,etcdctl} /opt/etcd/bin/ （2）创建etcd配置文件 cat > /opt/etcd/cfg/etcd.conf << EOF #[Member] ETCD_NAME="etcd-1" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS=" ETCD_LISTEN_CLIENT_URLS=" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS=" ETCD_ADVERTISE_CLIENT_URLS=" ETCD_INITIAL_CLUSTER="etcd-1= ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" EOF （3）systemd管理etcd cat > /usr/lib/systemd/system/etcd.service << EOF [Unit] Description=Etcd Server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify EnvironmentFile=/opt/etcd/cfg/etcd.conf ExecStart=/opt/etcd/bin/etcd \ --cert-file=/opt/etcd/ssl/server.pem \ --key-file=/opt/etcd/ssl/server-key.pem \ --peer-cert-file=/opt/etcd/ssl/server.pem \ --peer-key-file=/opt/etcd/ssl/server-key.pem \ --trusted-ca-file=/opt/etcd/ssl/ca.pem \ --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem \ --logger=zap Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF （4）拷贝刚才生成的证书把刚才生成的证书拷贝到配置文件中的路径： cp /TLS/etcd/ca*pem /TLS/etcd/server*pem /opt/etcd/ssl/
（5）将上面节点1所有的生成文件拷贝到节点2 scp -r /opt/etcd/ root@192.168.1.148:/opt/ scp /usr/lib/systemd/system/etcd.service root@192.168.1.148:/usr/lib/systemd/system/ 然后在节点2修改etcd.conf配置文件中的节点名称和当前服务器IP： vi /opt/etcd/cfg/etcd.conf #[Member] ETCD_NAME="etcd-2" # 修改此处，节点2改为etcd-2 ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS=" # 修改此处为当前服务器IP ETCD_LISTEN_CLIENT_URLS="# 修改此处为当前服务器IP #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="# 修改此处为当前服务器IP ETCD_ADVERTISE_CLIENT_URLS="# 修改此处为当前服务器IP ETCD_INITIAL_CLUSTER="etcd-1= ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" （6）分别在2个节点上操作，启动并设置开机启动 systemctl daemon-reload systemctl start etcd # 只有其中1节点启动正常，另1节点才启动正常 systemctl enable etcd （7）查看集群状态 $ systemctl status etcd # 以master为例 ● etcd.service - Etcd Server Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled; vendor preset: disabled) Main PID: 3193 (etcd) CGroup: /system.slice/etcd.service └─3193 /opt/etcd/bin/etcd --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/... Jan 31 20:38:43 k8s-m1 systemd[1]: Started Etcd Server. Hint: Some lines were ellipsized, use -l to show in full. $ ETCDCTL_API=3 /opt/etcd/bin/etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="endpoint health is healthy: successfully committed proposal: took = 32.800257ms is healthy: successfully committed proposal: took = 35.691684ms
安装Docker
下载地址： # 两个节点都需下载 wget 以下在所有节点操作，这里采用二进制安装，用yum安装也一样。（1）解压二进制包 tar zxvf docker-19.03.9.tgz mv docker/* /usr/bin （2）systemd管理docker cat > /usr/lib/systemd/system/docker.service << EOF [Unit] Description=Docker Application Container Engine Documentation= After=network-online.target firewalld.service Wants=network-online.target [Service] Type=notify ExecStart=/usr/bin/dockerd ExecReload=/bin/kill -s HUP $MAINPID LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity TimeoutStartSec=0 Delegate=yes KillMode=process Restart=on-failure StartLimitBurst=3 StartLimitInterval=60s [Install] WantedBy=multi-user.target EOF （3）创建配置文件 mkdir /etc/docker cat > /etc/docker/daemon.json << EOF { "registry-mirrors": [" } EOF # registry-mirrors 阿里云镜像加速器 （4）启动并设置开机启动 systemctl daemon-reload systemctl start docker systemctl enable docker # 确认启动正常，以master为例 $ systemctl status docker ● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) Docs: Main PID: 3258 (dockerd) CGroup: /system.slice/docker.service ├─3258 /usr/bin/dockerd └─3266 containerd --config /var/run/docker/containerd/containerd.toml --log-level info Jan 31 20:41:19 k8s-m1 systemd[1]: Started Docker Application Container Engine. Hint: Some lines were ellipsized, use -l to show in full.
部署Master Node
6.1 生成kube-apiserver证书
（1）自签证书颁发机构（CA） cd ~/TLS/k8s cat > ca-config.json << EOF { "signing": { "default": { "expiry": "87600h" }, "profiles": { "kubernetes": { "expiry": "87600h", "usages": [ "signing", "key encipherment", "server auth", "client auth" ] } } } } EOF cat > ca-csr.json << EOF { "CN": "kubernetes", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "Beijing", "ST": "Beijing", "O": "k8s", "OU": "System" } ] } EOF 生成证书： $ cfssl gencert -initca ca-csr.json | cfssljson -bare ca - 2022/01/31 20:48:05 [INFO] generating a new CA key and certificate from CSR 2022/01/31 20:48:05 [INFO] generate received request 2022/01/31 20:48:05 [INFO] received CSR 2022/01/31 20:48:05 [INFO] generating key: rsa-2048 2022/01/31 20:48:05 [INFO] encoded CSR 2022/01/31 20:48:05 [INFO] signed certificate with serial number 116190676499051077223279420283858349481970680819 $ ls *pem ca-key.pem ca.pem （2）使用自签CA签发kube-apiserver HTTPS证书创建证书申请文件： cat > server-csr.json << EOF { "CN": "kubernetes", "hosts": [ "10.0.0.1", "127.0.0.1", "192.168.1.147", "192.168.1.148", "kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster", "kubernetes.default.svc.cluster.local" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing", "O": "k8s", "OU": "System" } ] } EOF 注：上述文件hosts字段中IP为所有Master/LB/VIP IP，一个都不能少！为了方便后期扩容可以多写几个预留的IP。生成证书： $ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server 2022/01/31 20:48:29 [INFO] generate received request 2022/01/31 20:48:29 [INFO] received CSR 2022/01/31 20:48:29 [INFO] generating key: rsa-2048 2022/01/31 20:48:29 [INFO] encoded CSR 2022/01/31 20:48:29 [INFO] signed certificate with serial number 269930624984000095763889980982980819493992440261 2022/01/31 20:48:29 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for websites. For more information see the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum ( specifically, section 10.2.3 ("Information Requirements"). $ ls server*pem server-key.pem server.pem
6.2 从Github下载二进制文件
通过github查看下载链接， cd ~ wget 解压二进制包
mkdir -p /opt/kubernetes/{bin,cfg,ssl,logs} tar zxvf kubernetes-server-linux-amd64.tar.gz cd kubernetes/server/bin cp kube-apiserver kube-scheduler kube-controller-manager /opt/kubernetes/bin cp kubectl /usr/bin/
6.4 部署kube-apiserver
（1）创建配置文件 cat > /opt/kubernetes/cfg/kube-apiserver.conf << EOF KUBE_APISERVER_OPTS="--logtostderr=false \\ --v=2 \\ --log-dir=/opt/kubernetes/logs \\ --etcd-servers=\\ --bind-address=192.168.1.147 \\ --secure-port=6443 \\ --advertise-address=192.168.1.147 \\ --allow-privileged=true \\ --service-cluster-ip-range=10.0.0.0/24 \\ --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \\ --authorization-mode=RBAC,Node \\ --enable-bootstrap-token-auth=true \\ --token-auth-file=/opt/kubernetes/cfg/token.csv \\ --service-node-port-range=30000-32767 \\ --kubelet-client-certificate=/opt/kubernetes/ssl/server.pem \\ --kubelet-client-key=/opt/kubernetes/ssl/server-key.pem \\ --tls-cert-file=/opt/kubernetes/ssl/server.pem \\ --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \\ --client-ca-file=/opt/kubernetes/ssl/ca.pem \\ --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \\ --etcd-cafile=/opt/etcd/ssl/ca.pem \\ --etcd-certfile=/opt/etcd/ssl/server.pem \\ --etcd-keyfile=/opt/etcd/ssl/server-key.pem \\ --audit-log-maxage=30 \\ --audit-log-maxbackup=3 \\ --audit-log-maxsize=100 \\ --audit-log-path=/opt/kubernetes/logs/k8s-audit.log" EOF 注：上面两个\ \ 第一个是转义符，第二个是换行符，使用转义符是为了使用EOF保留换行符。 –logtostderr：启用日志 —v：日志等级 –log-dir：日志目录 –etcd-servers：etcd集群地址 –bind-address：监听地址 –secure-port： –advertise-address：集群通告地址 –allow-privileged：启用授权 –service-cluster-ip-range：Service虚拟IP地址段 –enable-admission-plugins：准入控制模块 –authorization-mode：认证授权，启用RBAC授权和节点自管理 –enable-bootstrap-token-auth：启用TLS bootstrap机制 –token-auth-file：bootstrap token文件 –service-node-port-range：Service nodeport类型默认分配端口范围 –kubelet-client-xxx：apiserver访问kubelet客户端证书 –tls-xxx-file：apiserver –etcd-xxxfile：连接Etcd集群证书 –audit-log-xxx：审计日志 （2）拷贝刚才生成的证书把刚才生成的证书拷贝到配置文件中的路径： cp /TLS/k8s/ca*pem /TLS/k8s/server*pem /opt/kubernetes/ssl/
（3）启用 TLS Bootstrapping 机制TLS Bootstraping：Master apiserver启用TLS认证后，Node节点kubelet和kube-proxy要与kube-apiserver进行通信，必须使用CA签发的有效证书才可以，当Node节点很多时，这种客户端证书颁发需要大量工作，同样也会增加集群扩展复杂度。为了简化流程，Kubernetes引入了TLSbootstraping机制来自动颁发客户端证书，kubelet会以一个低权限用户自动向apiserver申请证书，kubelet的证书由apiserver动态签署。所以强烈建议在Node上使用这种方式，目前主要用于kubelet，kube-proxy还是由我们统一颁发一个证书。TLS bootstraping 工作流程：
创建上述配置文件中token文件： cat > /opt/kubernetes/cfg/token.csv << EOF c47ffb939f5ca36231d9e3121a252940,kubelet-bootstrap,10001,"system:node-bootstrapper" EOF 格式：token，用户名，UID，用户组token也可自行生成替换： head -c 16 /dev/urandom | od -An -t x | tr -d ' ' （4）systemd管理apiserver cat > /usr/lib/systemd/system/kube-apiserver.service << EOF [Unit] Description=Kubernetes API Server Documentation= [Service] EnvironmentFile=/opt/kubernetes/cfg/kube-apiserver.conf ExecStart=/opt/kubernetes/bin/kube-apiserver \$KUBE_APISERVER_OPTS Restart=on-failure [Install] WantedBy=multi-user.target EOF （5）启动并设置开机启动 systemctl daemon-reload systemctl start kube-apiserver systemctl enable kube-apiserver # 确认状态运行正常 $ systemctl status kube-apiserver ● kube-apiserver.service - Kubernetes API Server Loaded: loaded (/usr/lib/systemd/system/kube-apiserver.service; enabled; vendor preset: disabled) Docs: Main PID: 3519 (kube-apiserver) CGroup: /system.slice/kube-apiserver.service └─3519 /opt/kubernetes/bin/kube-apiserver --logtostderr=false --v=2 --log-dir=/opt/kubernetes/logs --etcd-servers= Jan 31 20:50:15 k8s-m1 systemd[1]: Started Kubernetes API Server. Jan 31 20:50:15 k8s-m1 systemd[1]: Starting Kubernetes API Server... Jan 31 20:50:21 k8s-m1 kube-apiserver[3519]: E0131 20:50:21.270473 3519 controller.go:152] Unable to remove old endpoints f...orMsg: Hint: Some lines were ellipsized, use -l to show in full. （6）授权kubelet-bootstrap用户允许请求证书 kubectl create clusterrolebinding kubelet-bootstrap \ --clusterrole=system:node-bootstrapper \ --user=kubelet-bootstrap
6.5 部署kube-controller-manager
（1）创建配置文件 cat > /opt/kubernetes/cfg/kube-controller-manager.conf << EOF KUBE_CONTROLLER_MANAGER_OPTS="--logtostderr=false \\ --v=2 \\ --log-dir=/opt/kubernetes/logs \\ --leader-elect=true \\ --master=127.0.0.1:8080 \\ --bind-address=127.0.0.1 \\ --allocate-node-cidrs=true \\ --cluster-cidr=10.244.0.0/16 \\ --service-cluster-ip-range=10.0.0.0/24 \\ --cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \\ --cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \\ --root-ca-file=/opt/kubernetes/ssl/ca.pem \\ --service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \\ --experimental-cluster-signing-duration=87600h0m0s" EOF –master：通过本地非安全本地端口8080连接apiserver。–leader-elect：当该组件启动多个时，自动选举（HA）–cluster-signing-cert-file/–cluster-signing-key-file：自动为kubelet颁发证书的CA，与apiserver保持一致（2）systemd管理controller-manager cat > /usr/lib/systemd/system/kube-controller-manager.service << EOF [Unit] Description=Kubernetes Controller Manager Documentation= [Service] EnvironmentFile=/opt/kubernetes/cfg/kube-controller-manager.conf ExecStart=/opt/kubernetes/bin/kube-controller-manager \$KUBE_CONTROLLER_MANAGER_OPTS Restart=on-failure [Install] WantedBy=multi-user.target EOF （3）启动并设置开机启动 systemctl daemon-reload systemctl start kube-controller-manager systemctl enable kube-controller-manager # 确认状态运行正常 $ systemctl status kube-controller-manager ● kube-controller-manager.service - Kubernetes Controller Manager Loaded: loaded (/usr/lib/systemd/system/kube-controller-manager.service; enabled; vendor preset: disabled) Docs: Main PID: 3574 (kube-controller) CGroup: /system.slice/kube-controller-manager.service └─3574 /opt/kubernetes/bin/kube-controller-manager --logtostderr=false --v=2 --log-dir=/opt/kubernetes/logs --leader-elec... Jan 31 20:51:00 k8s-m1 systemd[1]: Started Kubernetes Controller Manager. Jan 31 20:51:00 k8s-m1 systemd[1]: Starting Kubernetes Controller Manager... Jan 31 20:51:00 k8s-m1 kube-controller-manager[3574]: Flag --experimental-cluster-signing-duration has been deprecated, use --...ration Hint: Some lines were ellipsized, use -l to show in full.
6.6 部署kube-scheduler
（1）创建配置文件 cat > /opt/kubernetes/cfg/kube-scheduler.conf << EOF KUBE_SCHEDULER_OPTS="--logtostderr=false \ --v=2 \ --log-dir=/opt/kubernetes/logs \ --leader-elect \ --master=127.0.0.1:8080 \ --bind-address=127.0.0.1" EOF –master：通过本地非安全本地端口8080连接apiserver。–leader-elect：当该组件启动多个时，自动选举（HA）（2）systemd管理scheduler cat > /usr/lib/systemd/system/kube-scheduler.service << EOF [Unit] Description=Kubernetes Scheduler Documentation= [Service] EnvironmentFile=/opt/kubernetes/cfg/kube-scheduler.conf ExecStart=/opt/kubernetes/bin/kube-scheduler \$KUBE_SCHEDULER_OPTS Restart=on-failure [Install] WantedBy=multi-user.target EOF （3）启动并设置开机启动 systemctl daemon-reload systemctl start kube-scheduler systemctl enable kube-scheduler # 确认状态运行正常 $ systemctl status kube-scheduler ● kube-scheduler.service - Kubernetes Scheduler Loaded: loaded (/usr/lib/systemd/system/kube-scheduler.service; enabled; vendor preset: disabled) Docs: Main PID: 3622 (kube-scheduler) CGroup: /system.slice/kube-scheduler.service └─3622 /opt/kubernetes/bin/kube-scheduler --logtostderr=false --v=2 --log-dir=/opt/kubernetes/logs --leader-elect --maste... Jan 31 20:51:40 k8s-m1 systemd[1]: Started Kubernetes Scheduler. Jan 31 20:51:40 k8s-m1 systemd[1]: Starting Kubernetes Scheduler... Jan 31 20:51:41 k8s-m1 kube-scheduler[3622]: I0131 20:51:41.106830 3622 registry.go:173] Registering SelectorSpread plugin Jan 31 20:51:41 k8s-m1 kube-scheduler[3622]: I0131 20:51:41.106928 3622 registry.go:173] Registering SelectorSpread plugin （4）查看集群状态生成kubectl连接集群的证书： cat > ~/TLS/k8s/admin-csr.json < 部署Worker Node 说明：下面还是在Master Node上操作，即同时作为Worker Node 7.1 创建工作目录并拷贝二进制文件 （1）在所有worker node创建工作目录： cd ~ mkdir -p /opt/kubernetes/{bin,cfg,ssl,logs} # 之前master上已创建，无需再创建 从master节点拷贝： cd kubernetes/server/bin cp kubelet kube-proxy /opt/kubernetes/bin # 本地拷贝 7.2 部署kubelet （1）创建配置文件 cat > /opt/kubernetes/cfg/kubelet.conf << EOF KUBELET_OPTS="--logtostderr=false \\ --v=2 \\ --log-dir=/opt/kubernetes/logs \\ --hostname-override=k8s-m1 \\ --network-plugin=cni \\ --kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \\ --bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \\ --config=/opt/kubernetes/cfg/kubelet-config.yml \\ --cert-dir=/opt/kubernetes/ssl \\ --pod-infra-container-image=lizhenliang/pause-amd64:3.0" EOF –hostname-override：显示名称，集群中唯一–network-plugin：启用CNI–kubeconfig：空路径，会自动生成，后面用于连接apiserver–bootstrap-kubeconfig：首次启动向apiserver申请证书–config：配置参数文件–cert-dir：kubelet证书生成目录–pod-infra-container-image：管理Pod网络容器的镜像（2）配置参数文件 cat > /opt/kubernetes/cfg/kubelet-config.yml << EOF kind: KubeletConfiguration apiVersion: kubelet.config.k8s.io/v1beta1 address: 0.0.0.0 port: 10250 readOnlyPort: 10255 cgroupDriver: cgroupfs clusterDNS: 10.0.0.2 clusterDomain: cluster.local failSwapOn: false authentication: anonymous: enabled: false webhook: cacheTTL: 2m0s enabled: true x509: clientCAFile: /opt/kubernetes/ssl/ca.pem authorization: mode: Webhook webhook: cacheAuthorizedTTL: 5m0s cacheUnauthorizedTTL: 30s evictionHard: imagefs.available: 15% memory.available: 100Mi nodefs.available: 10% nodefs.inodesFree: 5% maxOpenFiles: 1000000 maxPods: 110 EOF （3）生成bootstrap.kubeconfig文件 cd ~ KUBE_APISERVER="# apiserver IP:PORT TOKEN="c47ffb939f5ca36231d9e3121a252940" # 与token.csv里保持一致 # 生成 kubelet bootstrap kubeconfig 配置文件 kubectl config set-cluster kubernetes \ --certificate-authority=/opt/kubernetes/ssl/ca.pem \ --embed-certs=true \ --server=${KUBE_APISERVER} \ --kubeconfig=bootstrap.kubeconfig kubectl config set-credentials "kubelet-bootstrap" \ --token=${TOKEN} \ --kubeconfig=bootstrap.kubeconfig kubectl config set-context default \ --cluster=kubernetes \ --user="kubelet-bootstrap" \ --kubeconfig=bootstrap.kubeconfig kubectl config use-context default --kubeconfig=bootstrap.kubeconfig 拷贝到配置文件路径： cp bootstrap.kubeconfig /opt/kubernetes/cfg （4）systemd管理kubelet cat > /usr/lib/systemd/system/kubelet.service << EOF [Unit] Description=Kubernetes Kubelet After=docker.service [Service] EnvironmentFile=/opt/kubernetes/cfg/kubelet.conf ExecStart=/opt/kubernetes/bin/kubelet \$KUBELET_OPTS Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF （5）启动并设置开机启动 systemctl daemon-reload systemctl start kubelet systemctl enable kubelet # 确认状态正常运行 $ systemctl status kubelet ● kubelet.service - Kubernetes Kubelet Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled) Main PID: 3744 (kubelet) CGroup: /system.slice/kubelet.service └─3744 /opt/kubernetes/bin/kubelet --logtostderr=false --v=2 --log-dir=/opt/kubernetes/logs --hostname-override=k8s-m1 --... Jan 31 20:57:36 k8s-m1 systemd[1]: Started Kubernetes Kubelet. Jan 31 20:57:36 k8s-m1 systemd[1]: Starting Kubernetes Kubelet... 7.3 批准kubelet证书申请并加入集群 # 查看kubelet证书请求 $ kubectl get csr NAME AGE SIGNERNAME REQUESTOR CONDITION node-csr-mJUDZWs1ZgjeMxO5cKHmN93miRvt-aVFHE25fmcfEC0 20s kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Pending # 批准申请 $ kubectl certificate approve node-csr-mJUDZWs1ZgjeMxO5cKHmN93miRvt-aVFHE25fmcfEC0 certificatesigningrequest.certificates.k8s.io/node-csr-mJUDZWs1ZgjeMxO5cKHmN93miRvt-aVFHE25fmcfEC0 approved # 查看节点 $ kubectl get node NAME STATUS ROLES AGE VERSION k8s-m1 NotReady 7.4 部署kube-proxy （1）创建配置文件 cat > /opt/kubernetes/cfg/kube-proxy.conf << EOF KUBE_PROXY_OPTS="--logtostderr=false \\ --v=2 \\ --log-dir=/opt/kubernetes/logs \\ --config=/opt/kubernetes/cfg/kube-proxy-config.yml" EOF （2）配置参数文件 cat > /opt/kubernetes/cfg/kube-proxy-config.yml << EOF kind: KubeProxyConfiguration apiVersion: kubeproxy.config.k8s.io/v1alpha1 bindAddress: 0.0.0.0 metricsBindAddress: 0.0.0.0:10249 clientConnection: kubeconfig: /opt/kubernetes/cfg/kube-proxy.kubeconfig hostnameOverride: k8s-m1 clusterCIDR: 10.0.0.0/24 EOF （3）生成kube-proxy.kubeconfig文件生成kube-proxy证书： # 切换工作目录 cd TLS/k8s # 创建证书请求文件 cat > kube-proxy-csr.json << EOF { "CN": "system:kube-proxy", "hosts": [], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing", "O": "k8s", "OU": "System" } ] } EOF # 生成证书 $ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy 2022/01/31 20:59:20 [INFO] generate received request 2022/01/31 20:59:20 [INFO] received CSR 2022/01/31 20:59:20 [INFO] generating key: rsa-2048 2022/01/31 20:59:20 [INFO] encoded CSR 2022/01/31 20:59:20 [INFO] signed certificate with serial number 295992655043122107605462071788881029347482203196 2022/01/31 20:59:20 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for websites. For more information see the Baseline Requirements for the Issuance and Management of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum ( specifically, section 10.2.3 ("Information Requirements"). $ ls kube-proxy*pem kube-proxy-key.pem kube-proxy.pem 生成kubeconfig文件： KUBE_APISERVER=" kubectl config set-cluster kubernetes \ --certificate-authority=/opt/kubernetes/ssl/ca.pem \ --embed-certs=true \ --server=${KUBE_APISERVER} \ --kubeconfig=kube-proxy.kubeconfig kubectl config set-credentials kube-proxy \ --client-certificate=./kube-proxy.pem \ --client-key=./kube-proxy-key.pem \ --embed-certs=true \ --kubeconfig=kube-proxy.kubeconfig kubectl config set-context default \ --cluster=kubernetes \ --user=kube-proxy \ --kubeconfig=kube-proxy.kubeconfig kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig 拷贝到配置文件指定路径： cp kube-proxy.kubeconfig /opt/kubernetes/cfg/ （4）systemd管理kube-proxy cat > /usr/lib/systemd/system/kube-proxy.service << EOF [Unit] Description=Kubernetes Proxy After=network.target [Service] EnvironmentFile=/opt/kubernetes/cfg/kube-proxy.conf ExecStart=/opt/kubernetes/bin/kube-proxy \$KUBE_PROXY_OPTS Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF （5）启动并设置开机启动 systemctl daemon-reload systemctl start kube-proxy systemctl enable kube-proxy # 确认状态运行正常 $ systemctl status kube-proxy ● kube-proxy.service - Kubernetes Proxy Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled) Main PID: 4373 (kube-proxy) CGroup: /system.slice/kube-proxy.service └─4373 /opt/kubernetes/bin/kube-proxy --logtostderr=false --v=2 --log-dir=/opt/kubernetes/logs --config=/opt/kubernetes/c... Jan 31 21:00:25 k8s-m1 systemd[1]: Started Kubernetes Proxy. Jan 31 21:00:25 k8s-m1 systemd[1]: Starting Kubernetes Proxy... 7.5 部署CNI网络 （1）准备好CNI二进制文件： cd ~ wget 解压二进制包并移动到默认工作目录： mkdir -p /opt/cni/bin tar zxvf cni-plugins-linux-amd64-v0.8.6.tgz -C /opt/cni/bin 部署CNI网络，创建kube-flannel.yml： cat > kube-flannel.yml << EOF --- apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: psp.flannel.unprivileged annotations: seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default spec: privileged: false volumes: configMapsecretemptyDirhostPath allowedHostPaths: pathPrefix: "/etc/cni/net.d"pathPrefix: "/etc/kube-flannel"pathPrefix: "/run/flannel" readOnlyRootFilesystem: false # Users and groups runAsUser: rule: RunAsAny supplementalGroups: rule: RunAsAny fsGroup: rule: RunAsAny # Privilege Escalation allowPrivilegeEscalation: false defaultAllowPrivilegeEscalation: false # Capabilities allowedCapabilities: ['NET_ADMIN'] defaultAddCapabilities: [] requiredDropCapabilities: [] # Host namespaces hostPID: false hostIPC: false hostNetwork: true hostPorts: min: 0 max: 65535 # SELinux seLinux: # SELinux is unused in CaaSP rule: 'RunAsAny' --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: flannel rules: apiGroups: ['extensions'] resources: ['podsecuritypolicies'] verbs: ['use'] resourceNames: ['psp.flannel.unprivileged'] apiGroups: "" resources: pods verbs: get apiGroups: "" resources: nodes verbs: listwatch apiGroups: "" resources: nodes/status verbs: patch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: flannel roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: flannel subjects: kind: ServiceAccount name: flannel namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: flannel namespace: kube-system --- kind: ConfigMap apiVersion: v1 metadata: name: kube-flannel-cfg namespace: kube-system labels: tier: node app: flannel data: cni-conf.json: | { "name": "cbr0", "cniVersion": "0.3.1", "plugins": [ { "type": "flannel", "delegate": { "hairpinMode": true, "isDefaultGateway": true } }, { "type": "portmap", "capabilities": { "portMappings": true } } ] } net-conf.json: | { "Network": "10.244.0.0/16", "Backend": { "Type": "vxlan" } } --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-amd64 namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: matchExpressions: key: beta.kubernetes.io/os operator: In values: linuxkey: beta.kubernetes.io/arch operator: In values: amd64 hostNetwork: true tolerations: operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: name: install-cni image: quay.io/coreos/flannel:v0.11.0-amd64 command: cp args: -f/etc/kube-flannel/cni-conf.json/etc/cni/net.d/10-flannel.conflist volumeMounts: name: cni mountPath: /etc/cni/net.d name: flannel-cfg mountPath: /etc/kube-flannel/ containers: name: kube-flannel image: quay.io/coreos/flannel:v0.11.0-amd64 command: /opt/bin/flanneld args: --ip-masq--kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN"] env: name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: name: run mountPath: /run/flannel name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: name: run hostPath: path: /run/flannel name: cni hostPath: path: /etc/cni/net.d name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-arm64 namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: matchExpressions:key: beta.kubernetes.io/os operator: In values: linuxkey: beta.kubernetes.io/arch operator: In values: arm64 hostNetwork: true tolerations: operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: name: install-cni image: quay.io/coreos/flannel:v0.11.0-arm64 command: cp args: -f/etc/kube-flannel/cni-conf.json/etc/cni/net.d/10-flannel.conflist volumeMounts: name: cni mountPath: /etc/cni/net.d name: flannel-cfg mountPath: /etc/kube-flannel/ containers: name: kube-flannel image: quay.io/coreos/flannel:v0.11.0-arm64 command: /opt/bin/flanneld args: --ip-masq--kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN"] env: name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: name: run mountPath: /run/flannel name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: name: run hostPath: path: /run/flannel name: cni hostPath: path: /etc/cni/net.d name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-arm namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: matchExpressions:key: beta.kubernetes.io/os operator: In values: linuxkey: beta.kubernetes.io/arch operator: In values: arm hostNetwork: true tolerations: operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: name: install-cni image: quay.io/coreos/flannel:v0.11.0-arm command: cp args: -f/etc/kube-flannel/cni-conf.json/etc/cni/net.d/10-flannel.conflist volumeMounts: name: cni mountPath: /etc/cni/net.d name: flannel-cfg mountPath: /etc/kube-flannel/ containers: name: kube-flannel image: quay.io/coreos/flannel:v0.11.0-arm command: /opt/bin/flanneld args: --ip-masq--kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN"] env: name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: name: run mountPath: /run/flannel name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: name: run hostPath: path: /run/flannel name: cni hostPath: path: /etc/cni/net.d name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-ppc64le namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: matchExpressions:key: beta.kubernetes.io/os operator: In values: linuxkey: beta.kubernetes.io/arch operator: In values: ppc64le hostNetwork: true tolerations: operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: name: install-cni image: quay.io/coreos/flannel:v0.11.0-ppc64le command: cp args: -f/etc/kube-flannel/cni-conf.json/etc/cni/net.d/10-flannel.conflist volumeMounts: name: cni mountPath: /etc/cni/net.d name: flannel-cfg mountPath: /etc/kube-flannel/ containers: name: kube-flannel image: quay.io/coreos/flannel:v0.11.0-ppc64le command: /opt/bin/flanneld args: --ip-masq--kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN"] env: name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: name: run mountPath: /run/flannel name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: name: run hostPath: path: /run/flannel name: cni hostPath: path: /etc/cni/net.d name: flannel-cfg configMap: name: kube-flannel-cfg --- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds-s390x namespace: kube-system labels: tier: node app: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier: node app: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: matchExpressions:key: beta.kubernetes.io/os operator: In values: linuxkey: beta.kubernetes.io/arch operator: In values: s390x hostNetwork: true tolerations: operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: name: install-cni image: quay.io/coreos/flannel:v0.11.0-s390x command: cp args: -f/etc/kube-flannel/cni-conf.json/etc/cni/net.d/10-flannel.conflist volumeMounts: name: cni mountPath: /etc/cni/net.d name: flannel-cfg mountPath: /etc/kube-flannel/ containers: name: kube-flannel image: quay.io/coreos/flannel:v0.11.0-s390x command: /opt/bin/flanneld args: --ip-masq--kube-subnet-mgr resources: requests: cpu: "100m" memory: "50Mi" limits: cpu: "100m" memory: "50Mi" securityContext: privileged: false capabilities: add: ["NET_ADMIN"] env: name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: name: run mountPath: /run/flannel name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: name: run hostPath: path: /run/flannel name: cni hostPath: path: /etc/cni/net.d name: flannel-cfg configMap: name: kube-flannel-cfg EOF $ kubectl apply -f kube-flannel.yml podsecuritypolicy.policy/psp.flannel.unprivileged created Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole clusterrole.rbac.authorization.k8s.io/flannel created Warning: rbac.authorization.k8s.io/v1beta1 ClusterRoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRoleBinding clusterrolebinding.rbac.authorization.k8s.io/flannel created serviceaccount/flannel created configmap/kube-flannel-cfg created daemonset.apps/kube-flannel-ds-amd64 created daemonset.apps/kube-flannel-ds-arm64 created daemonset.apps/kube-flannel-ds-arm created daemonset.apps/kube-flannel-ds-ppc64le created daemonset.apps/kube-flannel-ds-s390x created $ kubectl get pods -n kube-system # 刚敲完命令为init状态，需要等待会儿 NAME READY STATUS RESTARTS AGE kube-flannel-ds-amd64-7k86t 1/1 Running 0 58s $ kubectl get node NAME STATUS ROLES AGE VERSION k8s-m1 Ready 7.6 授权apiserver访问kubelet cat > apiserver-to-kubelet-rbac.yaml << EOF apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: annotations: rbac.authorization.kubernetes.io/autoupdate: "true" labels: kubernetes.io/bootstrapping: rbac-defaults name: system:kube-apiserver-to-kubelet rules: apiGroups: "" resources: nodes/proxynodes/statsnodes/lognodes/specnodes/metricspods/log verbs: "*" --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:kube-apiserver namespace: "" roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:kube-apiserver-to-kubelet subjects: apiGroup: rbac.authorization.k8s.io kind: User name: kubernetes EOF $ kubectl apply -f apiserver-to-kubelet-rbac.yaml clusterrole.rbac.authorization.k8s.io/system:kube-apiserver-to-kubelet created clusterrolebinding.rbac.authorization.k8s.io/system:kube-apiserver created 7.7 新增加Worker Node （1）拷贝已部署好的Node相关文件到新节点在master节点将Worker Node涉及文件拷贝到新节点192.168.1.148 scp -r /opt/kubernetes root@192.168.1.148:/opt/ scp -r /usr/lib/systemd/system/{kubelet,kube-proxy}.service root@192.168.1.148:/usr/lib/systemd/system scp -r /opt/cni/ root@192.168.1.148:/opt/ scp /opt/kubernetes/ssl/ca.pem root@192.168.1.148:/opt/kubernetes/ssl （2）删除kubelet证书和kubeconfig文件（在k8s-n1上执行） rm /opt/kubernetes/cfg/kubelet.kubeconfig rm -f /opt/kubernetes/ssl/kubelet* 注：这几个文件是证书申请审批后自动生成的，每个Node不同，必须删除重新生成。（3）修改主机名（在k8s-n1上执行） vi /opt/kubernetes/cfg/kubelet.conf --hostname-override=k8s-n1 vi /opt/kubernetes/cfg/kube-proxy-config.yml hostnameOverride: k8s-n1 （4）启动并设置开机启动（在k8s-n1上执行） systemctl daemon-reload systemctl start kubelet systemctl enable kubelet systemctl start kube-proxy systemctl enable kube-proxy # 确认状态正常 $ systemctl status kubelet ● kubelet.service - Kubernetes Kubelet Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled) Main PID: 3121 (kubelet) CGroup: /system.slice/kubelet.service └─3121 /opt/kubernetes/bin/kubelet --logtostderr=false --v=2 --log-dir=/opt/kubernetes/logs --hostname-override=k8s-n1 --... Jan 31 21:17:46 k8s-n1 systemd[1]: Started Kubernetes Kubelet. Jan 31 21:17:46 k8s-n1 systemd[1]: Starting Kubernetes Kubelet... $ systemctl status kube-proxy ● kube-proxy.service - Kubernetes Proxy Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled) Main PID: 3153 (kube-proxy) CGroup: /system.slice/kube-proxy.service └─3153 /opt/kubernetes/bin/kube-proxy --logtostderr=false --v=2 --log-dir=/opt/kubernetes/logs --config=/opt/kubernetes/c... Jan 31 21:17:52 k8s-n1 systemd[1]: Started Kubernetes Proxy. Jan 31 21:17:52 k8s-n1 systemd[1]: Starting Kubernetes Proxy... Jan 31 21:17:52 k8s-n1 kube-proxy[3153]: E0131 21:17:52.550194 3153 node.go:125] Failed to retrieve node info: nodes "k8s-...t found Jan 31 21:17:53 k8s-n1 kube-proxy[3153]: E0131 21:17:53.747923 3153 node.go:125] Failed to retrieve node info: nodes "k8s-...t found Jan 31 21:17:55 k8s-n1 kube-proxy[3153]: E0131 21:17:55.805336 3153 node.go:125] Failed to retrieve node info: nodes "k8s-...t found Jan 31 21:18:00 k8s-n1 kube-proxy[3153]: E0131 21:18:00.123071 3153 node.go:125] Failed to retrieve node info: nodes "k8s-...t found Jan 31 21:18:09 k8s-n1 kube-proxy[3153]: E0131 21:18:09.223578 3153 node.go:125] Failed to retrieve node info: nodes "k8s-...t found Hint: Some lines were ellipsized, use -l to show in full. （5）在Master上批准新Node kubelet证书申请 $ kubectl get csr NAME AGE SIGNERNAME REQUESTOR CONDITION node-csr-R17zXNtPU2ozsmqePPT3LnsV1KJs2wh2OCmpdHQS-r4 46s kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Pending node-csr-mJUDZWs1ZgjeMxO5cKHmN93miRvt-aVFHE25fmcfEC0 20m kubernetes.io/kube-apiserver-client-kubelet kubelet-bootstrap Approved,Issued $ kubectl certificate approve node-csr-R17zXNtPU2ozsmqePPT3LnsV1KJs2wh2OCmpdHQS-r4 certificatesigningrequest.certificates.k8s.io/node-csr-R17zXNtPU2ozsmqePPT3LnsV1KJs2wh2OCmpdHQS-r4 approved （6）查看Node状态 $ kubectl get node # 敲完命令后要等待下，k8s-n1才会Ready NAME STATUS ROLES AGE VERSION k8s-m1 Ready 测试kubernetes集群 在Kubernetes集群中创建一个pod，验证是否正常运行： $ kubectl create deployment nginx --image=nginx deployment.apps/nginx created $ kubectl expose deployment nginx --port=80 --type=NodePort service/nginx exposed $ kubectl get pod,svc NAME READY STATUS RESTARTS AGE pod/nginx-6799fc88d8-qfpwf 0/1 ContainerCreating 0 10s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.0.0.1
版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。
相关文章
发表评论
暂时没有评论，来抢沙发吧~