kubeadm安装k8s 1.23.5

kubeadm安装k8s 1.23.5

一. 环境准备

1.1 配置yum阿里源

yum -y install wget mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.bak wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo yum clean all yum makecache

1.2 关闭防火墙

# 查看防火墙状态 firewall-cmd --state # 临时停止防火墙 systemctl stop firewalld.service # 禁止防火墙开机启动 systemctl disable firewalld.service

1.3 关闭selinux

# 查看selinux状态 getenforce # 临时关闭selinux setenforce 0 # 永久关闭selinux sed -i 's/^ *SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

1.4 关闭swap

# 临时关闭swap swapoff -a # 永久关闭swap sed -i.bak '/swap/s/^/#/' /etc/fstab # 查看 free -g

1.5 调整内核参数及模块

加载所需内核模块

cat < /etc/modules-load.d/k8s.conf br_netfilter EOF cat < /etc/modules-load.d/containerd.conf overlay br_netfilter EOF modprobe overlay modprobe br_netfilter

设置必需的 sysctl 参数，允许iptables检查桥接流量，这些参数在重新启动后仍然存在

cat < /etc/sysctl.d/99-kubernetes-cri.conf net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 EOF # 应用 sysctl 参数而无需重新启动 sudo sysctl --system

1.6 开启ipvs

不开启ipvs将会使用iptables进行数据包转发，但是效率低，所以推荐开通ipvs，使用

cat < /etc/sysconfig/modules/ipvs.modules #!/bin/bash modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack_ipv4 EOF # 加载模块 chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4 # 安装了ipset软件包 yum install ipset -y # 安装管理工具ipvsadm yum install ipvsadm -y

1.7 同步服务器时间

yum install chrony -y systemctl enable chronyd systemctl start chronyd [root@master ~]# chronyc sources 210 Number of sources = 4 MS Name/IP address Stratum Poll Reach LastRx Last sample =============================================================================== ^- ntp.wdc1.us.leaseweb.net 2 9 201 329 -8125us[-8125us] +/- 264ms ^- ntp5.flashdance.cx 2 9 373 189 -43ms[ -43ms] +/- 223ms ^+ time.cloudflare.com 3 8 377 197 +38ms[ +38ms] +/- 121ms ^* 119.28.183.184 2 8 155 30m -8460us[ -13ms] +/- 67ms [root@master ~]# date 2022年 03月 26日 星期六 15:11:32 CST

1.8 安装containerd

yum install -y yum-utils device-mapper-persistent-data lvm2 yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo # 查看最新版本 yum list containerd --showduplicates | sort -r yum install containerd -y # 安装了`containerd.io-1.5.11-3.1.el7.x86_64` containerd config default > /etc/containerd/config.toml systemctl start containerd systemctl enable containerd

配置

# 修改cgroups为systemd sed -i 's#SystemdCgroup = false#SystemdCgroup = true#' /etc/containerd/config.toml # 修改基础设施镜像 sed -i 's#sandbox_image = "k8s.gcr.io/pause:3.5"#sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.6"#' /etc/containerd/config.toml systemctl daemon-reload systemctl restart containerd

安装 CRI 客户端 crictl选择版本 https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.23.0/crictl-v1.23.0-linux-amd64.tar.gz tar zxvf crictl-v1.23.0-linux-amd64.tar.gz -C /usr/local/bin cat < /etc/crictl.yaml runtime-endpoint: unix:///run/containerd/containerd.sock image-endpoint: unix:///run/containerd/containerd.sock timeout: 10 debug: false EOF # 验证是否可用 crictl pull nginx:alpine crictl images crictl rmi nginx:alpine

1.9 修改hostname和hosts

修改hostname

# master节点 hostnamectl set-hostname master # node1节点 hostnamectl set-hostname node1 # node2节点 hostnamectl set-hostname node2

添加hosts

cat < /etc/hosts 192.168.4.27 master 192.168.4.28 node1 192.168.4.29 node2 EOF

二. 安装k8s

2.1 安装 kubelet、kubeadm、kubectl

添加kubernetes源

cat < /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF

然后安装 kubeadm、kubelet、kubectl

# 查看版本，最新版 1.23.5-0 yum list kubeadm --showduplicates | sort -r yum install -y kubelet-1.23.5-0 kubectl-1.23.5-0 kubeadm-1.23.5-0 [root@master ~]# kubeadm version kubeadm version: &version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:57:37Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"linux/amd64"}

修改kubelet配置

cat < /etc/sysconfig/kubelet KUBELET_KUBEADM_ARGS="--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock" EOF

启动kubelet服务，并设置开机自启

systemctl start kubelet systemctl enable kubelet

2.2 初始化k8s集群

2.2.1 master节点（二选一）

1. 通过配置文件初始化：

kubeadm config print init-defaults > kubeadm.yaml 修改为 cat < kubeadm.yaml apiVersion: kubeadm.k8s.io/v1beta3 bootstrapTokens: - groups: - system:bootstrappers:kubeadm:default-node-token token: abcdef.0123456789abcdef ttl: 24h0m0s usages: - signing - authentication kind: InitConfiguration localAPIEndpoint: advertiseAddress: 192.168.4.27 # apiserver 节点内网IP bindPort: 6443 nodeRegistration: criSocket: /run/containerd/containerd.sock # 修改为containerd imagePullPolicy: IfNotPresent name: master taints: - effect: NoSchedule key: node-role.kubernetes.io/master --- apiServer: timeoutForControlPlane: 4m0s apiVersion: kubeadm.k8s.io/v1beta3 certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controllerManager: {} dns: type: CoreDNS # dns类型 type: CoreDNS etcd: local: dataDir: /var/lib/etcd imageRepository: registry.aliyuncs.com/google_containers # 修改这个镜像能下载 kind: ClusterConfiguration kubernetesVersion: 1.23.5 # k8s版本 networking: dnsDomain: cluster.local podSubnet: 10.244.0.0/16 serviceSubnet: 10.96.0.0/12 scheduler: {} --- apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration mode: ipvs # kube-proxy 模式 EOF kubeadm init --config kubeadm.yaml

2. 直接初始化：

kube-proxy 模式是 iptables，可以通过kubectl edit configmap kube-proxy -n kube-system修改

kubeadm init \ --kubernetes-version v1.23.5 \ --apiserver-advertise-address 192.168.4.27 \ --control-plane-endpoint master \ --image-repository registry.aliyuncs.com/google_containers \ --pod-network-cidr 10.244.0.0/16 \ --cri-socket /run/containerd/containerd.sock

--kubernetes-version：指定的版本 --apiserver-advertise-address：K8S主节点的地址 --pod-network-cidr：pod的网络IP范围

如果您的网络运行在192.168..，需要将 pod-network-cidr 设置为10.0.0.0/16；如果您的网络是10.0..使用192.168.0.0/16，此时使用calico网络（如果设置错了部署calico 网络插件后coredns也运行不起来，会报错coredns Failed to list *v1.Endpoints，该错误解决办法参考-p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config

2.2.2 node节点

kubeadm join master:6443 --token f6e3hv.uk6ctfgehstt92jw \ --discovery-token-ca-cert-hash sha256:9962caed607e31de7b93732347c1ac681f216c290e6b35f91f3f5d67cd12cbcf

2.3 安装Calico网络插件（master节点执行）

mkdir -p /root/i && cd /root/i # 下载 curl -o /root/i/calico.yaml 查看一下版本`v3.22.2`，如果不是替换不生效 # 修改镜像 sed -i 's#docker.io/calico/cni:v3.22.2#registry.cn-shanghai.aliyuncs.com/wanfei/cni:v3.22.2#' /root/i/calico.yaml sed -i 's#docker.io/calico/pod2daemon-flexvol:v3.22.2#registry.cn-shanghai.aliyuncs.com/wanfei/pod2daemon-flexvol:v3.22.2#' /root/i/calico.yaml sed -i 's#docker.io/calico/node:v3.22.2#registry.cn-shanghai.aliyuncs.com/wanfei/node:v3.22.2#' /root/i/calico.yaml sed -i 's#docker.io/calico/kube-controllers:v3.22.2#registry.cn-shanghai.aliyuncs.com/wanfei/kube-controllers:v3.22.2#' /root/i/calico.yaml # 执行 kubectl apply -f /root/i/calico.yaml

等几分钟

[root@master i]# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE calico-kube-controllers-57845f44bb-tpvbr 1/1 Running 0 79s calico-node-fpfxj 1/1 Running 0 79s calico-node-qcvqx 1/1 Running 0 79s calico-node-r4gsf 1/1 Running 0 79s coredns-6d8c4cb4d-7bclr 1/1 Running 0 29m coredns-6d8c4cb4d-djwxf 1/1 Running 0 29m etcd-master 1/1 Running 0 29m kube-apiserver-master 1/1 Running 0 29m kube-controller-manager-master 1/1 Running 0 29m kube-proxy-pjkmd 1/1 Running 0 7m35s kube-proxy-snb84 1/1 Running 0 7m46s kube-proxy-tp7wm 1/1 Running 0 29m kube-scheduler-master 1/1 Running 0 29m [root@master i]# kubectl get nodes NAME STATUS ROLES AGE VERSION master Ready control-plane,master 29m v1.23.5 node1 Ready 8m4s v1.23.5 node2 Ready 7m53s v1.23.5

三. 安装其他工具

3.1 持久化存储 nfs

3.1.1 搭建NFS Server（随便安装那个节点，现在安装master节点）

yum -y install nfs-utils rpcbind #分配权限 mkdir /nfsdata && chmod 666 /nfsdata && chown nfsnobody /nfsdata # 配置挂载 cat < /etc/exports /nfsdata *(rw,no_root_squash,no_all_squash,sync) EOF # 启动 systemctl start rpcbind.service systemctl enable rpcbind.service systemctl start nfs.service systemctl enable nfs.service

3.1.2 安装NFS客户端(所有node节点)

如果不安装，使用StorageClass的nfs-client 的自动配置程序，我们也叫它 Provisioner所在的node节点就会一直ContainerCreating

[root@master nfs-client]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nfs-client-provisioner-798cfd7476-zrndd 0/1 ContainerCreating 0 3m53s node1

安装

yum -y install nfs-utils rpcbind systemctl start rpcbind.service systemctl enable rpcbind.service systemctl start nfs.service systemctl enable nfs.service [root@node1 ~]# showmount -e 192.168.4.27 Export list for 192.168.4.27: /nfsdata *

3.1.3 安装nfs-client-provisioner

设置StorageClass，自动生成PV

nfs-rbac.yaml

cat < nfs-rbac.yaml apiVersion: v1 kind: ServiceAccount metadata: name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default #根据实际环境设定namespace,下面类同 --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: nfs-client-provisioner-runner rules: - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "create", "delete"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] verbs: ["create", "update", "patch"] --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: run-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default roleRef: kind: ClusterRole name: nfs-client-provisioner-runner apiGroup: rbac.authorization.k8s.io --- kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: name: leader-locking-nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default rules: - apiGroups: [""] resources: ["endpoints"] verbs: ["get", "list", "watch", "create", "update", "patch"] --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: leader-locking-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default roleRef: kind: Role name: leader-locking-nfs-client-provisioner apiGroup: rbac.authorization.k8s.io EOF

nfs-storage.yaml

cat < nfs-storage.yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: nfs-storage provisioner: nfs-storage #这里的名称要和provisioner配置文件中的环境变量PROVISIONER_NAME保持一致 parameters: archiveOnDelete: "true" reclaimPolicy: Retain EOF

nfs-provisioner.yaml

cat < nfs-provisioner.yaml apiVersion: apps/v1 kind: Deployment metadata: name: nfs-client-provisioner labels: app: nfs-client-provisioner # replace with namespace where provisioner is deployed namespace: default #与RBAC文件中的namespace保持一致 spec: replicas: 1 selector: matchLabels: app: nfs-client-provisioner strategy: type: Recreate selector: matchLabels: app: nfs-client-provisioner template: metadata: labels: app: nfs-client-provisioner spec: serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-provisioner #image: quay.io/external_storage/nfs-client-provisioner:latest #这里特别注意，在k8s-1.20以后版本中使用上面提供的包，并不好用，这里我折腾了好久，才解决，后来在官方的github上，别人提的问题中建议使用下面这个包才解决的，我这里是下载后，传到我自已的仓库里 #easzlab/nfs-subdir-external-provisioner:v4.0.2 image: registry.cn-shanghai.aliyuncs.com/wanfei/nfs-subdir-external-provisioner:v4.0.2 volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env: - name: PROVISIONER_NAME value: nfs-storage #provisioner名称,请确保该名称与 nfs-StorageClass.yaml文件中的provisioner名称保持一致 - name: NFS_SERVER value: 192.168.4.27 #NFS Server IP地址 - name: NFS_PATH value: "/nfsdata" #NFS挂载卷 volumes: - name: nfs-client-root nfs: server: 192.168.4.27 #NFS Server IP地址 path: "/nfsdata" #NFS 挂载卷 EOF

安装

kubectl apply -f . [root@master nfs-client]# kubectl get pods | grep nfs-client nfs-client-provisioner-777fbf8b55-2ptbm 1/1 Running 0 34s

设置默认的StorageClass(有default)

kubectl patch storageclass nfs-storage -p '{ "metadata" : { "annotations" :{"storageclass.kubernetes.io/is-default-class": "true"}}}' [root@master ~]# kubectl get sc | grep nfs-storage nfs-storage (default) nfs-storage Retain Immediate false 71s # 取消default,值为"false" kubectl patch storageclass nfs-storage -p '{ "metadata" : { "annotations" :{"storageclass.kubernetes.io/is-default-class": "false"}}}'

参考 安装helm

3.2.1 下载

下载地址 helm]# wget helm]# tar -xvf helm-v3.8.1-linux-amd64.tar.gz linux-amd64/ linux-amd64/helm linux-amd64/README.md linux-amd64/LICENSE

3.2.2 安装

将helm移到/usr/local/bin目录

[root@master helm]# mv linux-amd64/helm /usr/local/bin [root@master helm]# helm version version.BuildInfo{Version:"v3.8.1", GitCommit:"0ad800ef43d3b826f31a5ad8dfbb4fe05d143688", GitTreeState:"clean", GoVersion:"go1.13.12"}

3.2.3 添加几个repo

helm repo add apphub https://apphub.aliyuncs.com helm repo add stable https://charts.helm.sh/stable helm repo add bitnami https://charts.bitnami.com/bitnami helm repo update # 例如搜索redis chart [root@master helm]# helm search repo redis NAME CHART VERSION APP VERSION DESCRIPTION apphub/prometheus-redis-exporter 3.2.2 1.3.4 Prometheus exporter for Redis metrics apphub/redis 10.5.3 5.0.7 Open source, advanced key-value store. It is of... apphub/redis-cache 0.5.0 4.0.12-alpine A pure in-memory redis cache, using statefulset... apphub/redis-ha 4.3.3 5.0.6 Highly available Kubernetes implementation of R... apphub/redis-operator 1.0.0 Redis Operator provides high availability redis... apphub/redispapa 0.0.1 0.0.1 利用redis的info信息对redis的使用情况进行监控的一... bitnami/redis 16.6.0 6.2.6 Redis(TM) is an open source, advanced key-value... bitnami/redis-cluster 7.4.1 6.2.6 Redis(TM) is an open source, scalable, distribu... stable/prometheus-redis-exporter 3.5.1 1.3.4 DEPRECATED Prometheus exporter for Redis metrics stable/redis 10.5.7 5.0.7 DEPRECATED Open source, advanced key-value stor... stable/redis-ha 4.4.6 5.0.6 DEPRECATED - Highly available Kubernetes implem... stable/sensu 0.2.5 0.28 DEPRECATED Sensu monitoring framework backed by... apphub/codis 3.2 3.2 A Helm chart for Codis

本文转自：https://blog.csdn.net/qq_38983728/article/details/123755691

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。