Centos 7 即将结束维护，虽然有很多种替代方案，这次使用 openEuler 看看。
OS版本：openEuler 22.03 (LTS-SP2)
K8S版本：1.28.x
一、基础操作
1、修改主机名
hostnamectl --static set-hostname [hostname]
2、关闭防火墙
service firewalld stop systemctl disable firewalld
3、关闭selinux
# 查看状态 sestatus sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config setenforce 0 # 查看状态 sestatus
4、关闭虚拟内存
# 临时关闭（重启后失效，会导致k8s启动失败） swapoff -a # 永久关闭 # 编辑/etc/fstab注释/dev/mapper/openeuler-swap none swap defaults 0 0这一行 vi /etc/fstab # 将swap那行删除或者注释掉， 比如： #/dev/mapper/openeuler-swap none swap defaults 0 0
5、加载内核模块 br_netfilter
modprobe br_netfilter echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables echo 1 > /proc/sys/net/ipv4/ip_forward
二、安装软件
1、配置kubernetes软件源
# 这会覆盖 /etc/yum.repos.d/kubernetes.repo 中现存的所有配置 cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 EOF
2、安装kubernetes
dnf install -y kubelet kubeadm kubectl systemctl enable kubelet.service
3、安装 containerd
从 containerd release 页面 https://github.com/containerd/containerd/releases 下载最新 containerd 压缩包：
wget https://github.com/containerd/containerd/releases/download/v1.6.24/containerd-1.6.24-linux-amd64.tar.gz dnf install tar tar Cxzvf /usr/local containerd-1.6.24-linux-amd64.tar.gz
解压完成后创建一个 systemctl 的 service 文件 /usr/lib/systemd/system/containerd.service 写入下面内容：
[Unit] Description=containerd container runtime Documentation=https://containerd.io After=network.target local-fs.target [Service] #uncomment to enable the experimental sbservice (sandboxed) version of containerd/cri integration #Environment="ENABLE_CRI_SANDBOXES=sandboxed" ExecStartPre=-/sbin/modprobe overlay ExecStart=/usr/local/bin/containerd Type=notify Delegate=yes KillMode=process Restart=always RestartSec=5 # Having non-zero Limit*s causes performance problems due to accounting overhead # in the kernel. We recommend using cgroups to do container-local accounting. LimitNPROC=infinity LimitCORE=infinity LimitNOFILE=infinity # Comment TasksMax if your systemd version does not supports it. # Only systemd 226 and above support this version. TasksMax=infinity OOMScoreAdjust=-999 [Install] WantedBy=multi-user.target
创建 containerd 配置文件 /etc/containerd/config.toml，并填入以下内容：
mkdir -p /etc/containerd
version = 2 [plugins."io.containerd.grpc.v1.cri"] sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9"
这里重点就是设置 pause 容器的仓库地址，默认的 gcr 仓库在国内是无法访问的。
然后执行 systemctl 命令，以服务的形式运行 containerd：
systemctl daemon-reload systemctl enable --now containerd
4、安装 runc
从 runc release 页面 https://github.com/opencontainers/runc/releases 下载最新 runc ，下载完成后执行命令：
wget https://github.com/opencontainers/runc/releases/download/v1.1.9/runc.amd64 install -m 755 runc.amd64 /usr/local/sbin/runc
5、安装 cni-plugin
从 github release 页面下载 https://github.com/containernetworking/plugins/releases 下载完成后执行解压命令：
mkdir -p /opt/cni/bin wget https://github.com/containernetworking/plugins/releases/download/v1.3.0/cni-plugins-linux-amd64-v1.3.0.tgz tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.3.0.tgz
三、开始部署
在当前目录创建一个 kubeadm-init-config.yaml 文件，填入以下内容：
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
bootstrapTokens:
- token: abcdef.0123456789abcdef
ttl: 24h0m0s
localAPIEndpoint:
advertiseAddress: 192.168.100.161
bindPort: 6443
nodeRegistration:
criSocket: unix:///var/run/containerd/containerd.sock
imagePullPolicy: IfNotPresent
taints: []
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: ClusterConfiguration
apiServer:
timeoutForControlPlane: 4m0s
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kubernetesVersion: 1.28.2
networking:
dnsDomain: cluster.local
serviceSubnet: 10.96.0.0/12
podSubnet: 172.30.0.0/16
scheduler: {}
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
failSwapOn: false
address: 0.0.0.0
enableServer: true
cgroupDriver: cgroupfs
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
ipvs:
strictARP: true
A、advertiseAddress 根据实际情况调整，配置为本节点管理网网卡的 IP，在后面添加节点时，会通过这个 IP 添加
B、kubernetesVersion 在我编写本文档时最新的 kubernetes 版本为 1.28.2
C、imageRepository 这个时我们能在国内环境安装成功的关键，kubeadm 默认的容器镜像仓库在国外，我们是无法访问的
执行初始化命令：
kubeadm init --config kubeadm-init-config.yaml
初始化成功后会输出以下提示：
Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.100.161:6443 --token abcdef.0123456789abcdef \ --discovery-token-ca-cert-hash sha256:b2701a81b5d227a9bf6cad39ccde394d98c3c3bcc75b75323cf9bfe8d713109f
根据提示执行下面命令：
mkdir -p $HOME/.kube cp -i /etc/kubernetes/admin.conf $HOME/.kube/config chown $(id -u):$(id -g) $HOME/.kube/config
然后使用 kubectl 验证：
[root@OpenEuler-1 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION openeuler-1 NotReady control-plane 2m11s v1.28.2
现在节点 STATUS 是 NotReady，这是因为没有安装网络插件
四、安装网络插件
wget https://docs.tigera.io/calico/latest/getting-started/kubernetes/self-managed-onprem/onpremises kubectl apply -f calico.yaml
安装完成后在使用 kubelet 命令验证：
# kubectl get nodes NAME STATUS ROLES AGE VERSION openeuler-1 Ready control-plane 40m v1.28.2 openeuler-2 Ready <none> 15m v1.28.2 openeuler-3 Ready <none> 15m v1.28.2
这时候节点的 STATUS 变成 Ready 了
五、
[root@OpenEuler-1 ~]# kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME openeuler-1 Ready control-plane 41m v1.28.2 192.168.100.161 <none> openEuler 22.03 (LTS-SP2) 5.10.0-153.28.0.105.oe2203sp2.x86_64 containerd://1.6.24 openeuler-2 Ready <none> 16m v1.28.2 192.168.100.162 <none> openEuler 22.03 (LTS-SP2) 5.10.0-153.28.0.105.oe2203sp2.x86_64 containerd://1.6.24 openeuler-3 Ready <none> 16m v1.28.2 192.168.100.163 <none> openEuler 22.03 (LTS-SP2) 5.10.0-153.28.0.105.oe2203sp2.x86_64 containerd://1.6.24
[root@OpenEuler-1 ~]# dnf install -y kubelet kubeadm kubectl Last metadata expiration check: 0:01:34 ago on 2023年10月09日 星期一 02时39分46秒. Dependencies resolved. =============================================================================================================================== Package Architecture Version Repository Size =============================================================================================================================== Installing: kubeadm x86_64 1.28.2-0 kubernetes 11 M kubectl x86_64 1.28.2-0 kubernetes 11 M kubelet x86_64 1.28.2-0 kubernetes 21 M Installing dependencies: conntrack-tools x86_64 1.4.6-6.oe2203sp2 everything 168 k containernetworking-plugins x86_64 1.1.1-2.oe2203sp2 OS 18 M cri-tools x86_64 1.26.0-0 kubernetes 8.6 M ebtables x86_64 2.0.11-11.oe2203sp2 update 81 k libnetfilter_cthelper x86_64 1.0.0-16.oe2203sp2 everything 20 k libnetfilter_cttimeout x86_64 1.0.0-15.oe2203sp2 everything 21 k libnetfilter_queue x86_64 1.0.5-2.oe2203sp2 OS 25 k socat x86_64 1.7.3.2-8.oe2203sp2 everything 155 k Transaction Summary =============================================================================================================================== Install 11 Packages Total download size: 70 M Installed size: 302 M
[root@OpenEuler-1 ~]# kubeadm config images list W1009 02:48:53.679858 5039 version.go:104] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get "https://dl.k8s.io/release/stable-1.txt": context deadline exceeded (Client.Timeout exceeded while awaiting headers) W1009 02:48:53.680144 5039 version.go:105] falling back to the local client version: v1.28.2 registry.k8s.io/kube-apiserver:v1.28.2 registry.k8s.io/kube-controller-manager:v1.28.2 registry.k8s.io/kube-scheduler:v1.28.2 registry.k8s.io/kube-proxy:v1.28.2 registry.k8s.io/pause:3.9 registry.k8s.io/etcd:3.5.9-0 registry.k8s.io/coredns/coredns:v1.10.1
这样就安装完成了吧！
ChiuYut
2023年10月09日