Ubuntu部署 Kubernetes1.23
文章目录
- Ubuntu部署 Kubernetes1.23
- 资源列表
- 基础环境
- 一、环境准备(三台主机都要执行)
- 1.1、安装常用软件
- 1.2、关闭交换分区
- 1.3、开启IPv4转发和内核优化
- 1.4、时间同步
- 二、安装Docker(三台主机都要执行)
- 2.1、卸载残留Docker软件包
- 2.2、更新软件包
- 2.3、安装Docker依赖
- 2.4、添加Docker官方GPG密钥
- 2.5、添加Docker软件源
- 2.6、安装Docker
- 2.7、配置用户组(可选)
- 2.8、安装工具
- 2.9、开启Docker
- 2.10、配置Docker加速器
- 三、部署Kubernetes集群
- 3.1、配置Kubernetes的YUM源(三台主机都要执行)
- 3.2、查看Kubernetes可用版本
- 3.3、安装kubeadm管理工具三台主机都要执行
- 3.4、设置Kubelet开机启动三台主机都要执行
- 四、kubeadm初始化集群
- 4.1、master节点生成初始化配置文件
- 4.2、master节点修改初始化配置文件
- 4.3、master节点拉取所需镜像
- 4.4、master初始化集群
- 4.5、master节点复制k8s认证文件到用户的home目录
- 4.6、Node节点加入集群
- 4.7、在master主机查看节点状态
- 五、安装flannel网络插件
- 5.1、安装flannel网络插件master
- 5.2、查看节点状态
资源列表
操作系统 | 配置 | 主机名 | IP | 所需软件 |
---|---|---|---|---|
Ubuntu 24 | 2C4G | master | 192.168.93.139 | Docker Ce、kube-apiserver、kube-controller-manager、kube-scheduler、kubelet、Etcd、kube-proxy |
Ubuntu 24 | 2C4G | node1 | 192.168.93.143 | Docker CE、kubectl、kube-proxy、Flnnel |
Ubuntu 24 | 2C4G | node2 | 192.168.93.144 | Docker CE、kubectl、kube-proxy、Flnnel |
基础环境
- 修改主机名
sudo hostnamectl set-hostname master
sudo hostnamectl set-hostname node1
sudo hostnamectl set-hostname node2
- 切换root用户
sudo -i
- 绑定hosts解析
cat >> /etc/hosts << EOF
192.168.93.139 master
192.168.93.143 node1
192.168.93.144 node2
EOF
一、环境准备(三台主机都要执行)
- 在正式开始部署kubernetes集群之前,先要进行如下准备工作。基础环境相关配置操作,在三台主机master、node01、node02上都需要执行。
1.1、安装常用软件
# 更新软件仓库
sudo apt update# 安装常用软件
sudo apt install vim lrzsz unzip wget net-tools tree bash-completion telnet -y
1.2、关闭交换分区
- kubeadm不支持swap交换分区
# 临时关闭
swapoff -a# 永久关闭
sed -i '/swap/s/^/#/' /etc/fstab
1.3、开启IPv4转发和内核优化
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sudo sysctl --system
1.4、时间同步
sudo apt -y install ntpdate
ntpdate ntp.aliyun.com
二、安装Docker(三台主机都要执行)
- 所有节点都要操作
- 在Ubuntu系统中安装Docker时,官方推荐Ubuntu系统最好是64位,可以在终端执行
uname -a
命令查看当前系统是否位64位操作系统。
2.1、卸载残留Docker软件包
for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done
2.2、更新软件包
- 在终端中执行以下命令来更新Ubuntu软件包列表和已安装软件的版本升级
sudo apt update
sudo apt upgrade
2.3、安装Docker依赖
- Docker在Ubuntu上依赖一些软件包,执行以下命令来安装这些依赖
apt-get -y install ca-certificates curl gnupg lsb-release
2.4、添加Docker官方GPG密钥
- 执行以下命令来添加Docker官方的GPG密钥
# 最终回显OK表示运行命令正确
curl -fsSL http://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -
2.5、添加Docker软件源
-
注意:该命令需要使用
root
权限 -
执行以下命令来添加Docker的软件源
# 需要管理员交互式按一下回车键
sudo add-apt-repository "deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
2.6、安装Docker
- 执行以下命令安装Docker
apt-get -y install docker-ce docker-ce-cli containerd.io
2.7、配置用户组(可选)
- 默认情况下,只有root用户和Docker组的用户才能运行Docker命令。我们可以将当前用户添加到Docker组,以避免每次使用时都需要使用sudo。
- 注意:重新登录才能使更改生效
sudo usermod -aG docker $USER
2.8、安装工具
apt-get -y install apt-transport-https ca-certificates curl software-properties-common
2.9、开启Docker
systemctl start docker
systemctl enable docker
2.10、配置Docker加速器
vim /etc/docker/daemon.json
{"registry-mirrors": ["https://registry.docker-cn.com","http://hub-mirror.c.163.com","https://docker.mirrors.ustc.edu.cn","https://kfwkfulq.mirror.aliyuncs.com"]
}# 重启Docker
systemctl daemon-reload
systemctl restart docker
三、部署Kubernetes集群
- 准备好基础环境和Docker环境,下面就开始通过kubeadm来部署kubernetes集群
3.1、配置Kubernetes的YUM源(三台主机都要执行)
- 这里使用aliyun的源
# 安装软件包
apt-get install -y apt-transport-https ca-certificates curl# 下载Kubernetes GPG密钥
curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg# 将GPG密钥添加到APT的密钥管理中
cat /usr/share/keyrings/kubernetes-archive-keyring.gpg | sudo apt-key add -# 指定软件仓库位置
echo "deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list# 更新软件仓库
apt-get update
3.2、查看Kubernetes可用版本
apt-cache madison kubeadm
3.3、安装kubeadm管理工具三台主机都要执行
- kubectl:命令行管理工具、kubeadm:安装K8S集群工具、kubelet管理容器工具
# 安装1.23版本的Kubernetes,因为1.23以后Kubernetes就不再支持Docker做底层容器运行时
apt-get install -y kubelet=1.23.0-00 kubeadm=1.23.0-00 kubectl=1.23.0-00# 锁定版本,防止自动升级
apt-mark hold kubelet kubeadm kubectl# 查看版本
kubelet --version
kubeadm version
kubectl version
3.4、设置Kubelet开机启动三台主机都要执行
systemctl enable kubelet
四、kubeadm初始化集群
4.1、master节点生成初始化配置文件
root@master:~# kubeadm config print init-defaults > init-config.yaml
4.2、master节点修改初始化配置文件
root@master:~# vim init-config.yaml
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:- system:bootstrappers:kubeadm:default-node-tokentoken: abcdef.0123456789abcdefttl: 24h0m0susages:- signing- authentication
kind: InitConfiguration
localAPIEndpoint:advertiseAddress: 192.168.93.139 # master节点IP地址bindPort: 6443
nodeRegistration:criSocket: /var/run/dockershim.sockimagePullPolicy: IfNotPresentname: master # 如果使用域名保证可以解析,或直接使用IP地址taints: null
---
apiServer:timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns: {}
etcd:local:dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers # 默认地址国内无法访问,修改为国内地址
kind: ClusterConfiguration
kubernetesVersion: 1.23.0 # 指定kubernetes部署的版本
networking:dnsDomain: cluster.localserviceSubnet: 10.96.0.0/12 # service资源的网段,集群内部的网络podSubnet: 10.244.0.0/16 # 新增加Pod资源网段,需要与下面的pod网络插件地址一致
scheduler: {}
4.3、master节点拉取所需镜像
# 查看初始化需要的镜像
root@master:~# kubeadm config images list --config=init-config.yaml
registry.aliyuncs.com/google_containers/kube-apiserver:v1.23.0
registry.aliyuncs.com/google_containers/kube-controller-manager:v1.23.0
registry.aliyuncs.com/google_containers/kube-scheduler:v1.23.0
registry.aliyuncs.com/google_containers/kube-proxy:v1.23.0
registry.aliyuncs.com/google_containers/pause:3.6
registry.aliyuncs.com/google_containers/etcd:3.5.1-0
registry.aliyuncs.com/google_containers/coredns:v1.8.6# 拉取所需镜像
root@master:~# kubeadm config images pull --config=init-config.yaml
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-apiserver:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-controller-manager:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-scheduler:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/kube-proxy:v1.23.0
[config/images] Pulled registry.aliyuncs.com/google_containers/pause:3.6
[config/images] Pulled registry.aliyuncs.com/google_containers/etcd:3.5.1-0
[config/images] Pulled registry.aliyuncs.com/google_containers/coredns:v1.8.6# 查看拉取的镜像
oot@master:~# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.aliyuncs.com/google_containers/kube-apiserver v1.23.0 e6bf5ddd4098 2 years ago 135MB
registry.aliyuncs.com/google_containers/kube-proxy v1.23.0 e03484a90585 2 years ago 112MB
registry.aliyuncs.com/google_containers/kube-controller-manager v1.23.0 37c6aeb3663b 2 years ago 125MB
registry.aliyuncs.com/google_containers/kube-scheduler v1.23.0 56c5af1d00b5 2 years ago 53.5MB
registry.aliyuncs.com/google_containers/etcd 3.5.1-0 25f8c7f3da61 2 years ago 293MB
registry.aliyuncs.com/google_containers/coredns v1.8.6 a4ca41631cc7 2 years ago 46.8MB
hello-world latest feb5d9fea6a5 2 years ago 13.3kB
registry.aliyuncs.com/google_containers/pause 3.6 6270bb605e12 2 years ago 683kB
4.4、master初始化集群
root@master:~# kubeadm init --config=init-config.yaml
[init] Using Kubernetes version: v1.23.0
[preflight] Running pre-flight checks[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 27.1.1. Latest validated version: 20.10
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local master] and IPs [10.96.0.1 192.168.93.139]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [localhost master] and IPs [192.168.93.139 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [localhost master] and IPs [192.168.93.139 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 5.002861 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.23" in namespace kube-system with the configuration for the kubelets in the cluster
NOTE: The "kubelet-config-1.23" naming of the kubelet ConfigMap is deprecated. Once the UnversionedKubeletConfigMap feature gate graduates to Beta the default name will become just "kubelet-config". Kubeadm upgrade will handle this transition transparently.
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node master as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: abcdef.0123456789abcdef
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy
#############################################################
Your Kubernetes control-plane has initialized successfully!
#############################################################
To start using your cluster, you need to run the following as a regular user:#############################################################mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/config
#############################################################Alternatively, if you are the root user, you can run:export KUBECONFIG=/etc/kubernetes/admin.confYou should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:https://kubernetes.io/docs/concepts/cluster-administration/addons/Then you can join any number of worker nodes by running the following on each as root:
#############################################################
kubeadm join 192.168.93.139:6443 --token abcdef.0123456789abcdef \--discovery-token-ca-cert-hash sha256:0d52197239b2e78ab5bfcae2b0800881c32729b970452f183b018abc002c0256
#############################################################
4.5、master节点复制k8s认证文件到用户的home目录
root@master:~# mkdir -p $HOME/.kube
root@master:~# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
root@master:~# sudo chown $(id -u):$(id -g) $HOME/.kube/config
4.6、Node节点加入集群
- 直接把
master
节点初始化之后的最后回显的token复制粘贴到node节点回车即可,无须做任何配置
# node1
root@node1:~# kubeadm join 192.168.93.139:6443 --token abcdef.0123456789abcdef \> --discovery-token-ca-cert-hash sha256:0d52197239b2e78ab5bfcae2b0800881c32729b970452f183b018abc002c0256
[preflight] Running pre-flight checks[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 27.1.1. Latest validated version: 20.10
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
W0728 12:55:56.892027 8586 utils.go:69] The recommended value for "resolvConf" in "KubeletConfiguration" is: /run/systemd/resolve/resolv.conf; the provided value is: /run/systemd/resolve/resolv.conf
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
# node2
root@node2:~# kubeadm join 192.168.93.139:6443 --token abcdef.0123456789abcdef \> --discovery-token-ca-cert-hash sha256:0d52197239b2e78ab5bfcae2b0800881c32729b970452f183b018abc002c0256
[preflight] Running pre-flight checks[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 27.1.1. Latest validated version: 20.10
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
W0728 12:56:01.262467 9134 utils.go:69] The recommended value for "resolvConf" in "KubeletConfiguration" is: /run/systemd/resolve/resolv.conf; the provided value is: /run/systemd/resolve/resolv.conf
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
4.7、在master主机查看节点状态
- 在初始化k8s-master时并没有网络相关的配置,所以无法跟node节点通信,因此状态都是“Not Ready”。但是通过kubeadm join加入的node节点已经在master上可以看到
root@master:~# kubectl get node
NAME STATUS ROLES AGE VERSION
master NotReady control-plane,master 4m3s v1.23.0
node1 NotReady <none> 93s v1.23.0
node2 NotReady <none> 90s v1.23.0
五、安装flannel网络插件
- flannel是一个轻量级的网络插件,基于虚拟网络的方式,使用了多种后端实现,如基于Overlay的VXLAN和基于Host-Gateway的方式。它创建了一个覆盖整个集群的群集网络,使得Pod可以跨节点通信
5.1、安装flannel网络插件master
- 可以使用的网络插件有很多,本次使用flannel
root@master:~# kubectl apply -f kube-flannel.yaml
namespace/kube-flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
5.2、查看节点状态
# 查看Node节点状态
root@master:~# kubectl get node
NAME STATUS ROLES AGE VERSION
master Ready control-plane,master 6m31s v1.23.0
node1 Ready <none> 4m1s v1.23.0
node2 Ready <none> 3m58s v1.23.0
# 查看所有Pod状态
root@master:~# kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-flannel kube-flannel-ds-24zvx 1/1 Running 0 72s
kube-flannel kube-flannel-ds-297b4 1/1 Running 0 72s
kube-flannel kube-flannel-ds-x4zkl 1/1 Running 0 72s
kube-system coredns-6d8c4cb4d-58lcm 1/1 Running 0 7m1s
kube-system coredns-6d8c4cb4d-vqbw4 1/1 Running 0 7m1s
kube-system etcd-master 1/1 Running 0 7m14s
kube-system kube-apiserver-master 1/1 Running 0 7m16s
kube-system kube-controller-manager-master 1/1 Running 0 7m16s
kube-system kube-proxy-bwsdq 1/1 Running 0 4m47s
kube-system kube-proxy-jsvmj 1/1 Running 0 4m44s
kube-system kube-proxy-znwcq 1/1 Running 0 7m1s
kube-system kube-scheduler-master 1/1 Running 0 7m15s
# 查看组件状态
root@master:~# kubectl get cs
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health":"true","reason":""}