二进制部署kubernetes集群的推荐方式

软件版本：

软件版本
containerdv1.6.5
etcdv3.5.0
kubernetesv1.24.0

一、系统环境

1.1 环境准备

角色IP服务
k8s-master01192.168.10.10etcd、containerd、kube-apiserver、kube-scheduler、kube-controller-manager、kubele、kube-proxy
k8s-node01后续etcd、containerd、kubele、kube-proxy
k8s-node02后续etcd、containerd、kubele、kube-proxy

1.2 环境初始化

#关闭防火墙和selinux
systemctl stop firewalld
systemctl disable firewalld
sed -i 's/enforcing/disabled/' /etc/selinux/config  
setenforce 0 #关闭swap
swapoff -a  
sed -ri 's/.*swap.*/#&/' /etc/fstab#将桥接的IPv4流量传递到iptables的链，lsmod | grep br_netfilter可以查看，sudo modprobe显示加载
sudo modprobe br_netfilter
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF#将桥接的IPv4流量传递到iptables的链
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system #时间同步
yum install ntpdate -y
ntpdate time.windows.com
clock -w#安装ipvs
yum install ipset ipvsadm -y yum install ipset ipvsadm -y # 
cat >> /etc/sysconfig/modules/ipvs.modules << EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_sh
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- nf_conntrack_ipv4
EOFchmod +x /etc/sysconfig/modules/ipvs.modules

1.3 文件夹初始化

准备文件夹

mkdir -p /usr/local/k8s-install
mkdir -p /usr/local/k8s-install/cert
mkdir -p /usr/local/k8s-install/tools
mkdir /etc/kubernetes
mkdir /etc/kubernetes/pki
mkdir /etc/kubernetes/manifests

环境文件夹

mkdir -p /opt/etcd/{bin,ssl,cfg,data,wal}

二、软件下载

curl -L https://github.com/cloudflare/cfssl/releases/download/v1.5.0/cfssl_1.5.0_linux_amd64 -o cfssl
chmod +x cfssl
curl -L https://github.com/cloudflare/cfssl/releases/download/v1.5.0/cfssljson_1.5.0_linux_amd64 -o cfssljson
chmod +x cfssljson
curl -L https://github.com/cloudflare/cfssl/releases/download/v1.5.0/cfssl-certinfo_1.5.0_linux_amd64 -o cfssl-certinfo
chmod +x cfssl-certinfo
mv {cfssl,cfssljson,cfssl-certinfo} /usr/local/bin

kubernetes：

k8s二进制包下载
cd /usr/local/k8s-install/tools
wget -c https://dl.k8s.io/v1.24.0/kubernetes-server-linux-amd64.tar.gztar -zxf kubernetes-server-linux-amd64.tar.gz
cd kubernetes/server/bincp {kube-apiserver,kube-controller-manager,kube-scheduler,kubelet,kube-proxy,kubectl} /usr/local/bin

三、生成证书

3.1 证书签发的配置文件

cat > /usr/local/k8s-install/cert/ca-config.json <<END
{"signing": {"default": {"expiry": "876000h"},"profiles": {"kubernetes": {"usages": ["signing","key encipherment","server auth","client auth"],"expiry": "876000h"},"etcd": {"usages": ["signing","key encipherment","server auth","client auth"],"expiry": "876000h"}}}
}
END
cp /usr/local/k8s-install/cert/ca-config.json /etc/kubernetes

3.2 etcd CA证书

rm -rf /usr/local/k8s-install/cert/etcd/
mkdir /usr/local/k8s-install/cert/etcd/
cd /usr/local/k8s-install/cert/etcd/
cat > etcd-ca-csr.json <<END
{"CN": "etcd","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","ST": "Beijing","L": "Beijing","O": "etcd","OU": "etcd"}],"ca": {"expiry": "87600h"}
}
END
cfssl gencert -initca etcd-ca-csr.json | cfssljson -bare ca -cp {ca.pem,ca-key.pem} /opt/etcd/ssl/

3.3 etcd 证书签发

cd /usr/local/k8s-install/cert/etcd
cat > etcd-csr.json<< EOF
{"CN":"etcd","hosts":["127.0.0.1","k8s-master01","192.168.10.10"],"key":{"algo":"rsa","size":2048},"names":[{"C":"CN","L":"BeiJing","ST":"BeiJing","O": "Kubernetes","OU": "etcd"}]
}
EOFcfssl gencert -ca=/usr/local/k8s-install/cert/etcd/ca.pem -ca-key=/usr/local/k8s-install/cert/etcd/ca-key.pem -config=/usr/local/k8s-install/cert/ca-config.json -profile=etcd etcd-csr.json | cfssljson -bare etcdcp {etcd-key.pem,etcd.pem} /opt/etcd/ssl/

3.4 k8s CA证书

cat > /usr/local/k8s-install/cert/ca-csr.json <<END   
{"CN": "kubernetes","key": {      "algo": "rsa",     "size": 2048},             "names":[{         "C": "CN",     "ST": "Beijing","L": "Beijing",          "O": "Kubernetes",         "OU": "Kubernetes"     }]      
}                        
ENDcd /usr/local/k8s-install/cert
cfssl gencert -initca ca-csr.json | cfssljson -bare ca
cp {ca.pem,ca-key.pem} /etc/kubernetes/pki

3.5 apiserver证书签发

rm -rf /usr/local/k8s-install/cert/apiserver
mkdir /usr/local/k8s-install/cert/apiserver
cd /usr/local/k8s-install/cert/apiservercat > server-csr.json << END
{"CN": "kubernetes","hosts": ["127.0.0.1","10.0.0.1","192.168.10.10","k8s-master01","kubernetes","kubernetes.default","kubernetes.default.svc","kubernetes.default.svc.cluster","kubernetes.default.svc.cluster.local"],"key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","ST": "Beijing","L": "Beijing","O": "Kubernetes","OU": "Kubernetes"}]
}
ENDcfssl gencert -ca=/etc/kubernetes/pki/ca.pem -ca-key=/etc/kubernetes/pki/ca-key.pem  --config=/usr/local/k8s-install/cert/ca-config.json -profile=kubernetes  server-csr.json | cfssljson -bare servercp -r {server.pem,server-key.pem} /etc/kubernetes/pki

3.6 front-proxy证书签发

官方文档：配置聚合层 | Kubernetes

ca证书生成：

rm -rf /usr/local/k8s-install/cert/font-proxy
mkdir /usr/local/k8s-install/cert/font-proxy
cd /usr/local/k8s-install/cert/font-proxy
cat > front-proxy-ca-csr.json  << EOF 
{"CN": "kubernetes","key": {"algo": "rsa","size": 2048}
}
EOFcfssl gencert   -initca front-proxy-ca-csr.json | cfssljson -bare front-proxy-ca

证书签发

cd /usr/local/k8s-install/cert/font-proxy
cat > front-proxy-client-csr.json  << EOF 
{"CN": "front-proxy-client","key": {"algo": "rsa","size": 2048}
}
EOFcfssl gencert   -ca=front-proxy-ca.pem   -ca-key=front-proxy-ca-key.pem   -config=/usr/local/k8s-install/cert/ca-config.json   -profile=kubernetes   front-proxy-client-csr.json | cfssljson -bare front-proxy-clientcp {front-proxy-ca.pem,front-proxy-ca-key.pem,front-proxy-client-key.pem,front-proxy-client.pem} /etc/kubernetes/pki

3.7 kube-controller-manager证书签发

rm -rf /usr/local/k8s-install/cert/kube-controller-manager
mkdir /usr/local/k8s-install/cert/kube-controller-manager
cd /usr/local/k8s-install/cert/kube-controller-manager
cat > kube-controller-manager-csr.json << EOF
{"CN": "system:kube-controller-manager","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "BeiJing", "ST": "BeiJing","O": "system:masters","OU": "Kubernetes"}]
}
EOFcfssl gencert -ca=/etc/kubernetes/pki/ca.pem -ca-key=/etc/kubernetes/pki/ca-key.pem -config=/usr/local/k8s-install/cert/ca-config.json -profile=kubernetes kube-controller-manager-csr.json | cfssljson -bare kube-controller-manager
cp {kube-controller-manager.pem,kube-controller-manager-key.pem} /etc/kubernetes/pki

3.8 scheduler证书签发

rm -rf /usr/local/k8s-install/cert/kube-scheduler
mkdir /usr/local/k8s-install/cert/kube-scheduler
cd /usr/local/k8s-install/cert/kube-scheduler
cat > kube-scheduler-csr.json << EOF
{"CN": "system:kube-scheduler","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "BeiJing","ST": "BeiJing","O": "system:kube-scheduler","OU": "Kubernetes"}]
}
EOFcfssl gencert -ca=/etc/kubernetes/pki/ca.pem -ca-key=/etc/kubernetes/pki/ca-key.pem -config=/usr/local/k8s-install/cert/ca-config.json -profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler
cp {kube-scheduler.pem,kube-scheduler-key.pem} /etc/kubernetes/pki

3.9 admin证书签发

rm -rf /usr/local/k8s-install/cert/admin
mkdir /usr/local/k8s-install/cert/admin
cd /usr/local/k8s-install/cert/admin
cat > admin-csr.json <<EOF
{"CN": "admin","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "BeiJing","ST": "BeiJing","O": "system:masters","OU": "Kubernetes"}]
}
EOFcfssl gencert -ca=/etc/kubernetes/pki/ca.pem -ca-key=/etc/kubernetes/pki/ca-key.pem -config=/etc/kubernetes/ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin
cp {admin.pem,admin-key.pem} /etc/kubernetes/pki

3.10 ServiceAccount Key生成

k8s每创建一个ServiceAccount，都会分配一个Secret，而Secret里面的秘钥就是sa生成的。

mkdir /usr/local/k8s-install/cert/sa
cd /usr/local/k8s-install/cert/sa
openssl genrsa -out sa.key 2048
openssl rsa -in sa.key -pubout -out sa.pub
cp {sa.key,sa.pub} /etc/kubernetes/pki

3.11 kube-proxy证书签发

rm -rf /usr/local/k8s-install/cert/kube-proxy
mkdir /usr/local/k8s-install/cert/kube-proxy
cd /usr/local/k8s-install/cert/kube-proxy
cat > kube-proxy-csr.json << END
{"CN": "system:kube-proxy","hosts": [],"key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","ST": "Beijing","L": "Beijing","O": "Kubernetes","OU": "System"}]
}
ENDcfssl gencert -ca=/etc/kubernetes/pki/ca.pem -ca-key=/etc/kubernetes/pki/ca-key.pem  --config=/etc/kubernetes/ca-config.json -profile=kubernetes   kube-proxy-csr.json | cfssljson -bare kube-proxycp -r {kube-proxy.pem,kube-proxy-key.pem} /etc/kubernetes/pki

四、部署containerd

#安装containerd
wget -c https://github.com/containerd/containerd/releases/download/v1.6.5/cri-containerd-cni-1.6.5-linux-amd64.tar.gz
tar -zxf cri-containerd-cni-1.6.5-linux-amd64.tar.gz  -C /#安装libseccomp
wget http://rpmfind.net/linux/centos/8-stream/BaseOS/x86_64/os/Packages/libseccomp-2.5.1-1.el8.x86_64.rpm
rpm -ivh libseccomp-2.5.1-1.el8.x86_64.rpm#开机启动
systemctl enable containerd --now

五、部署etcd

5.1 安装

wget -c https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-amd64.tar.gz
tar -zxf etcd-v3.5.0-linux-amd64.tar.gz
cd etcd-v3.5.0-linux-amd64
mv {etcd,etcdctl,etcdutl} /opt/etcd/bin/#环境变量
echo "PATH=\$PATH:/opt/etcd/bin" >> /etc/profile
source /etc/profile

5.2 证书准备

证书文件在3.3已经准备了

5.3 配置文件

/opt/etcd/cfg/etcd.yaml 配置文件：etcd/etcd.conf.yml.sample at main · etcd-io/etcd · GitHub

192.168.10.10

name: "etcd-1"
data-dir: "/opt/etcd/data"
wal-dir: "/opt/etcd/wal"
# 用于侦听对等流量的逗号分隔的url列表。
listen-peer-urls: https://192.168.10.10:2380
# 用于侦听客户机通信的逗号分隔的url列表。
listen-client-urls: https://192.168.10.10:2379
# 这个成员的对等url的列表，以通告给集群的其他成员。url需要是逗号分隔的列表。
initial-advertise-peer-urls: https://192.168.10.10:2380
#这个成员的对等url的列表，以通告给集群的其他成员。url需要是逗号分隔的列表。
advertise-client-urls: https://192.168.10.10:2379
# Initial cluster configuration for bootstrapping.
initial-cluster: 'etcd-1=https://192.168.10.10:2380,etcd-2=https://192.168.66.41:2380,etcd-3=https://192.168.66.42:2380'
# Initial cluster token for the etcd cluster during bootstrap.
initial-cluster-token: 'etcd-cluster'
# Initial cluster state ('new' or 'existing').
initial-cluster-state: 'new'
client-transport-security:# Path to the client server TLS cert file.cert-file: /opt/etcd/ssl/etcd.pem# Path to the client server TLS key file.key-file: /opt/etcd/ssl/etcd-key.pem# Path to the client server TLS trusted CA cert file.trusted-ca-file: /opt/etcd/ssl/ca.pem
peer-transport-security:# Path to the peer server TLS cert file.cert-file: /opt/etcd/ssl/etcd.pem# Path to the peer server TLS key file.key-file: /opt/etcd/ssl/etcd-key.pem# Path to the peer server TLS trusted CA cert file.trusted-ca-file: /opt/etcd/ssl/ca.pem

192.168.66.41

name: "etcd-2"
listen-peer-urls: https://192.168.66.41:2380
# 用于侦听客户机通信的逗号分隔的url列表。
listen-client-urls: https://192.168.66.41:2379
# 这个成员的对等url的列表，以通告给集群的其他成员。url需要是逗号分隔的列表。
initial-advertise-peer-urls: https://192.168.66.41:2380
#这个成员的对等url的列表，以通告给集群的其他成员。url需要是逗号分隔的列表。
advertise-client-urls: https://192.168.66.41:2379

192.168.66.42

name: "etcd-2"
listen-peer-urls: https://192.168.66.42:2380
# 用于侦听客户机通信的逗号分隔的url列表。
listen-client-urls: https://192.168.66.42:2379
# 这个成员的对等url的列表，以通告给集群的其他成员。url需要是逗号分隔的列表。
initial-advertise-peer-urls: https://192.168.66.42:2380
#这个成员的对等url的列表，以通告给集群的其他成员。url需要是逗号分隔的列表。
advertise-client-urls: https://192.168.66.42:2379

5.4 服务配置

cat > /usr/lib/systemd/system/etcd.service << EOF 
[Unit] 
Description=Etcd Server 
After=network.target 
After=network-online.target 
Wants=network-online.target 
[Service] 
Type=notify 
ExecStart=/opt/etcd/bin/etcd  --config-file /opt/etcd/cfg/etcd.yml 
Restart=on-failure 
LimitNOFILE=65536 
[Install] 
WantedBy=multi-user.target 
EOF

5.5 启动服务

systemctl enable etcd --now

5.6 集群状态查看

[root@k8s-master01 k8s]# etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/etcd.pem --key=/opt/etcd/ssl/etcd-key.pem --endpoints="https://192.168.10.10:2379,https://192.168.66.41:2379,https://192.168.66.42:2379" endpoint status --write-out=table
+------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|        ENDPOINT        |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://192.168.10.10:2379 | 1f46bee47a4f04aa |   3.5.0 |   20 kB |     false |      false |        15 |         68 |                 68 |        |
| https://192.168.66.41:2379 |   b3e5838df5f510 |   3.5.0 |   20 kB |     false |      false |        15 |         68 |                 68 |        |
| https://192.168.66.42:2379 | a437554da4f2a14c |   3.5.0 |   25 kB |      true |      false |        15 |         68 |                 68 |        |
+------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

六、用户环境配置

rm -rf /root/.kube
mkdir /root/.kube
KUBE_CONFIG="/root/.kube/config"
KUBE_APISERVER="https://192.168.10.10:6443"#设置集群参数
kubectl config set-cluster kubernetes \--certificate-authority=/etc/kubernetes/pki/ca.pem \--embed-certs=true \--server=${KUBE_APISERVER} \--kubeconfig=${KUBE_CONFIG}
#设置客户端认证参数
kubectl config set-credentials kubernetes-admin \--client-certificate=/etc/kubernetes/pki/admin.pem \--client-key=/etc/kubernetes/pki/admin-key.pem \--embed-certs=true \--kubeconfig=${KUBE_CONFIG}
#设置上下文参数
kubectl config set-context kubernetes-admin@kubernetes \--cluster=kubernetes \--user=kubernetes-admin \--kubeconfig=${KUBE_CONFIG}
#设置默认上下文
kubectl config use-context kubernetes-admin@kubernetes --kubeconfig=${KUBE_CONFIG}

七、部署kube-apiserver

7.1 kube-apiserver 服务配置

cat > /usr/lib/systemd/system/kube-apiserver.service << EOF 
[Unit] 
Description=Kubernetes API Server 
Documentation=https://github.com/kubernetes/kubernetes 
[Service] 
ExecStart=/usr/local/bin/kube-apiserver --v=2  \
--logtostderr=true  \
--advertise-address=192.168.10.10 \
--bind-address=192.168.10.10 \
--service-node-port-range=30000-32767  \
--allow-privileged=true  \
--authorization-mode=RBAC,Node  \
--client-ca-file=/etc/kubernetes/pki/ca.pem  \
--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,ResourceQuota   \
--enable-bootstrap-token-auth=true  \
--etcd-cafile=/opt/etcd/ssl/ca.pem  \
--etcd-certfile=/opt/etcd/ssl/etcd.pem  \
--etcd-keyfile=/opt/etcd/ssl/etcd-key.pem  \
--etcd-servers=https://192.168.10.10:12379,https://192.168.10.10:22379,https://192.168.10.10:32379 \
--kubelet-client-certificate=/etc/kubernetes/pki/server.pem  \
--kubelet-client-key=/etc/kubernetes/pki/server-key.pem  \
--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname \
--proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.pem \
--proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client-key.pem \
--requestheader-allowed-names=kube-proxy \
--requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem \
--requestheader-extra-headers-prefix=X-Remote-Extra- \
--requestheader-group-headers=X-Remote-Group \
--requestheader-username-headers=X-Remote-User \
--secure-port=6443  \
--service-account-issuer=https://kubernetes.default.svc.cluster.local \
--service-account-signing-key-file=/etc/kubernetes/pki/sa.key \
--service-account-key-file=/etc/kubernetes/pki/sa.pub  \
--service-cluster-ip-range=10.0.0.0/24  \
--tls-cert-file=/etc/kubernetes/pki/server.pem  \
--tls-private-key-file=/etc/kubernetes/pki/server-key.pem \
--runtime-config=settings.k8s.io/v1alpha1=true Restart=on-failure 
RestartSec=10s
LimitNOFILE=65535
[Install] 
WantedBy=multi-user.target 
EOF

说明：

-logtostderr：启用日志 
-v：日志等级 
–log-dir：日志目录 
–etcd-servers：etcd 集群地址 
–bind-address：监听地址 
–secure-port：https 安全端口 
–advertise-address：集群通告地址 
–allow-privileged：启用授权 
–service-cluster-ip-range：Service 虚拟 IP 地址段 
–enable-admission-plugins：准入控制模块 
–authorization-mode：认证授权，启用 RBAC 授权和节点自管理 
–enable-bootstrap-token-auth：启用 TLS bootstrap 机制 
–token-auth-file：bootstrap token 文件 
--service-account-issuer 此参数可作为服务账户令牌发放者的身份标识
–service-node-port-range：Service nodeport 类型默认分配端口范围 
–kubelet-client-xxx：apiserver 访问 kubelet 客户端证书 
–tls-xxx-file：apiserver https 证书 
--enable-aggregator-routing=true如果你未在运行 API 服务器的主机上运行 kube-proxy，则必须确保使用以下 kube-apiserver 标志启用系统
–etcd-xxxfile：连接 Etcd 集群证书 
–audit-log-xxx：审计日志

如果你未在运行 API 服务器的主机上运行 kube-proxy，则必须确保使用以下 kube-apiserver 标志启用系统：

--enable-aggregator-routing=true

7.2 设置开机启动

systemctl daemon-reload 
systemctl enable kube-apiserver --now

八、kube-controller-manager

8.1 kubeconfig配置

rm -rf /etc/kubernetes/controller-manager.conf
KUBE_CONFIG="/etc/kubernetes/controller-manager.conf"
KUBE_APISERVER="https://192.168.10.10:6443"# 设置一个集群项
kubectl config set-cluster kubernetes \--certificate-authority=/etc/kubernetes/pki/ca.pem \--embed-certs=true \--server=${KUBE_APISERVER} \--kubeconfig=${KUBE_CONFIG}
# 设置一个环境项，一个上下文
kubectl config set-context system:kube-controller-manager@kubernetes \--cluster=kubernetes \--user=system:kube-controller-manager \--kubeconfig=${KUBE_CONFIG}
# 设置一个证书
kubectl config set-credentials system:kube-controller-manager \--client-certificate=/etc/kubernetes/pki/kube-controller-manager.pem \--client-key=/etc/kubernetes/pki/kube-controller-manager-key.pem \--embed-certs=true \--kubeconfig=${KUBE_CONFIG}kubectl config use-context system:kube-controller-manager@kubernetes --kubeconfig=${KUBE_CONFIG}

8.2 服务配置

cat > /usr/lib/systemd/system/kube-controller-manager.service << END
[Unit] 
Description=Kubernetes Controller Manager 
Documentation=https://github.com/kubernetes/kubernetes 
[Service] 
ExecStart=/usr/local/bin/kube-controller-manager  \
--v=2 \
--logtostderr=true \
--bind-address=127.0.0.1   \
--root-ca-file=/etc/kubernetes/pki/ca.pem \
--cluster-signing-cert-file=/etc/kubernetes/pki/ca.pem   \
--cluster-signing-key-file=/etc/kubernetes/pki/ca-key.pem  \
--service-account-private-key-file=/etc/kubernetes/pki/sa.key  \
--kubeconfig=/etc/kubernetes/controller-manager.conf \
--leader-elect=true  \
--use-service-account-credentials=true \
--node-monitor-grace-period=40s \
--node-monitor-period=5s \
--pod-eviction-timeout=2m0s \
--controllers=*,bootstrapsigner,tokencleaner \
--allocate-node-cidrs=true \
--cluster-cidr=10.244.0.0/16   \
--requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.pem \
--node-cidr-mask-size=24Restart=always
RestartSec=10s
[Install] 
WantedBy=multi-user.target
END
参数说明
–cluster-cidr string集群中 Pod 的 CIDR 范围。要求 --allocate-node-cidrs 标志为 true。
–service-cluster-ip-range集群中 Service 对象的 CIDR 范围。要求 --allocate-node-cidrs 标志为 true。
-v, --v int日志级别详细程度取值。
–kubeconfigKubernetes 认证文件
–leader-elect当该组件启动多个时，自动选举（HA）
–cluster-signing-cert-file --cluster-signing-key-file自动为 kubelet 颁发证书的 CA，与 apiserver 保持一致

8.3 启动并设置开机启动

systemctl daemon-reload 
systemctl enable kube-controller-manager --now

九、部署kube-scheduler

9.1 kubeconfig配置

rm -rf /etc/kubernetes/kube-scheduler.conf
KUBE_CONFIG="/etc/kubernetes/kube-scheduler.conf"
KUBE_APISERVER="https://192.168.10.10:6443"kubectl config set-cluster kubernetes \--certificate-authority=/etc/kubernetes/pki/ca.pem \--embed-certs=true \--server=${KUBE_APISERVER} \--kubeconfig=${KUBE_CONFIG}
kubectl config set-credentials system:kube-scheduler \--client-certificate=/etc/kubernetes/pki/kube-scheduler.pem \--client-key=/etc/kubernetes/pki/kube-scheduler-key.pem \--embed-certs=true \--kubeconfig=${KUBE_CONFIG}
kubectl config set-context system:kube-scheduler@kubernetes \--cluster=kubernetes \--user=system:kube-scheduler \--kubeconfig=${KUBE_CONFIG}
kubectl config use-context system:kube-scheduler@kubernetes --kubeconfig=${KUBE_CONFIG}

9.2 服务配置

cat > /usr/lib/systemd/system/kube-scheduler.service  << END
[Unit] 
Description=Kubernetes Scheduler 
Documentation=https://github.com/kubernetes/kubernetes
[Service] 
ExecStart=/usr/local/bin/kube-scheduler \
--v=2 \
--logtostderr=true \
--bind-address=127.0.0.1 \
--leader-elect=true \
--kubeconfig=/etc/kubernetes/kube-scheduler.confRestart=always
RestartSec=10s
[Install] 
WantedBy=multi-user.target 
END

9.3 启动并设置开机启动

systemctl daemon-reload 
systemctl enable kube-scheduler --now

十、TLS Bootstrapping配置

启用 TLS Bootstrapping 机制 TLS Bootstraping：Master apiserver 启用 TLS 认证后，Node 节点 kubelet 和 kube- proxy 要与 kube-apiserver 进行通信，必须使用 CA 签发的有效证书才可以，当 Node 节点很多时，这种客户端证书颁发需要大量工作，同样也会增加集群扩展复杂度。为了简化流程，Kubernetes 引入了 TLS bootstraping 机制来自动颁发客户端证书，kubelet 会以一个低权限用户自动向 apiserver 申请证书，kubelet 的证书由 apiserver 动态签署。
所以强烈建议在 Node 上使用这种方式，目前主要用于 kubelet，kube-proxy 还是由我们统一颁发一个证书。

TLS bootstraping 工作流程：
在这里插入图片描述

#随机token，但是只需要16个字符
head -c 16 /dev/urandom | od -An -t x | tr -d ' '
# 值如下： 737b177d9823531a433e368fcdb16f5f# 生成16个字符的
head -c 8 /dev/urandom | od -An -t x | tr -d ' '
# d683399b7a553977

生成16个字符的

[root@k8s-master01 ~]# head -c 8 /dev/urandom | od -An -t x | tr -d ' '
271c0f7e7a3bd6cd

创建kubelet上下文信息

KUBE_CONFIG="/etc/kubernetes/bootstrap-kubelet.conf"
KUBE_APISERVER="https://192.168.10.10:6443"
TOKEN_ID="2678ad"
TOKEN="271c0f7e7a3bd6cd"kubectl config set-cluster kubernetes \--certificate-authority=/etc/kubernetes/pki/ca.pem \--embed-certs=true \--server=${KUBE_APISERVER} \--kubeconfig=${KUBE_CONFIG}
kubectl config set-credentials kubelet-bootstrap \--token=${TOKEN_ID}.${TOKEN} \--kubeconfig=${KUBE_CONFIG}
kubectl config set-context kubelet-bootstrap@kubernetes \--cluster=kubernetes \--user=kubelet-bootstrap \--kubeconfig=${KUBE_CONFIG}
kubectl config use-context kubelet-bootstrap@kubernetes --kubeconfig=${KUBE_CONFIG}

创建集群引导权限文件：

TOKEN_ID="2678ad"
TOKEN="271c0f7e7a3bd6cd"cat > /usr/local/k8s-install/yaml/bootstrap.secret.yaml << END
apiVersion: v1
kind: Secret
metadata:name: bootstrap-token-${TOKEN_ID}namespace: kube-system
type: bootstrap.kubernetes.io/token
stringData:description: "The default bootstrap token generated by 'kubelet '."token-id: ${TOKEN_ID}token-secret: ${TOKEN}usage-bootstrap-authentication: "true"usage-bootstrap-signing: "true"auth-extra-groups:  system:bootstrappers:default-node-token,system:bootstrappers:worker,system:bootstrappers:ingress---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: kubelet-bootstrap
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:node-bootstrapper
subjects:
- apiGroup: rbac.authorization.k8s.iokind: Groupname: system:bootstrappers:default-node-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: node-autoapprove-bootstrap
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:certificates.k8s.io:certificatesigningrequests:nodeclient
subjects:
- apiGroup: rbac.authorization.k8s.iokind: Groupname: system:bootstrappers:default-node-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: node-autoapprove-certificate-rotation
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:certificates.k8s.io:certificatesigningrequests:selfnodeclient
subjects:
- apiGroup: rbac.authorization.k8s.iokind: Groupname: system:nodes
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:annotations:rbac.authorization.kubernetes.io/autoupdate: "true"labels:kubernetes.io/bootstrapping: rbac-defaultsname: system:kube-apiserver-to-kubelet
rules:- apiGroups:- ""resources:- nodes/proxy- nodes/stats- nodes/log- nodes/spec- nodes/metricsverbs:- "*"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: system:kube-apiservernamespace: ""
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:kube-apiserver-to-kubelet
subjects:- apiGroup: rbac.authorization.k8s.iokind: Username: kube-apiserver
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: system:kubernetesnamespace: ""
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:kube-apiserver-to-kubelet
subjects:- apiGroup: rbac.authorization.k8s.iokind: Username: kubernetes
ENDkubectl create -f /usr/local/k8s-install/yaml/bootstrap.secret.yaml

十一、部署kubelet

11.1 KubeletConfiguration配置

x509参考文章：https://kubernetes.io/search/?q=x509

cat > /etc/kubernetes/kubelet-conf.yml <<END
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
address: 0.0.0.0
port: 10250
readOnlyPort: 10255
authentication:anonymous:enabled: falsewebhook:cacheTTL: 2m0senabled: truex509:clientCAFile: /etc/kubernetes/pki/ca.pem
authorization:mode: Webhookwebhook:cacheAuthorizedTTL: 5m0scacheUnauthorizedTTL: 30s
cgroupDriver: systemd
cgroupsPerQOS: true
clusterDNS:
- 10.0.0.2
clusterDomain: cluster.local
featureGates:IPv6DualStack: true
containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
enableDebuggingHandlers: true
enforceNodeAllocatable:
- pods
eventBurst: 10
eventRecordQPS: 5
evictionHard:imagefs.available: 15%memory.available: 100Minodefs.available: 10%nodefs.inodesFree: 5%
evictionPressureTransitionPeriod: 5m0s  
failSwapOn: true
fileCheckFrequency: 20s
hairpinMode: promiscuous-bridge
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 20s
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
imageMinimumGCAge: 2m0s
iptablesDropBit: 15
iptablesMasqueradeBit: 14
kubeAPIBurst: 10
kubeAPIQPS: 5
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxPods: 110
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
registryBurst: 10
registryPullQPS: 5
resolvConf: /etc/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 2m0s
serializeImagePulls: true
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
volumeStatsAggPeriod: 1m0sEND

11.2 服务配置

cat > /usr/lib/systemd/system/kubelet.service <<END
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/kubernetes/kubernetes
After=containerd.service
Requires=containerd.service[Service]
ExecStart=/usr/local/bin/kubelet \--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf  \--kubeconfig=/etc/kubernetes/kubelet.conf \--config=/etc/kubernetes/kubelet-conf.yml \--container-runtime=remote \--container-runtime-endpoint=unix:///run/containerd/containerd.sock  \--node-labels=node.kubernetes.io/node= Restart=always
StartLimitInterval=0
RestartSec=10[Install]
WantedBy=multi-user.target
END

11.3 启动服务及开机设置

systemctl daemon-reload 
systemctl enable kubelet --now

11.4 批准 kubelet 证书申请并加入集群(这里不需要)

# 查看 kubelet 证书请求 
[root@k8s-master01 k8s-install]# kubectl get csr
NAME                                                   AGE     SIGNERNAME                                    REQUESTOR           REQUESTEDDURATION   CONDITION
node-csr-NrSNw-Gx8kR7VerABxUgHoM1mu71VbB8x598UXWOwM0   4m12s   kubernetes.io/kube-apiserver-client-kubelet   kubelet-bootstrap   <none>              Pending
# 批准申请 
[root@k8s-master01 k8s-install]# kubectl certificate approve node-csr-NrSNw-Gx8kR7VerABxUgHoM1mu71VbB8x598UXWOwM0
certificatesigningrequest.certificates.k8s.io/node-csr-NrSNw-Gx8kR7VerABxUgHoM1mu71VbB8x598UXWOwM0 approved
# 查看节点 
[root@k8s-master01 k8s-install]# kubectl get node
NAME     STATUS   ROLES    AGE   VERSION
node01   Ready    <none>   19s   v1.24.0

十二、部署kube-proxy

12.1 创建ServiceAccount

非常重要：kubernetes v1.24.0 更新之后进行创建 ServiceAccount 不会自动生成 Secret 需要对其手动创建

#创建kube-proxy的sa 
kubectl -n kube-system create serviceaccount kube-proxycat<<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:name: kube-proxy-tokennamespace: kube-systemannotations:kubernetes.io/service-account.name: "kube-proxy"
EOF#创建角色绑定
kubectl create clusterrolebinding system:kube-proxy \
--clusterrole system:node-proxier \
--serviceaccount kube-system:kube-proxy

12.2 kubeconfig 配置

JWT_TOKEN=$(kubectl -n kube-system get secret/kube-proxy-token --output=jsonpath='{.data.token}' | base64 -d)
KUBE_CONFIG="/etc/kubernetes/kube-proxy.conf"
KUBE_APISERVER="https://192.168.10.10:6443"kubectl config set-cluster kubernetes \
--certificate-authority=/etc/kubernetes/pki/ca.pem \
--embed-certs=true \
--server=${KUBE_APISERVER} \
--kubeconfig=${KUBE_CONFIG}kubectl config set-credentials kubernetes \
--token=${JWT_TOKEN} \
--kubeconfig=${KUBE_CONFIG}kubectl config set-context kubernetes \
--cluster=kubernetes \
--user=kubernetes \
--kubeconfig=${KUBE_CONFIG}kubectl config use-context kubernetes \
--kubeconfig=${KUBE_CONFIG}

12.3 kube-proxy yaml配置

cat > /etc/kubernetes/kube-proxy.yaml <<END
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
clientConnection:acceptContentTypes: ""burst: 10contentType: application/vnd.kubernetes.protobufkubeconfig: /etc/kubernetes/kube-proxy.conf  qps: 5
clusterCIDR: 10.244.0.0/16  
configSyncPeriod: 15m0s
conntrack:maxPerCore: 32768min: 131072tcpCloseWaitTimeout: 1h0m0stcpEstablishedTimeout: 24h0m0s
enableProfiling: false
healthzBindAddress: 0.0.0.0:10256
hostnameOverride: ""
iptables:masqueradeAll: falsemasqueradeBit: 14minSyncPeriod: 0ssyncPeriod: 30s
ipvs:minSyncPeriod: 5sscheduler: "rr"syncPeriod: 30s
kind: KubeProxyConfiguration
metricsBindAddress: 127.0.0.1:10249
mode: "ipvs"
nodePortAddresses: null
oomScoreAdj: -999
portRange: ""
udpIdleTimeout: 250ms
END

12.4 服务配置

cat > /usr/lib/systemd/system/kube-proxy.service <<END
[Unit]
Description=Kubernetes Kube Proxy
Documentation=https://github.com/kubernetes/kubernetes
After=network.target[Service]
ExecStart=/usr/local/bin/kube-proxy \--config=/etc/kubernetes/kube-proxy.yaml \--v=2Restart=always
RestartSec=10s[Install]
WantedBy=multi-user.target
END

12.5 启动并设置开机启动

systemctl daemon-reload 
systemctl enable kube-proxy --now

十三、部署 CNI 网络

13.1 下载安装

cd /usr/local/k8s-install/yaml
wget https://docs.projectcalico.org/manifests/calico.yaml --no-check-certificate
# 修改CALICO_IPV4POOL_CIDR  10.244.0.0/16
kubectl apply -f calico.yaml

13.2 查看

kubectl get pods -n kube-system 
kubectl get node

十四、部署CoreDNS

14.1 保持配置

在前面配置中：

/usr/lib/systemd/system/kube-apiserver.service：

--service-cluster-ip-range=10.0.0.0/24

/etc/kubernetes/kubelet-conf.yml中指定

clusterDNS:
- 10.0.0.2
clusterDomain: cluster.local

安装CoreDNS：（所以安装dns时指定-r 10.0.0.0/24 -i 10.0.0.2及domain cluster.local）

#jq支持
yum install -y epel-release
yum install -y jqcd /usr/local/k8s-install/tools/
yum install git -y 
git clone https://github.com/coredns/deployment.git
cd deployment/kubernetes
./deploy.sh -r 10.0.0.0/24 -i 10.0.0.2 -d cluster.local  >  coredns.yaml
kubectl create -f coredns.yaml

十五、新增加 Worker Node

拷贝kubelet和kubeproxy配置及证书，因为在实体机中部署，现在只有单台服务器留到后续更新。。。。

十六、部署nginx测试部署

16.1 部署nginx

kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80 --type=NodePort

16.2 查看

[root@k8s-master01 kubernetes]# kubectl get pod,svc -o wide    
NAME                        READY   STATUS    RESTARTS   AGE     IP              NODE           NOMINATED NODE   READINESS GATES
pod/nginx-8f458dc5b-f7cnd   1/1     Running   0          3m48s   172.16.32.130   k8s-master01   <none>           <none>NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE     SELECTOR
service/kubernetes   ClusterIP   10.0.0.1     <none>        443/TCP        114m    <none>
service/nginx        NodePort    10.0.0.47    <none>        80:30824/TCP   3m47s   app=nginx[root@k8s-master01 kubernetes]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.16.32.128:30824 rr-> 172.16.32.130:80             Masq    1      0          0         
TCP  192.168.10.10:30824 rr-> 172.16.32.130:80             Masq    1      0          1         
TCP  192.168.100.101:30824 rr-> 172.16.32.130:80             Masq    1      0          0         
TCP  10.0.0.1:443 rr-> 192.168.10.10:6443            Masq    1      6          0         
TCP  10.0.0.2:53 rr-> 172.16.32.129:53             Masq    1      0          0         
TCP  10.0.0.2:9153 rr-> 172.16.32.129:9153           Masq    1      0          0         
TCP  10.0.0.47:80 rr-> 172.16.32.130:80             Masq    1      0          0         
TCP  10.88.0.1:30824 rr-> 172.16.32.130:80             Masq    1      0          0         
UDP  10.0.0.2:53 rr-> 172.16.32.129:53             Masq    1      0          0

网络转发分析：

192.168.10.10:30824->172.16.32.130:80

16.3 http访问

[root@k8s-master01 kubernetes]# curl http://192.168.10.10:30824/
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p><p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p>
</body>
</html>

注意：网上文章大多时各种抄袭，可能都没有验证，所以自己实操部署方式，并且记录验证过后的步骤，希望分享出来帮助需要的朋友

参考文章：
自建高可用k8s集群搭建
PKI 证书和要求 | Kubernetes

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.rhkb.cn/news/172150.html

如若内容造成侵权/违法违规/事实不符，请联系长河编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

ARM 汇编指令 orreq 的使用

ARM 汇编指令 orreq 的使用

orreq 阅读代码时&#xff0c;发现有个【组合指令】 orreq&#xff0c; orr 一般是 OR&#xff0c;也就是或操作&#xff0c;后面加个 eq 表示什么呢&#xff1f; 比如下面的代码&#xff1a;前面一个操作&#xff0c; tst&#xff0c;好像没做实际的操作&#xff0c;可能影响…
阅读更多...
[AutoSAR系列] 1.3 AutoSar 架构

[AutoSAR系列] 1.3 AutoSar 架构

依AutoSAR及经验辛苦整理&#xff0c;原创保护&#xff0c;禁止转载。 专栏 《深入浅出AutoSAR》 1. 整体架构 ​ 图片来源&#xff1a; AutoSar 官网 从官往图中可以看出autosar作为汽车ECU软件架构&#xff0c;是通过分层来实现软硬件隔离。就像大多数操作系统一样&#xff…
阅读更多...
Docker swarm集群之compose启动多服务

Docker swarm集群之compose启动多服务

Docker swarm集群之compose启动多服务 本篇文章是在搭建过Swarm集群基础上进行的&#xff0c;如未搭建过请移步 &#xff1a; [Docker swarm 集群搭建 - Wanwan’s Blog (wanwancloud.cn)] 环境信息 主机名IP主机配置master10.10.10.32c2gnode0110.10.10.42c2gnode0210.10.…
阅读更多...
项目中拖拽元素，可以使用html的draggable属性，当然也可以用第三方插件interact

项目中拖拽元素，可以使用html的draggable属性，当然也可以用第三方插件interact

项目中拖拽元素&#xff0c;可以使用html的draggable属性&#xff0c;当然也可以用第三方插件interact 一、安装二、引用三、使用 一、安装 npm install interactjs二、引用 import interact from interactjs三、使用 <div class"drag_box"> &…
阅读更多...
DevOps持续集成-Jenkins（1）

DevOps持续集成-Jenkins（1）

文章目录 DevOpsDevOps概述Code阶段工具&#xff08;centos7-gitlab主机&#xff09;Windows下安装Git&#xff08;作用是&#xff1a;使我们可以上传代码到GitLab&#xff09;Linux下安装GitLab⭐&#xff08;作用是&#xff1a;运行一个GitLab接收代码&#xff09;环境准备先…
阅读更多...
【axios】axios的基本使用

【axios】axios的基本使用

一、 Axios简介 1、 Axios是什么&#xff1f; Axios是一个基于promise的HTTP库&#xff0c;类似于jQuery的ajax&#xff0c;用于http请求。可以应用于浏览器端和node.js&#xff0c;既可以用于客户端&#xff0c;也可以用于node.js编写的服务端。 2.、Axios特性 支持Promis…
阅读更多...
一文讲明：企业知识库的作用和搭建方法

一文讲明：企业知识库的作用和搭建方法

在现代商务环境中&#xff0c;企业面临着大量的信息和知识流动。这些信息和知识散落在各个部门、团队甚至个人之间&#xff0c;难以进行有效的整合和利用。而企业知识库的出现解决了这一问题。它提供了一个统一的平台&#xff0c;将分散的信息汇聚到一个集中的数据库中&#xf…
阅读更多...
Redis实现方式开启新篇章，解决分布式环境下的资源竞争问题，提升系统稳定性

Redis实现方式开启新篇章，解决分布式环境下的资源竞争问题，提升系统稳定性

前言 分布式锁一般有三种实现方式&#xff1a; 数据库乐观锁&#xff1b;基于Redis的分布式锁&#xff1b;基于ZooKeeper的分布式锁 本篇博客将介绍第二种方式&#xff0c;基于Redis实现分布式锁。 虽然网上已经有各种介绍Redis分布式锁实现的博客&#xff0c;然而他们的实…
阅读更多...
jenkins如何安装？

jenkins如何安装？

docker pull jenkins/jenkins:lts-centos7-jdk8 2.docker-compose.yml version: 3 services:jenkins:image: jenkins/jenkins:lts-centos7-jdk8container_name: my-jenkinsports:- "8080:8080" # 映射 Jenkins Web 界面端口volumes:- jenkins_home:/var/jenkins_h…
阅读更多...
C++数据结构X篇_21_插入排序（稳定的排序）

C++数据结构X篇_21_插入排序（稳定的排序）

文章目录 1. 插入排序原理2. 算法图解3. 核心代码&#xff1a;4. 插入排序整体代码实现 1. 插入排序原理 插入排序是一种最简单直观的排序算法&#xff0c;它的工作原理是通过构建有序序列&#xff0c;对于未排序数据&#xff0c;在已排序序列中从后向前扫描&#xff0c;找到相…
阅读更多...
toluaframework中C#怎么调用Lua的方法以及无GC方法

toluaframework中C#怎么调用Lua的方法以及无GC方法

toluaframework中C#怎么调用Lua的方法 问题Util.CallMethodLuaManager.CallFunctionLuaFunction.LazyCall 解决方案LuaFunction脚本无GC消耗的调用 用法总结 问题 用过luaframework框架的人应该都知道框架提供了Util的工具类&#xff0c;工具类提供了一个方法就是Util.CallMet…
阅读更多...
可自由搭建的能源管理平台，轻松实现高效节能

可自由搭建的能源管理平台，轻松实现高效节能

随着科技的不断发展&#xff0c;能源问题越来越重要。为了提高能源的利用效率&#xff0c;减少能源浪费&#xff0c;能源用能企业纷纷开始注重能源管理工作&#xff0c;并想要一款可以进行高效管理的工具。智慧能源管理平台&#xff0c;是一款可自由搭建的能源管理平台&#xf…
阅读更多...
零基础Linux_23(多线程)线程安全+线程互斥(加锁)+死锁

零基础Linux_23(多线程)线程安全+线程互斥(加锁)+死锁

目录 1. 线程安全 1.1 线程不安全前期 1.2 线程不安全原因 2. 线程互斥 2.1 加锁保护&#xff08;代码&#xff09; 2.2 锁的本质 3. 可重入对比线程安全 4. 死锁 4.1 死锁的必要条件 4.2 避免死锁 5. 笔试面试题 答案及解析 本篇完。 1. 线程安全 基于上一篇线程…
阅读更多...
改善游戏体验：数据分析与可视化的威力

改善游戏体验：数据分析与可视化的威力

当今&#xff0c;电子游戏已经超越了娱乐&#xff0c;成为一种文化现象&#xff0c;汇聚了全球数十亿的玩家。游戏制作公司正采用越来越复杂的技术来提高游戏质量&#xff0c;同时游戏数据分析和可视化工具变得不可或缺。 数据的力量&#xff1a;解析游戏体验 游戏制作涉及到大…
阅读更多...
C51--单片机中断

C51--单片机中断

51单片机是单线程模式&#xff0c;需要用到硬件中断。 一、中断系统 中断系统是为使CPU具有对外界紧急事件的实时处理能力而设置的。 当中央处理器CPU正在处理某件事的时候&#xff0c;外界发生了紧急事件请求&#xff0c;要求CPU暂停当前工作&#xff0c;转而去处理这个紧急…
阅读更多...
软考高项-计算题（3）

软考高项-计算题（3）

题10 问题一 EV50*0.525 问题二 EACBAC/CPI CPIEV/AC25/28 EAC50*28/2556 问题三 因为CPI<1&#xff0c;所以项目实际费用超支 题11 PV2000500010000750006500020000177000 AC2100450012000860006000015000179600 EV200050001000075000*0.965000*0.720000*0.351370…
阅读更多...
CANOE 仿真+测试

CANOE 仿真+测试

仿真测试 CANoe的自动化测试系统简介Canoe TFS常用函数测试判别函数测试架构函数测试报告函数检测函数 创建自动化测试工程其他常用函数 CANoe的自动化测试系统简介 基于CANoe的自动化测试系统架构&#xff0c;根据ECU的测试环境和测试规范&#xff0c;搭建基于CANoe的测试系统…
阅读更多...
【置顶帖】关于博主/关于博客/博客大事记

【置顶帖】关于博主/关于博客/博客大事记

关于博主 ● 信息安全从业者 ● 注册信息安全认证专家资质 ● CSDN认证业界专家、安全博客专家 、全栈安全领域优质创作者 ● 中国信通院【2021-GOLF IT新治理领导力论坛】演讲嘉宾 ● 安世加【2021-EISS企业信息安全峰会-上海】演讲嘉宾 ● CSDN【2022-隐私计算论坛】演讲嘉宾…
阅读更多...
uni-app打包apk实现自动更新

uni-app打包apk实现自动更新

一、直接复制粘贴就可用(豪横) app.vue文件里写 //app.vue里写 <script>export default {onShow: function() {console.log(App Show)},onHide: function() {console.log(App Hide)},onLaunch: function() {let appVersion uni.getSystemInfo({success: function(e) {ap…
阅读更多...
springboot在线招聘系统

springboot在线招聘系统

springboot在线招聘管理系统&#xff0c;java在线招聘管理系统&#xff0c;在线招聘管理系统 运行环境&#xff1a; JAVA版本&#xff1a;JDK1.8 IDE类型&#xff1a;IDEA、Eclipse都可运行 数据库类型&#xff1a;MySql&#xff08;8.x版本都可&#xff09; 硬件环境&#xf…
阅读更多...
最新文章
推荐文章

Copyright @ 2022~2023

友情链接： 我的编程经验分享 一条长河网 尧图网站开发