K8s中TSL证书如何续期

TSL是什么

K8s中的作用是什么？

在 Kubernetes（K8s）中，TSL 指的是 Transport Layer Security，也就是传输层安全协议。它是用来保护在网络上传输的数据的安全性和隐私性。

TSL 在 Kubernetes 中的作用包括：

加密通信：
- TSL 确保 Kubernetes 各个组件（如 API Server、etcd、Kubelet 等）之间的通信是加密的，防止未经授权的用户截获敏感信息。
身份验证：
- TSL 证书被用来对客户端和服务器进行身份验证。Kubernetes 的 API Server 通常会使用 TSL 证书来验证客户端（如 kubectl 或其他服务）的身份，确保只有经过认证的用户或服务才能访问集群资源。
数据完整性：
- TSL 确保数据在传输过程中不会被篡改。通过加密哈希技术，它可以验证接收到的数据是否与发送时保持一致，防止攻击者在传输过程中修改数据。
支持 HTTPS：
- Kubernetes API Server 使用 TSL 来支持安全的 HTTPS 连接，而不是不安全的 HTTP。这样确保管理员、开发人员或其他组件通过 API 访问 Kubernetes 时的数据安全性。

总结一下就是：通过使用 TSL，Kubernetes 可以确保集群内的通信安全，防止常见的网络攻击如中间人攻击（MITM）。

K8s中TSL证书的具体使用形式

CA证书（Certificate Authority Certificate）：这是用于签发其他证书的根证书。它通常由集群管理员生成，用于签署 Kubernetes 集群中的各类证书，包括 API Server、kubelet 等组件。CA 证书的有效期一般较长（通常为 10 年），主要是因为更换 CA 证书的影响较大，涉及到重新签发和更新集群中所有其他证书的过程，维护成本较高。
API Server证书：API Server 是 Kubernetes 集群中最核心的组件之一，它与客户端（如 kubectl）、其他控制组件（如 kube-scheduler 和 kube-controller-manager），以及集群节点通信时需要使用证书进行身份验证。API Server 证书的默认有效期通常为 1 年，主要是出于以下几个考虑：
- 安全性：证书的有效期越短，定期轮换越频繁，能够减少密钥泄露或加密算法被攻破的风险。API Server 作为集群中重要的组件，通信和身份验证的安全性极为重要，因此缩短证书有效期能够减少潜在的安全风险。
- 自动化更新：在现代 Kubernetes 集群中，已经引入了自动证书轮换机制。Kubernetes 会在证书接近过期时自动重新生成并更新证书，因此一年期的证书不会对集群的日常运行造成太大影响。（这块具体对版本的要求是1.8+，具体的详情可以查看一下官网的介绍：https://kubernetes.io/blog/2017/09/kubernetes-18-security-workloads-and/）

K8s中TSL证书如何获取的

在 Kubernetes (K8s) 集群中，证书的生成方式取决于你使用的安装方法和具体的需求。Kubernetes 中的证书主要用于确保集群组件之间的安全通信。通常有两种方式生成和管理证书：自动生成 和 手动生成。

1. 自动生成证书

Kubeadm安装：如果你使用 kubeadm 进行集群安装，Kubernetes 会自动生成和管理集群组件所需的证书。在集群初始化时，kubeadm 会自动创建一个自签名的 CA（Certificate Authority，证书颁发机构）证书，并基于该 CA 生成 API Server、kubelet、etcd 等组件所需的其他证书。这些证书的默认有效期是预定义的，比如：
- CA 证书的有效期通常为 10 年。
- 集群组件（如 API Server）的证书默认有效期为 1 年。
证书自动轮换：在现代 Kubernetes 版本中，集群组件的证书支持自动轮换。例如，API Server 和其他组件的证书接近过期时，Kubernetes 会自动更新证书，无需手动干预。

2. 手动申请和管理证书

自定义证书：如果你的组织有更严格的安全要求或需要使用外部的证书颁发机构（CA）签发的证书，你可以手动生成和管理 Kubernetes 集群所需的证书。例如，使用 openssl 或 cfssl 来生成你自己的证书并配置它们。
- 在这种情况下，你需要确保每个 Kubernetes 组件（如 API Server、Controller Manager、Scheduler、kubelet、etcd）都配置好你手动申请的证书。
- 手动申请证书的过程需要你手动管理证书的颁发、续期和更新，因此维护起来较为复杂。

3. 使用外部 CA

你可以使用组织内部的证书颁发机构（如企业内部的 PKI 基础设施）或公开的 CA 来签发 Kubernetes 组件所需的证书。在这种情况下，你需要将 Kubernetes 中的组件配置为使用由该外部 CA 签发的证书，而不是使用 kubeadm 自动生成的证书。

如何进行续期

查看证书的过期时间

ca证书有效期

openssl x509 -in /etc/kubernetes/pki/ca.crt -noout -text  |grep Not

apiserver证书有效期

openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text  |grep Not

如何延长过期时间

`update-kubeadm-cert.sh`文件上传到K8s的Master节点

脚本的下载地址：https://github.com/yuyicai/update-kube-cert

脚本命令

#!/usr/bin/env bashset -o errexit
set -o pipefail
# set -o xtrace# set output color
NC='\033[0m'
RED='\033[31m'
GREEN='\033[32m'
YELLOW='\033[33m'
BLUE='\033[34m'
# set default cri
CRI="docker"log::err() {printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${RED}ERROR${NC}] %b\n" "$@"
}log::info() {printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][INFO] %b\n" "$@"
}log::warning() {printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${YELLOW}WARNING${NC}] \033[0m%b\n" "$@"
}check_file() {if [[ ! -r ${1} ]]; thenlog::err "can not find ${1}"exit 1fi
}# get x509v3 subject alternative name from the old certificate
cert::get_subject_alt_name() {local cert=${1}.crtlocal alt_namecheck_file "${cert}"alt_name=$(openssl x509 -text -noout -in "${cert}" | grep -A1 'Alternative' | tail -n1 | sed 's/[[:space:]]*Address//g')printf "%s\n" "${alt_name}"
}# get subject from the old certificate
cert::get_subj() {local cert=${1}.crtlocal subjcheck_file "${cert}"subj=$(openssl x509 -text -noout -in "${cert}" | grep "Subject:" | sed 's/Subject:/\//g;s/\,/\//;s/[[:space:]]//g')printf "%s\n" "${subj}"
}cert::backup_file() {local file=${1}if [[ ! -e ${file}.old-$(date +%Y%m%d) ]]; thencp -rp "${file}" "${file}.old-$(date +%Y%m%d)"log::info "backup ${file} to ${file}.old-$(date +%Y%m%d)"elselog::warning "does not backup, ${file}.old-$(date +%Y%m%d) already exists"fi
}# check certificate expiration
cert::check_cert_expiration() {local cert=${1}.crtlocal cert_expirescert_expires=$(openssl x509 -text -noout -in "${cert}" | awk -F ": " '/Not After/{print$2}')printf "%s\n" "${cert_expires}"
}# check kubeconfig expiration
cert::check_kubeconfig_expiration() {local config=${1}.conflocal certlocal cert_expirescert=$(grep "client-certificate-data" "${config}" | awk '{print$2}' | base64 -d)cert_expires=$(openssl x509 -text -noout -in <(printf "%s" "${cert}") | awk -F ": " '/Not After/{print$2}')printf "%s\n" "${cert_expires}"
}# check etcd certificates expiration
cert::check_etcd_certs_expiration() {local certlocal certscerts=("${ETCD_CERT_CA}""${ETCD_CERT_SERVER}""${ETCD_CERT_PEER}""${ETCD_CERT_HEALTHCHECK_CLIENT}""${ETCD_CERT_APISERVER_ETCD_CLIENT}")for cert in "${certs[@]}"; doif [[ ! -r ${cert} ]]; thenprintf "%-50s%-30s\n" "${cert}.crt" "$(cert::check_cert_expiration "${cert}")"fidone
}# check master certificates expiration
cert::check_master_certs_expiration() {local certslocal kubeconfslocal certlocal confcerts=("${CERT_CA}""${CERT_APISERVER}""${CERT_APISERVER_KUBELET_CLIENT}""${FRONT_PROXY_CA}""${FRONT_PROXY_CLIENT}")# add support for super_admin.conf, which was added after k8s v1.30.if [ -f "${CONF_SUPER_ADMIN}.conf" ]; thenkubeconfs=("${CONF_CONTROLLER_MANAGER}""${CONF_SCHEDULER}""${CONF_ADMIN}""${CONF_SUPER_ADMIN}")else kubeconfs=("${CONF_CONTROLLER_MANAGER}""${CONF_SCHEDULER}""${CONF_ADMIN}")fiprintf "%-50s%-30s\n" "CERTIFICATE" "EXPIRES"for conf in "${kubeconfs[@]}"; doif [[ ! -r ${conf} ]]; thenprintf "%-50s%-30s\n" "${conf}.config" "$(cert::check_kubeconfig_expiration "${conf}")"fidonefor cert in "${certs[@]}"; doif [[ ! -r ${cert} ]]; thenprintf "%-50s%-30s\n" "${cert}.crt" "$(cert::check_cert_expiration "${cert}")"fidone
}# check all certificates expiration
cert::check_all_expiration() {cert::check_master_certs_expirationcert::check_etcd_certs_expiration
}# generate certificate whit client, server or peer
# Args:
#   $1 (the name of certificate)
#   $2 (the type of certificate, must be one of client, server, peer)
#   $3 (the subject of certificates)
#   $4 (the validity of certificates) (days)
#   $5 (the name of ca)
#   $6 (the x509v3 subject alternative name of certificate when the type of certificate is server or peer)
cert::gen_cert() {local cert_name=${1}local cert_type=${2}local subj=${3}local cert_days=${4}local ca_name=${5}local alt_name=${6}local ca_cert=${ca_name}.crtlocal ca_key=${ca_name}.keylocal cert=${cert_name}.crtlocal key=${cert_name}.keylocal csr=${cert_name}.csrlocal common_csr_conf='distinguished_name = dn\n[dn]\n[v3_ext]\nkeyUsage = critical, digitalSignature, keyEncipherment\n'for file in "${ca_cert}" "${ca_key}" "${cert}" "${key}"; docheck_file "${file}"donecase "${cert_type}" inclient)csr_conf=$(printf "%bextendedKeyUsage = clientAuth\n" "${common_csr_conf}");;server)csr_conf=$(printf "%bextendedKeyUsage = serverAuth\nsubjectAltName = %b\n" "${common_csr_conf}" "${alt_name}");;peer)csr_conf=$(printf "%bextendedKeyUsage = serverAuth, clientAuth\nsubjectAltName = %b\n" "${common_csr_conf}" "${alt_name}");;*)log::err "unknow, unsupported certs type: ${YELLOW}${cert_type}${NC}, supported type: client, server, peer"exit 1;;esac# gen csropenssl req -new -key "${key}" -subj "${subj}" -reqexts v3_ext \-config <(printf "%b" "${csr_conf}") \-out "${csr}" >/dev/null 2>&1# gen certopenssl x509 -in "${csr}" -req -CA "${ca_cert}" -CAkey "${ca_key}" -CAcreateserial -extensions v3_ext \-extfile <(printf "%b" "${csr_conf}") \-days "${cert_days}" -out "${cert}" >/dev/null 2>&1rm -f "${csr}"
}cert::update_kubeconf() {local cert_name=${1}local kubeconf_file=${cert_name}.conflocal cert=${cert_name}.crtlocal key=${cert_name}.keylocal subjlocal cert_base64check_file "${kubeconf_file}"# get the key from the old kubeconfgrep "client-key-data" "${kubeconf_file}" | awk '{print$2}' | base64 -d >"${key}"# get the old certificate from the old kubeconfgrep "client-certificate-data" "${kubeconf_file}" | awk '{print$2}' | base64 -d >"${cert}"# get subject from the old certificatesubj=$(cert::get_subj "${cert_name}")cert::gen_cert "${cert_name}" "client" "${subj}" "${CERT_DAYS}" "${CERT_CA}"# get certificate base64 codecert_base64=$(base64 -w 0 "${cert}")# set certificate base64 code to kubeconfsed -i 's/client-certificate-data:.*/client-certificate-data: '"${cert_base64}"'/g' "${kubeconf_file}"rm -f "${cert}"rm -f "${key}"
}cert::update_etcd_cert() {local subjlocal subject_alt_namelocal cert# generate etcd server,peer certificate# /etc/kubernetes/pki/etcd/server# /etc/kubernetes/pki/etcd/peerfor cert in ${ETCD_CERT_SERVER} ${ETCD_CERT_PEER}; dosubj=$(cert::get_subj "${cert}")subject_alt_name=$(cert::get_subject_alt_name "${cert}")cert::gen_cert "${cert}" "peer" "${subj}" "${CERT_DAYS}" "${ETCD_CERT_CA}" "${subject_alt_name}"log::info "${GREEN}updated ${BLUE}${cert}.conf${NC}"done# generate etcd healthcheck-client,apiserver-etcd-client certificate# /etc/kubernetes/pki/etcd/healthcheck-client# /etc/kubernetes/pki/apiserver-etcd-clientfor cert in ${ETCD_CERT_HEALTHCHECK_CLIENT} ${ETCD_CERT_APISERVER_ETCD_CLIENT}; dosubj=$(cert::get_subj "${cert}")cert::gen_cert "${cert}" "client" "${subj}" "${CERT_DAYS}" "${ETCD_CERT_CA}"log::info "${GREEN}updated ${BLUE}${cert}.conf${NC}"done# restart etcdcase $CRI in"docker")docker ps | awk '/k8s_etcd/{print$1}' | xargs -r -I '{}' docker restart {} >/dev/null 2>&1 || true;;"containerd")crictl ps | awk '/etcd-/{print$(NF-1)}' | xargs -r -I '{}' crictl stopp {} >/dev/null 2>&1 || true;;esaclog::info "restarted etcd with ${CRI}"
}cert::update_master_cert() {local subjlocal subject_alt_namelocal conf# generate apiserver server certificate# /etc/kubernetes/pki/apiserversubj=$(cert::get_subj "${CERT_APISERVER}")subject_alt_name=$(cert::get_subject_alt_name "${CERT_APISERVER}")cert::gen_cert "${CERT_APISERVER}" "server" "${subj}" "${CERT_DAYS}" "${CERT_CA}" "${subject_alt_name}"log::info "${GREEN}updated ${BLUE}${CERT_APISERVER}.crt${NC}"# generate apiserver-kubelet-client certificate# /etc/kubernetes/pki/apiserver-kubelet-clientsubj=$(cert::get_subj "${CERT_APISERVER_KUBELET_CLIENT}")cert::gen_cert "${CERT_APISERVER_KUBELET_CLIENT}" "client" "${subj}" "${CERT_DAYS}" "${CERT_CA}"log::info "${GREEN}updated ${BLUE}${CERT_APISERVER_KUBELET_CLIENT}.crt${NC}"# generate kubeconf for controller-manager,scheduler and kubelet# /etc/kubernetes/controller-manager,scheduler,admin,kubelet.conf,super_admin(added after k8s v1.30.)if [ -f "${CONF_SUPER_ADMIN}.conf" ]; thenconf_list="${CONF_CONTROLLER_MANAGER} ${CONF_SCHEDULER} ${CONF_ADMIN} ${CONF_KUBELET} ${CONF_SUPER_ADMIN}"else conf_list="${CONF_CONTROLLER_MANAGER} ${CONF_SCHEDULER} ${CONF_ADMIN} ${CONF_KUBELET}"fifor conf in ${conf_list}; doif [[ ${conf##*/} == "kubelet" ]]; then# https://github.com/kubernetes/kubeadm/issues/1753set +egrep kubelet-client-current.pem /etc/kubernetes/kubelet.conf >/dev/null 2>&1kubelet_cert_auto_update=$?set -eif [[ "$kubelet_cert_auto_update" == "0" ]]; thenlog::info "does not need to update kubelet.conf"continuefifi# update kubeconfcert::update_kubeconf "${conf}"log::info "${GREEN}updated ${BLUE}${conf}.conf${NC}"# copy admin.conf to ${HOME}/.kube/configif [[ ${conf##*/} == "admin" ]]; thenmkdir -p "${HOME}/.kube"local config=${HOME}/.kube/configlocal config_backupconfig_backup=${HOME}/.kube/config.old-$(date +%Y%m%d)if [[ -f ${config} ]] && [[ ! -f ${config_backup} ]]; thencp -fp "${config}" "${config_backup}"log::info "backup ${config} to ${config_backup}"ficp -fp "${conf}.conf" "${HOME}/.kube/config"log::info "copy the admin.conf to ${HOME}/.kube/config"fidone# generate front-proxy-client certificate# /etc/kubernetes/pki/front-proxy-clientsubj=$(cert::get_subj "${FRONT_PROXY_CLIENT}")cert::gen_cert "${FRONT_PROXY_CLIENT}" "client" "${subj}" "${CERT_DAYS}" "${FRONT_PROXY_CA}"log::info "${GREEN}updated ${BLUE}${FRONT_PROXY_CLIENT}.crt${NC}"# restart apiserver, controller-manager, scheduler and kubeletfor item in "apiserver" "controller-manager" "scheduler"; docase $CRI in"docker")docker ps | awk '/k8s_kube-'${item}'/{print$1}' | xargs -r -I '{}' docker restart {} >/dev/null 2>&1 || true;;"containerd")crictl ps | awk '/kube-'${item}'-/{print $(NF-1)}' | xargs -r -I '{}' crictl stopp {} >/dev/null 2>&1 || true;;esaclog::info "restarted ${item} with ${CRI}"donesystemctl restart kubelet || truelog::info "restarted kubelet"
}main() {local node_type=$1# read the optionsARGS=`getopt -o c: --long cri: -- "$@"`eval set -- "$ARGS"# extract options and their arguments into variables.while truedocase "$1" in-c|--cri)case "$2" in"docker"|"containerd")CRI=$2shift 2;;*)echo 'Unsupported cri. Valid options are "docker", "containerd".'exit 1;;esac;;--)shiftbreak;;*)echo "Invalid arguments."exit 1;;esacdoneCERT_DAYS=3650KUBE_PATH=/etc/kubernetesPKI_PATH=${KUBE_PATH}/pki# master certificates path# apiserverCERT_CA=${PKI_PATH}/caCERT_APISERVER=${PKI_PATH}/apiserverCERT_APISERVER_KUBELET_CLIENT=${PKI_PATH}/apiserver-kubelet-clientCONF_CONTROLLER_MANAGER=${KUBE_PATH}/controller-managerCONF_SCHEDULER=${KUBE_PATH}/schedulerCONF_ADMIN=${KUBE_PATH}/adminCONF_SUPER_ADMIN=${KUBE_PATH}/super-adminCONF_KUBELET=${KUBE_PATH}/kubelet# front-proxyFRONT_PROXY_CA=${PKI_PATH}/front-proxy-caFRONT_PROXY_CLIENT=${PKI_PATH}/front-proxy-client# etcd certificates pathETCD_CERT_CA=${PKI_PATH}/etcd/caETCD_CERT_SERVER=${PKI_PATH}/etcd/serverETCD_CERT_PEER=${PKI_PATH}/etcd/peerETCD_CERT_HEALTHCHECK_CLIENT=${PKI_PATH}/etcd/healthcheck-clientETCD_CERT_APISERVER_ETCD_CLIENT=${PKI_PATH}/apiserver-etcd-clientcase ${node_type} in# etcd)# # update etcd certificates#   cert::update_etcd_cert# ;;master)# check certificates expirationcert::check_master_certs_expiration# backup $KUBE_PATH to $KUBE_PATH.old-$(date +%Y%m%d)cert::backup_file "${KUBE_PATH}"# update master certificates and kubeconflog::info "${GREEN}updating...${NC}"cert::update_master_certlog::info "${GREEN}done!!!${NC}"# check certificates expiration after certificates updatedcert::check_master_certs_expiration;;all)# check certificates expirationcert::check_all_expiration# backup $KUBE_PATH to $KUBE_PATH.old-$(date +%Y%m%d)cert::backup_file "${KUBE_PATH}"# update etcd certificateslog::info "${GREEN}updating...${NC}"cert::update_etcd_cert# update master certificates and kubeconfcert::update_master_certlog::info "${GREEN}done!!!${NC}"# check certificates expiration after certificates updatedcert::check_all_expiration;;check)# check certificates expirationcert::check_all_expiration;;*)log::err "unknown, unsupported cert type: ${node_type}, supported type: \"all\", \"master\""printf "Documentation: https://github.com/yuyicai/update-kube-certexample:'\033[32m./update-kubeadm-cert.sh all\033[0m' update all etcd certificates, master certificates and kubeconf/etc/kubernetes├── admin.conf├── super-admin.conf├── controller-manager.conf├── scheduler.conf├── kubelet.conf└── pki├── apiserver.crt├── apiserver-etcd-client.crt├── apiserver-kubelet-client.crt├── front-proxy-client.crt└── etcd├── healthcheck-client.crt├── peer.crt└── server.crt'\033[32m./update-kubeadm-cert.sh master\033[0m' update only master certificates and kubeconf/etc/kubernetes├── admin.conf├── super-admin.conf├── controller-manager.conf├── scheduler.conf├── kubelet.conf└── pki├── apiserver.crt├── apiserver-kubelet-client.crt└── front-proxy-client.crt
"exit 1;;esac
}main "$@"

Master节点执行相应脚本

# 给update-kubeadm-cert.sh证书授权可执行权限
chmod +x update-kubeadm-cert.sh

# 执行下面命令，修改证书过期时间，把时间延长到10年
./update-kubeadm-cert.sh all

查询是否签发正常

# 在k8s-master1节点查询Pod是否正常,能查询出数据说明证书签发完成
kubectl get pods -n kube-system

进一步确认是否延期成功

#  查看apiserver证书
[root@master01 ~]# openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text  |grep NotNot Before: Oct 17 07:38:48 2024 GMTNot After : Oct 15 07:38:48 2034 GMT
您在 /var/spool/mail/root 中有新邮件
# 查看etcd证书
[root@master01 ~]# openssl x509 -in /etc/kubernetes/pki/apiserver-etcd-client.crt  -noout -text  |grep NotNot Before: Oct 17 07:38:48 2024 GMTNot After : Oct 15 07:38:48 2034 GMT
# 查看fron-proxy证书
[root@master01 ~]# openssl x509 -in /etc/kubernetes/pki/front-proxy-ca.crt  -noout -text  |grep NotNot Before: Sep 10 07:54:00 2024 GMTNot After : Sep  8 07:54:00 2034 GMT