目录
一、整体架构
二、部署MySQL主从
三、部署Redis哨兵
四、部署WordPress
五、注意事项
一、整体架构
本项目为在一主三从的Kubernetes集群上部署WordPress博客。因为WordPress部分容器版本自行集成Apache和PHP服务,因此在Kubernetes上部署WordPress只需提供MySQL存储数据和Redis缓存Session会话即可。Kubernetes集群基础架构如下:
#k8s-master01 172.29.7.10
#k8s-node01 172.29.7.11
#k8s-node02 172.29.7.12
#k8s-node03 172.29.7.13
#nfs-server 172.29.7.20
#参考往期博客部署NFS-CSI-Driver使Kubernetes集群能调用NFS本地服务器
root@k8s-master01:~# cat /etc/hosts 127.0.0.1 localhost 172.29.7.10 k8s-master01.wlm.com k8s-master01 kubeapi.wlm.com kubeapi 172.29.7.11 k8s-node01.wlm.com k8s-node01 172.29.7.12 k8s-node02.wlm.com k8s-node02 172.29.7.13 k8s-node03.wlm.com k8s-node03 root@k8s-master01:~# kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master01 Ready control-plane 17d v1.29.10 k8s-node01 Ready <none> 17d v1.29.10 k8s-node02 Ready <none> 17d v1.29.10 k8s-node03 Ready <none> 17d v1.29.10
二、部署MySQL主从
为了确保数据的高可用和容灾性部署MySQL主从为WordPress提供服务。MySQL为有状态服务应该使用Operator或Statefulset进行部署,此处涉及到主从复制配置复杂,因此简化使用Deployment进行部署。
#创建项目存放位置
root@k8s-master01:~# mkdir -pv wordpress/mysql/
root@k8s-master01:~# cd wordpress/mysql/#创建PVC实现持久化
root@k8s-master01:~/wordpress/mysql# cat mysql-pvc.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata:name: mysql-pvc-masternamespace: wordpress spec:accessModes:- ReadWriteOnceresources:requests:storage: 10GistorageClassName: nfs-csi --- apiVersion: v1 kind: PersistentVolumeClaim metadata:name: mysql-pvc-slavenamespace: wordpress spec:accessModes:- ReadWriteOnceresources:requests:storage: 10GistorageClassName: nfs-csi
#创建Secret存放MySQL密码(密码为wlm123使用base64加密后如下)
root@k8s-master01:~/wordpress/mysql# kubectl create secret generic mysql-secret --from-literal='root-password'='wlm123' --from-literal='db-password'='wlm123' --dry-run=client -o yaml > mysql-secret.yaml root@k8s-master01:~/wordpress/mysql# cat mysql-secret.yaml apiVersion: v1 data:db-password: d2xtMTIzroot-password: d2xtMTIz kind: Secret metadata:namespace: wordpressname: mysql-secret
#创建ConfigMap为MySQL提供配置文件,同时实现主从复制
root@k8s-master01:~/wordpress/mysql# cat mysql-config.yaml apiVersion: v1 kind: ConfigMap metadata:name: mysql-confignamespace: wordpress data:my.cnf: |[mysqld]server-id=1log_bin=/var/lib/mysql/mysql-bin.logbind-address = 0.0.0.0character-set-server = utf8mb4default_authentication_plugin=mysql_native_password[client]default-character-set = utf8mb4 root@k8s-master01:~/wordpress/mysql# cat mysql-config-slave.yaml apiVersion: v1 kind: ConfigMap metadata:name: mysql-config-slavenamespace: wordpress data:my.cnf: |[mysqld]server-id=22log_bin=/var/lib/mysql/mysql-bin.logbind-address = 0.0.0.0character-set-server = utf8mb4default_authentication_plugin=mysql_native_passwordread-only=ONrelay_log=relay-logrelay_log_index=relay-log.index[client]default-character-set = utf8mb4
#可选操作,创建Service Account赋予MySQL服务名称空间管理员身份,确保其有权限读取Secret
root@k8s-master01:~/wordpress/mysql# cat mysql-sc.yaml apiVersion: v1 kind: ServiceAccount metadata:creationTimestamp: nullname: mysql-scnamespace: wordpress root@k8s-master01:~/wordpress/mysql# cat rbac.yaml apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata:namespace: wordpressname: secret-reader rules: - apiGroups: ["*"]resources: ["*"]verbs: ["*"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata:namespace: wordpressname: read-secrets subjects: - kind: ServiceAccountname: mysql-scnamespace: wordpress roleRef:kind: Rolename: secret-readerapiGroup: rbac.authorization.k8s.io
#部署MySQL应用
root@k8s-master01:~/wordpress/mysql# cat mysql-deploy.yaml --- apiVersion: apps/v1 kind: Deployment metadata:name: mysql-masternamespace: wordpress spec:replicas: 1selector:matchLabels:app: mysqlrole: mastertemplate:metadata:labels:app: mysqlrole: masterspec:serviceAccountName: mysql-sccontainers:- name: mysqlimage: mysql:5.7env:- name: MYSQL_ROOT_PASSWORDvalueFrom:secretKeyRef:name: mysql-secretkey: root-password- name: MYSQL_DATABASEvalue: "wp"- name: MYSQL_USERvalue: "wpuser"- name: MYSQL_PASSWORDvalueFrom:secretKeyRef:name: mysql-secretkey: db-passwordports:- containerPort: 3306volumeMounts:- name: mysql-datamountPath: /var/lib/mysql- name: mysql-configmountPath: /etc/mysql/conf.dvolumes:- name: mysql-datapersistentVolumeClaim:claimName: mysql-pvc-master- name: mysql-configconfigMap:name: mysql-config --- apiVersion: v1 kind: Service metadata:name: mysql-masternamespace: wordpress spec:ports:- port: 3306targetPort: 3306selector:app: mysqlrole: master root@k8s-master01:~/wordpress/mysql# cat mysql-deploy-slave.yaml --- apiVersion: apps/v1 kind: Deployment metadata:name: mysql-slavenamespace: wordpress spec:replicas: 1selector:matchLabels:app: mysqlrole: slavetemplate:metadata:labels:app: mysqlrole: slavespec:serviceAccountName: mysql-sccontainers:- name: mysqlimage: mysql:5.7env:- name: MYSQL_ROOT_PASSWORDvalueFrom:secretKeyRef:name: mysql-secretkey: root-password- name: MYSQL_DATABASEvalue: "wp"- name: MYSQL_USERvalue: "wpuser"- name: MYSQL_PASSWORDvalueFrom:secretKeyRef:name: mysql-secretkey: db-passwordports:- containerPort: 3306volumeMounts:- name: mysql-datamountPath: /var/lib/mysql- name: mysql-config-slavemountPath: /etc/mysql/conf.dvolumes:- name: mysql-datapersistentVolumeClaim:claimName: mysql-pvc-slave- name: mysql-config-slaveconfigMap:name: mysql-config-slave --- apiVersion: v1 kind: Service metadata:name: mysql-slavenamespace: wordpress spec:ports:- port: 3306targetPort: 3306selector:app: mysqlrole: slave
#进入MySQL创建复制账户并进行主从配置,确保IO线程和SQL线程正常运行
root@k8s-master01:~/wordpress/mysql# kubectl apply -f . root@k8s-master01:~/wordpress/mysql# kubectl exec -it mysql-master-7ff66bfb86-bvlfg -n wordpress -- mysql -uroot -pwlm123 mysql> create user 'replica'@'%' identified by 'wlm123'; mysql> grant replication slave on *.* to 'replica'@'%'; mysql> flush privileges; mysql> show master status\G; *************************** 1. row ***************************File: mysql-bin.000003Position: 1538Binlog_Do_DB:Binlog_Ignore_DB: Executed_Gtid_Set: 1 row in set (0.00 sec) root@k8s-master01:~/wordpress/mysql# kubectl exec -it mysql-slave-58cf89f5dd-nh4sz -n wordpress -- mysql -uroot -pwlm123 mysql> CHANGE MASTER TO-> MASTER_HOST='mysql-master',-> MASTER_USER='replica',-> MASTER_PASSWORD='wlm123',-> MASTER_LOG_FILE='mysql-bin.000003',-> MASTER_LOG_POS=1538; mysql> START SLAVE; mysql> show slave status\G; *************************** 1. row ***************************Slave_IO_State: Waiting for master to send eventMaster_Host: mysql-masterMaster_User: replicaMaster_Port: 3306Master_Port: 3306Connect_Retry: 60Master_Log_File: mysql-bin.000003Read_Master_Log_Pos: 1538Relay_Log_File: relay-log.000002Relay_Log_Pos: 320Relay_Master_Log_File: mysql-bin.000003Slave_IO_Running: YesSlave_SQL_Running: Yes
三、部署Redis哨兵
为了确保Redis故障自动转移,使用Satefulset配置Redis一主两从三哨兵。
#创建项目目录
root@k8s-master01:~# mkdir wordpress/redis/
root@k8s-master01:~# cd wordpress/redis/#创建PVC实现持久化
root@k8s-master01:~/wordpress/redis# cat redis-pvc.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata:name: redis-pvcnamespace: wordpress spec:accessModes:- ReadWriteManyresources:requests:storage: 10GistorageClassName: nfs-csi
#创建ConfigMap为Redis提供配置文件
root@k8s-master01:~/wordpress/redis# cat redis-config.yaml apiVersion: v1 kind: ConfigMap metadata:name: redis-confignamespace: wordpress data:redis.conf: |bind 0.0.0.0masterauth 123456requirepass 123456port 6379protected-mode noslave-read-only yes root@k8s-master01:~/wordpress/redis# cat redis-sen-config.yaml apiVersion: v1 kind: ConfigMap metadata:name: redis-sentinel-confignamespace: wordpress data:sentinel.conf: |port 26379sentinel monitor mymaster redis-0.redis-svc.wordpress.svc.cluster.local 6379 2sentinel down-after-milliseconds mymaster 5000sentinel failover-timeout mymaster 10000sentinel auth-pass mymaster 123456sentinel resolve-hostnames yes
#部署HeadLess Service使Redis各Pod可通过名称相互解析IP地址
root@k8s-master01:~/wordpress/redis# cat redis-svc.yaml --- apiVersion: v1 kind: Service metadata:name: redis-svcnamespace: wordpress spec:clusterIP: Noneports:- port: 6379targetPort: 6379selector:app: redis --- apiVersion: v1 kind: Service metadata:name: redis-sentinelnamespace: wordpress spec:clusterIP: Noneports:- port: 26379targetPort: 26379selector:app: redis-sentinel
#部署redis应用
root@k8s-master01:~/wordpress/redis# cat redis.yaml apiVersion: apps/v1 kind: StatefulSet metadata:name: redisnamespace: wordpress spec:serviceName: redis-svcreplicas: 3selector:matchLabels:app: redistemplate:metadata:labels:app: redisspec:containers:- name: redisimage: redis:6.2-alpineports:- containerPort: 6379volumeMounts:- name: redis-datamountPath: /data- name: redis-configmountPath: /usr/local/etc/rediscommand:- sh- -c- |cp /usr/local/etc/redis/redis.conf /data/redis.conf && \redis-server /data/redis.confvolumes:- name: redis-configconfigMap:name: redis-config- name: redis-datapersistentVolumeClaim:claimName: redis-pvc root@k8s-master01:~/wordpress/redis# cat redis-sen.yaml apiVersion: apps/v1 kind: StatefulSet metadata:name: redis-sentinelnamespace: wordpress spec:serviceName: redis-sentinelreplicas: 3selector:matchLabels:app: redis-sentineltemplate:metadata:labels:app: redis-sentinelspec:containers:- name: redis-sentinelimage: redis:6.2-alpineports:- containerPort: 26379volumeMounts:- name: redis-sentinel-configmountPath: /usr/local/etc/redisreadOnly: true- name: redis-sen-datamountPath: /datacommand:- sh- -c- |cp /usr/local/etc/redis/sentinel.conf /data/sentinel.conf && \redis-sentinel /data/sentinel.confvolumes:- name: redis-sentinel-configconfigMap:name: redis-sentinel-config- name: redis-sen-datapersistentVolumeClaim:claimName: redis-pvc
#手动配置Redis的主从复制,进入哨兵Pod确保哨兵状态正常
root@k8s-master01:~/wordpress/redis# kubectl apply -f redis-pvc.yaml root@k8s-master01:~/wordpress/redis# kubectl apply -f redis-config.yaml root@k8s-master01:~/wordpress/redis# kubectl apply -f redis-sen-config.yaml root@k8s-master01:~/wordpress/redis# kubectl apply -f redis-svc.yaml root@k8s-master01:~/wordpress/redis# kubectl apply -f redis.yaml root@k8s-master01:~/wordpress/redis# kubectl exec -it redis-1 -n wordpress -- sh /data # redis-cli -a 123456 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 127.0.0.1:6379> replicaof redis-0.redis-svc.wordpress.svc.cluster.local 6379 OK 127.0.0.1:6379> exit /data # exit root@k8s-master01:~/wordpress/redis# kubectl exec -it redis-2 -n wordpress -- sh /data # redis-cli -a 123456 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 127.0.0.1:6379> replicaof redis-0.redis-svc.wordpress.svc.cluster.local 6379 OK root@k8s-master01:~/wordpress/redis# kubectl exec -it redis-0 -n wordpress -- sh /data # redis-cli -a 123456 Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 127.0.0.1:6379> info replication # Replication role:master connected_slaves:2 slave0:ip=10.244.3.38,port=6379,state=online,offset=70,lag=1 slave1:ip=10.244.2.38,port=6379,state=online,offset=70,lag=1 master_failover_state:no-failover master_replid:09a96f14ebbcf02e358bd675d853dc5797743af5 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:70 second_repl_offset:-1 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:1 repl_backlog_histlen:70 root@k8s-master01:~/wordpress/redis# kubectl exec -it redis-sentinel-0 -n wordpress -- sh /data # redis-cli -p 26379 127.0.0.1:26379> info sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=10.244.1.61:6379,slaves=2,sentinels=3
四、部署WordPress
WordPress为无状态应用使用Deployment即可完成部署,可根据业务情况进行控制副本数量。
#创建项目目录
root@k8s-master01:~# mkdir wordpress/deploy/
root@k8s-master01:~# cd wordpress/deploy/#创建PVC实现持久化并部署应用
root@k8s-master01:~/wordpress/deploy# cat pvc.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata:name: wordpress-pvcnamespace: wordpress spec:accessModes:- ReadWriteOnceresources:requests:storage: 10GistorageClassName: nfs-csi root@k8s-master01:~/wordpress/deploy# cat wordpress.yaml apiVersion: apps/v1 kind: Deployment metadata:name: wordpressnamespace: wordpress spec:replicas: 2selector:matchLabels:app: wordpresstemplate:metadata:labels:app: wordpressspec:containers:- name: wordpressimage: wordpress:6.7-php8.1-apacheenv:- name: WORDPRESS_DB_HOSTvalue: "mysql-master:3306"- name: WORDPRESS_DB_USERvalue: "wpuser"- name: WORDPRESS_DB_PASSWORDvalue: "wlm123"- name: WORDPRESS_DB_NAMEvalue: "wp"- name: WORDPRESS_REDIS_HOSTvalue: "redis-svc:6379"ports:- containerPort: 80volumeMounts:- name: wordpress-storagemountPath: /var/www/htmlvolumes:- name: wordpress-storagepersistentVolumeClaim:claimName: wordpress-pvc --- apiVersion: v1 kind: Service metadata:name: wordpressnamespace: wordpress spec:type: LoadBalancerselector:app: wordpressports:- protocol: TCPport: 80targetPort: 80 root@k8s-master01:~/wordpress/deploy# kubectl get svc -n wordpress NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE mysql-master ClusterIP 10.104.10.121 <none> 3306/TCP 16d mysql-slave ClusterIP 10.104.11.245 <none> 3306/TCP 16d redis-sentinel ClusterIP None <none> 26379/TCP 3d1h redis-svc ClusterIP None <none> 6379/TCP 3d1h wordpress LoadBalancer 10.101.6.62 172.29.7.52 80:31292/TCP 15d
#通过LoadBalance暴露的External-IP访问Web页面进行配置
#为WordPress配置Ingress服务,方便外部流量管理
root@k8s-master01:~/wordpress/deploy# kubectl create ingress wordpress --rule="wordpress.wlm.com/*"=wordpress:80 --class=nginx -n wordpress --dry-run=client -o yaml > ingress-wordpress.yaml root@k8s-master01:~/wordpress/deploy# cat ingress-wordpress.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata:creationTimestamp: nullname: wordpressnamespace: wordpress spec:ingressClassName: nginxrules:- host: wordpress.wlm.comhttp:paths:- backend:service:name: wordpressport:number: 80path: /pathType: Prefix status:loadBalancer: {} root@k8s-master01:~/wordpress/deploy# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx-controller LoadBalancer 10.107.247.232 172.29.7.51 80:31631/TCP,443:32678/TCP 16d ingress-nginx-controller-admission ClusterIP 10.106.45.66 <none> 443/TCP 16d
#通过客户端修改本地hosts文件使域名指向Ingress的IP进行博客访问
五、注意事项
MySQL主从需要确保各容器内配置的server-id唯一,若扩容MySQL从服务需要手动修改配置确保server-id唯一性。
MySQL配置文件需要修改默认认证插件至mysql_native_password避免密码验证失败。
Redis使用6.2版本时需要再Sentinel配置文件内加参数sentinel resolve-hostnames yes,否则无法解析pod名称,即使添加无头服务也不能解析,使用kubectl log pod会有相关提示,在reids-0内使用nslookup命令可验证无头服务已正常生效。
Redis哨兵模式需要修改配置文件,Configmap使用只读挂载时,需要用命令把配置文件复制到持久化的data目录,使用sentinel命令指定data目录下的配置文件