K8S部署MySQL5.7的主从服务

mysql-slave-0是master
mysql-slave-1是slave
当mysql写的时候，找headless service中的 mysql-slave-0.mysql57-slave-headless；当mysql读的时候，找clusterip service中的mysql57-slave-read读，实现读写分离。
statefulset维护两个副本，一个主一个从
因为主从的配置不同所以通过configmap来解决主从配置不同的问题

创建命名空间

cat mysql-cluster-s.yaml

apiVersion: v1
kind: Namespace
metadata:name: mysql-cluster

创建配置文件

这一配置不适合生产环境。 MySQL 设置都使用的是不安全的默认值，这是因为我们想把重点放在 Kubernetes 中运行有状态应用程序的一般模式上。

# cat configmap.yaml 
apiVersion: v1
kind: ConfigMap
metadata:name: mysql57-slave-cfgnamespace: mysql-clusterlabels:app: mysql57-slave
data:master.cnf: |# Apply this config only on the master.[mysqld]log-bin  # 主mysql激活二进制日志slave.cnf: |# Apply this config only on slaves.[mysqld]super-read-only   # 从mysql上面设置为只读

创建service服务

cat service.yaml

为 StatefulSet 成员提供稳定的 DNS 表项的无头服务（Headless Service）

apiVersion: v1
kind: Service
metadata:name: mysql57-slave-headlessnamespace: mysql-clusterlabels:app: mysql57-slave
spec:ports:- name: mysqlport: 3306clusterIP: Noneselector:app: mysql57-slave
---
# Client service for connecting to any MySQL instance for reads.
# For writes, you must instead connect to the master: mysql-0.mysql.
apiVersion: v1
kind: Service
metadata:name: mysql57-slave-readnamespace: mysql-clusterlabels:app: mysql57-slave
spec:ports:- name: mysqlport: 3306selector:app: mysql57-slave

创建sts服务

apiVersion: apps/v1
kind: StatefulSet
metadata:name: mysql57-slavenamespace: mysql-cluster
spec:selector:matchLabels:app: mysql57-slaveserviceName: mysql57-slave-headlessreplicas: 2template:metadata:labels:app: mysql57-slavespec:initContainers:- name: init-mysqlimage: mysql:5.7#image: mysql:8.0.36command:- bash- "-c"- |set -ex# Generate mysql server-id from pod ordinal index.[[ `tail -n 1 /etc/hosts` =~ -([0-9]+)$ ]] || exit 1#[[ `hostid` =~ -([0-9]+)$ ]] || exit 1ordinal=${BASH_REMATCH[1]}echo [mysqld] > /mnt/conf.d/server-id.cnf# Add an offset to avoid reserved server-id=0 value.echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf# Copy appropriate conf.d files from config-map to emptyDir.if [[ $ordinal -eq 0 ]]; thencp /mnt/config-map/master.cnf /mnt/conf.d/elsecp /mnt/config-map/slave.cnf /mnt/conf.d/fivolumeMounts:- name: confmountPath: /mnt/conf.d- name: config-mapmountPath: /mnt/config-map- name: clone-mysql#image: registry.cn-hangzhou.aliyuncs.com/hxpdocker/xtrabackup:1.0# 该镜像通过附件下载xtrabackup:1.0image: gcr.io/google-samples/xtrabackup:1.0command:- bash- "-c"- |set -ex# Skip the clone if data already exists.[[ -d /var/lib/mysql/mysql ]] && exit 0# Skip the clone on master (ordinal index 0).[[ `tail -n 1 /etc/hosts` =~ -([0-9]+)$ ]] || exit 1#[[ `hostid` =~ -([0-9]+)$ ]] || exit 1ordinal=${BASH_REMATCH[1]}[[ $ordinal -eq 0 ]] && exit 0# Clone data from previous peer.# mysql-$(($ordinal-1)).mysql->mysql57-slave-0.mysql57-slave-headlessncat --recv-only mysql57-slave-$(($ordinal-1)).mysql57-slave-headless 3307 | xbstream -x -C /var/lib/mysql# Prepare the backup.xtrabackup --prepare --target-dir=/var/lib/mysqlvolumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.dcontainers:- name: mysqlimage: mysql:5.7env:- name: MYSQL_ALLOW_EMPTY_PASSWORDvalue: "1"ports:- name: mysqlcontainerPort: 3306volumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.dresources:requests:cpu: 500mmemory: 500MilivenessProbe:exec:command: ["mysqladmin", "ping"]initialDelaySeconds: 30periodSeconds: 10timeoutSeconds: 5readinessProbe:exec:# Check we can execute queries over TCP (skip-networking is off).command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]initialDelaySeconds: 5periodSeconds: 2timeoutSeconds: 1- name: xtrabackup#image: registry.cn-hangzhou.aliyuncs.com/hxpdocker/xtrabackup:1.0#该镜像通过附件下载导入image: gcr.io/google-samples/xtrabackup:1.0ports:- name: xtrabackupcontainerPort: 3307command:- bash- "-c"- |set -excd /var/lib/mysql# Determine binlog position of cloned data, if any.if [[ -f xtrabackup_slave_info && "x$(<xtrabackup_slave_info)" != "x" ]]; then# XtraBackup already generated a partial "CHANGE MASTER TO" query# because we're cloning from an existing slave. (Need to remove the tailing semicolon!)cat xtrabackup_slave_info | sed -E 's/;$//g' > change_master_to.sql.in# Ignore xtrabackup_binlog_info in this case (it's useless).rm -f xtrabackup_slave_info xtrabackup_binlog_infoelif [[ -f xtrabackup_binlog_info ]]; then# We're cloning directly from master. Parse binlog position.[[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1rm -f xtrabackup_binlog_info xtrabackup_slave_infoecho "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.infi# Check if we need to complete a clone by starting replication.if [[ -f change_master_to.sql.in ]]; thenecho "Waiting for mysqld to be ready (accepting connections)"until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; doneecho "Initializing replication from clone position"## 修改名字mysql-$(($ordinal-1)).mysql->mysql57-slave-0.mysql57-slave-headlessmysql -h 127.0.0.1 -e "$(<change_master_to.sql.in), MASTER_HOST='mysql57-slave-0.mysql57-slave-headless', MASTER_USER='root', MASTER_PASSWORD='',MASTER_CONNECT_RETRY=10; START SLAVE;" || exit 1# In case of container restart, attempt this at-most-once.mv change_master_to.sql.in change_master_to.sql.origfi# Start a server to send backups when requested by peers.exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \"xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"volumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.dresources:requests:cpu: 100mmemory: 100Mivolumes:- name: confemptyDir: {}- name: config-mapconfigMap:name: mysql57-slave-cfgvolumeClaimTemplates:- metadata:name: dataspec:accessModes: ["ReadWriteOnce"]#存储类storageClassName: longhornresources:requests:storage: 200Gi

查看mysql的主从状态是否正常

show slave status\G;

完整的yaml文件如下

apiVersion: v1
kind: Namespace
metadata:name: mysql-cluster
---
# cat configmap.yaml 
apiVersion: v1
kind: ConfigMap
metadata:name: mysql57-slave-cfgnamespace: mysql-clusterlabels:app: mysql57-slave
data:master.cnf: |# Apply this config only on the master.[mysqld]log-bin  # 主mysql激活二进制日志slave.cnf: |# Apply this config only on slaves.[mysqld]super-read-only   # 从mysql上面设置为只读
---
# cat service.yaml 
apiVersion: v1
kind: Service
metadata:name: mysql57-slave-headlessnamespace: mysql-clusterlabels:app: mysql57-slave
spec:ports:- name: mysqlport: 3306clusterIP: Noneselector:app: mysql57-slave
---
# Client service for connecting to any MySQL instance for reads.
# For writes, you must instead connect to the master: mysql-0.mysql.
apiVersion: v1
kind: Service
metadata:name: mysql57-slave-readnamespace: mysql-clusterlabels:app: mysql57-slave
spec:ports:- name: mysqlport: 3306selector:app: mysql57-slave
---
# cat statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:name: mysql57-slavenamespace: mysql-cluster
spec:selector:matchLabels:app: mysql57-slaveserviceName: mysql57-slave-headlessreplicas: 2template:metadata:labels:app: mysql57-slavespec:initContainers:- name: init-mysqlimage: mysql:5.7#image: mysql:8.0.36command:- bash- "-c"- |set -ex# Generate mysql server-id from pod ordinal index.[[ `tail -n 1 /etc/hosts` =~ -([0-9]+)$ ]] || exit 1#[[ `hostid` =~ -([0-9]+)$ ]] || exit 1ordinal=${BASH_REMATCH[1]}echo [mysqld] > /mnt/conf.d/server-id.cnf# Add an offset to avoid reserved server-id=0 value.echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf# Copy appropriate conf.d files from config-map to emptyDir.if [[ $ordinal -eq 0 ]]; thencp /mnt/config-map/master.cnf /mnt/conf.d/elsecp /mnt/config-map/slave.cnf /mnt/conf.d/fivolumeMounts:- name: confmountPath: /mnt/conf.d- name: config-mapmountPath: /mnt/config-map- name: clone-mysql#image: registry.cn-hangzhou.aliyuncs.com/hxpdocker/xtrabackup:1.0# 该镜像通过附件下载xtrabackup:1.0image: gcr.io/google-samples/xtrabackup:1.0command:- bash- "-c"- |set -ex# Skip the clone if data already exists.[[ -d /var/lib/mysql/mysql ]] && exit 0# Skip the clone on master (ordinal index 0).[[ `tail -n 1 /etc/hosts` =~ -([0-9]+)$ ]] || exit 1#[[ `hostid` =~ -([0-9]+)$ ]] || exit 1ordinal=${BASH_REMATCH[1]}[[ $ordinal -eq 0 ]] && exit 0# Clone data from previous peer.# mysql-$(($ordinal-1)).mysql->mysql57-slave-0.mysql57-slave-headlessncat --recv-only mysql57-slave-$(($ordinal-1)).mysql57-slave-headless 3307 | xbstream -x -C /var/lib/mysql# Prepare the backup.xtrabackup --prepare --target-dir=/var/lib/mysqlvolumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.dcontainers:- name: mysqlimage: mysql:5.7env:- name: MYSQL_ALLOW_EMPTY_PASSWORDvalue: "1"ports:- name: mysqlcontainerPort: 3306volumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.dresources:requests:cpu: 500mmemory: 500MilivenessProbe:exec:command: ["mysqladmin", "ping"]initialDelaySeconds: 30periodSeconds: 10timeoutSeconds: 5readinessProbe:exec:# Check we can execute queries over TCP (skip-networking is off).command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]initialDelaySeconds: 5periodSeconds: 2timeoutSeconds: 1- name: xtrabackup#image: registry.cn-hangzhou.aliyuncs.com/hxpdocker/xtrabackup:1.0#该镜像通过附件下载导入image: gcr.io/google-samples/xtrabackup:1.0ports:- name: xtrabackupcontainerPort: 3307command:- bash- "-c"- |set -excd /var/lib/mysql# Determine binlog position of cloned data, if any.if [[ -f xtrabackup_slave_info && "x$(<xtrabackup_slave_info)" != "x" ]]; then# XtraBackup already generated a partial "CHANGE MASTER TO" query# because we're cloning from an existing slave. (Need to remove the tailing semicolon!)cat xtrabackup_slave_info | sed -E 's/;$//g' > change_master_to.sql.in# Ignore xtrabackup_binlog_info in this case (it's useless).rm -f xtrabackup_slave_info xtrabackup_binlog_infoelif [[ -f xtrabackup_binlog_info ]]; then# We're cloning directly from master. Parse binlog position.[[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1rm -f xtrabackup_binlog_info xtrabackup_slave_infoecho "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.infi# Check if we need to complete a clone by starting replication.if [[ -f change_master_to.sql.in ]]; thenecho "Waiting for mysqld to be ready (accepting connections)"until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; doneecho "Initializing replication from clone position"## 修改名字mysql-$(($ordinal-1)).mysql->mysql57-slave-0.mysql57-slave-headlessmysql -h 127.0.0.1 -e "$(<change_master_to.sql.in), MASTER_HOST='mysql57-slave-0.mysql57-slave-headless', MASTER_USER='root', MASTER_PASSWORD='',MASTER_CONNECT_RETRY=10; START SLAVE;" || exit 1# In case of container restart, attempt this at-most-once.mv change_master_to.sql.in change_master_to.sql.origfi# Start a server to send backups when requested by peers.exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \"xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"volumeMounts:- name: datamountPath: /var/lib/mysqlsubPath: mysql- name: confmountPath: /etc/mysql/conf.dresources:requests:cpu: 100mmemory: 100Mivolumes:- name: confemptyDir: {}- name: config-mapconfigMap:name: mysql57-slave-cfgvolumeClaimTemplates:- metadata:name: dataspec:accessModes: ["ReadWriteOnce"]#存储类storageClassName: longhornresources:requests:storage: 200Gi

参考链接：

https://kubernetes.io/zh-cn/docs/tasks/run-application/run-replicated-stateful-application/

https://blog.51cto.com/u_13045706/10035400