文章目录
- 简介
- 一.条件及环境说明
- 4.2.创建configmap配置
- 4.3.创建statefulset和service headless配置
- 4.4.授权配置
- 4.5.创建service配置
- 五.安装完后的配置
- 六.安装说明
简介
该文搭建的rabbitmq集群是采用rabbitmq_peer_discovery_k8s的形式进行搭建,是通过该插件自动从k8s api中读取节点信息并组建rabbitmq集群,采用搭建方式是一个statefulset,三个副本的形式,所以要保证数据的持久可以采用hostpath+节点亲和性,也可以采用pvc的形式,本文将讲述采用pvc的形式来报障数据的持久化。
一.条件及环境说明
k8s版本k8s-1.29.4,采用共享存储的形式搭建,共享存储采用自建的root-ceph的形式,如果是采用阿里云的集群可以通过挂载硬盘、挂载nas或者对象存储来实现数据持久化,采用共享存储的缺点就是普遍存在读写性能较差的问题。
4.2.创建configmap配置
将如下配置保存到autotest-rabbitmq-config.yaml,配置中配置默认的vhost和用户密码,并初始化好集群节点【安装前规划好节点名称】信息。
apiVersion: v1
kind: ConfigMap
metadata:name: autozx-rabbitmq-confignamespace: zx-applabels:appname: pcauto-zxapp: autozx-rabbitmq-config
data:enabled_plugins: |[rabbitmq_management,rabbitmq_peer_discovery_k8s].rabbitmq.conf: |cluster_name = autozx-rabbitmqlisteners.tcp.default = 5672default_vhost = /default_user = admindefault_pass = pconlinedefault_user_tags.administrator = truedefault_user_tags.management = truedefault_user_tags.custom_tag = truechannel_max = 1024tcp_listen_options.backlog = 2048cluster_formation.peer_discovery_backend = rabbit_peer_discovery_k8scluster_formation.k8s.host = kubernetes.default.svc.cluster.localcluster_formation.k8s.address_type = hostnamecluster_formation.node_cleanup.interval = 30cluster_formation.node_cleanup.only_log_warning = truecluster_partition_handling = autohealqueue_master_locator=min-mastersloopback_users.guest = falsecluster_formation.k8s.hostname_suffix = .autozx-rabbitmq.zx-app.svc.cluster.local
4.3.创建statefulset和service headless配置
采用3副本
apiVersion: apps/v1
kind: StatefulSet
metadata:name: autozx-rabbitmq namespace: zx-app labels:appname: pcauto-zxapp: autozx-rabbitmq
spec:serviceName: "autozx-rabbitmq"replicas: 3selector:matchLabels:app: autozx-rabbitmqtemplate:metadata:labels:app: autozx-rabbitmqspec:containers:- name: rabbitmq-serverimage: pcgroup-registry-vpc.cn-shenzhen.cr.aliyuncs.com/public/rabbitmq:3.12.14-management imagePullPolicy: IfNotPresentenv:- name: RABBITMQ_ERLANG_COOKIEvalue: "YZSDHWMFSMKEMBDHSGGZ"- name: K8S_SERVICE_NAMEvalue: autozx-rabbitmq - name: POD_IPvalueFrom:fieldRef:fieldPath: status.podIP- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: RABBITMQ_USE_LONGNAMEvalue: "true"- name: RABBITMQ_NODENAMEvalue: rabbit@$(POD_NAME).$(K8S_SERVICE_NAME).$(POD_NAMESPACE).svc.cluster.localports:- name: httpcontainerPort: 15672 - name: amqpcontainerPort: 5672 readinessProbe:exec:command:- rabbitmq-diagnostics- statusinitialDelaySeconds: 20periodSeconds: 60timeoutSeconds: 10volumeMounts:- name: rbmq-datamountPath: /var/lib/rabbitmq- name: rabbitmq-config-volumemountPath: /etc/rabbitmq/ restartPolicy: Always serviceAccountName: rabbitmq-clusterterminationGracePeriodSeconds: 30volumes:- name: rabbitmq-config-volumeconfigMap:name: autozx-rabbitmq-configvolumeClaimTemplates:- metadata:name: rbmq-dataspec:accessModes:- ReadWriteManystorageClassName: example-storageclassresources:requests:storage: 20Gi
---
apiVersion: v1
kind: Service
metadata:name: autozx-rabbitmqnamespace: zx-applabels:appname: pcauto-zxapp: autozx-rabbitmq
spec:ports:- port: 5672clusterIP: Noneselector:app: autozx-rabbitmq
注:如果为了避免相同的pod分配到同一节点可以加上如下配置:
affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- autozx-rabbitmqtopologyKey: "kubernetes.io/hostname"
4.4.授权配置
配置serviceAccount、role、RoleBinding来给statefulset进行授权读取节点信息
apiVersion: v1
kind: Service
metadata:name: autozx-rabbitmq-managenamespace: zx-applabels:app: autozx-rabbitmq-manageappname: pcauto-zx
spec:ports:- port: 5672name: amqp- port: 15672name: httpselector:app: autozx-rabbitmqtype: LoadBalancer
[root@autobbs-docker-240-213 rabbitmq]# cat rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:name: rabbitmq-clusternamespace: zx-app
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: rabbitmq-clusternamespace: zx-app
rules:
- apiGroups: [""]resources: ["endpoints"]verbs: ["get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: rabbitmq-clusternamespace: zx-app
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: rabbitmq-cluster
subjects:
- kind: ServiceAccountname: rabbitmq-clusternamespace: zx-app
4.5.创建service配置
给amqp端口5672和管理端口15672配置一个service的代理服务.
apiVersion: v1
kind: Service
metadata:name: autozx-rabbitmq-managenamespace: zx-applabels:app: autozx-rabbitmq-manageappname: pcauto-zx
spec:ports:- port: 5672name: amqp- port: 15672name: httpselector:app: autozx-rabbitmqtype: LoadBalancer
五.安装完后的配置
将rabbitmq的集群设置成3节点的镜像集群,在4.4步骤操作完以后就可以通过loadbalancer的IP进行登录控制台了:http://ip:15672,用configmap中设置的 default_user和default_pass来登录。
镜像模式设置:
设置后:
镜像模式可以通过命令设置:
设置demo的vhost的镜像执行命令:
rabbitmqctl set_policy -p demo ha-all "^" '{"ha-mode":"all","ha-sync-mode":"automatic"}'
设置默认vhost /的镜像模式:
rabbitmqctl set_policy ha-all "^" '{"ha-mode":"all","ha-sync-mode":"automatic"}'
六.安装说明
- 使用配置前将配置中的autozx替换成自己需要的命名,配置的命名空间为:zx-app,修改成自己的命名空间,appname为pcauto-zx,修改成自己的应用名称,该标签也可以删除。
- 配置中的镜像地址采用了私有的镜像地址,镜像是dockerhub上下载的rabbitmq:3.12.14-management 镜像到私有镜像仓库的,如果能直通外网的k8s可以直接用rabbitmq:3.12.14-management 。否则配置一个可以通的镜像地址。
- 默认的用户名和密码是admin。
- pvc模板中的存储类:example-storageclass 修改成自己的存储类
- "五.安装后的配置"由于配置方法都是一样,就采用了上一篇文章中的截图