k8s 1.28.2 集群部署 docker registry 接入 MinIO 存储

文章目录

@[toc]
docker registry 部署
生成 htpasswd 文件
生成 secret 文件

生成 registry 配置文件
创建 service
创建 statefulset
创建 ingress
验证 docker registry

docker registry 监控
docker registry ui

docker registry dockerfile

docker registry 配置文件

S3 storage driver

registry:2.8.3 Image hierarchy

docker registry 部署

生成 htpasswd 文件

<username> <password> 改成自己想配置的，如果密码有特殊字符，要用单引号包起来

docker run --rm \docker.m.daocloud.io/httpd:latest \htpasswd -Bbn <username> <password> > htpasswd

生成 secret 文件

kubectl create secret generic docker-registry-auth \-n registry \--from-file=htpasswd

生成 registry 配置文件

因为涉及到 MinIO 的 accesskey 和 secretkey，这里采用 secret 的方式来生成配置文件

---
apiVersion: v1
kind: Secret
metadata:name: docker-registry-cmnamespace: registry
stringData:config.yml: |-version: 0.1log:level: infofields:service: registrystorage:delete:enabled: truecache:blobdescriptor: inmemorys3:accesskey: wJpkHB8rznvZBRLfKmBzsecretkey: ZHIyklv5tktYvGR0iFqBiL9NKh7JKbhyDR9SNAYpregion: defaultregionendpoint: http://minio.api.devops.icuforcepathstyle: trueaccelerate: falsebucket: docker-registryencrypt: falsesecure: falsev4auth: truechunksize: 5242880multipartcopymaxconcurrency: 10http:addr: :5000debug:addr: :5001prometheus:enabled: truepath: /metricsheaders:X-Content-Type-Options: [nosniff]health:storagedriver:enabled: trueinterval: 10sthreshold: 3auth:htpasswd:realm: basic-realmpath: /auth/htpasswd
type: Opaque

创建 service

---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/name: docker-registryname: docker-registry-svcnamespace: registry
spec:ports:- name: httpport: 5000targetPort: http- name: http-metricsport: 5001targetPort: http-metricsselector:app.kubernetes.io/name: docker-registrytype: ClusterIP

创建 statefulset

---
apiVersion: apps/v1
kind: StatefulSet
metadata:labels:app.kubernetes.io/name: docker-registryname: docker-registrynamespace: registry
spec:replicas: 1selector:matchLabels:app.kubernetes.io/name: docker-registryserviceName: docker-registry-svctemplate:metadata:labels:app.kubernetes.io/name: docker-registryspec:affinity:podAntiAffinity:preferredDuringSchedulingIgnoredDuringExecution:- podAffinityTerm:labelSelector:matchLabels:app.kubernetes.io/name: docker-registrytopologyKey: kubernetes.io/hostnameweight: 1containers:- image: docker.m.daocloud.io/registry:2.8.3livenessProbe:failureThreshold: 60initialDelaySeconds: 5periodSeconds: 10successThreshold: 1tcpSocket:port: httptimeoutSeconds: 1name: docker-registryports:- containerPort: 5000name: http- containerPort: 5001name: http-metricsreadinessProbe:failureThreshold: 60initialDelaySeconds: 5periodSeconds: 10successThreshold: 1tcpSocket:port: httptimeoutSeconds: 1resources:limits:cpu: 2000mmemory: 2.5Girequests:cpu: 100mmemory: 100MistartupProbe:failureThreshold: 60initialDelaySeconds: 5periodSeconds: 10successThreshold: 1tcpSocket:port: httptimeoutSeconds: 1volumeMounts:- mountPath: /etc/docker/registryname: config- mountPath: /authname: authterminationGracePeriodSeconds: 30volumes:- name: configsecret:secretName: docker-registry-cm- name: authsecret:secretName: docker-registry-auth

创建 ingress

没有 ingress 可以开 nodeport 来实现

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:annotations:nginx.ingress.kubernetes.io/proxy-body-size: 5Gname: docker-registrynamespace: registry
spec:ingressClassName: nginxrules:- host: registry.devops.icuhttp:paths:- backend:service:name: docker-registry-svcport:number: 5000path: /pathType: Prefix

验证 docker registry

/etc/docker/daemon.json 增加 registry 地址

"insecure-registries": ["ip:端口"]
# 或者
"insecure-registries": ["域名"]

登录 docker registry

docker login http://registry.devops.icu

修改 tag

docker tag docker.m.daocloud.io/registry:2.8.3 registry.devops.icu/registry:2.8.3

上传镜像

docker push registry.devops.icu/registry:2.8.3

docker registry 监控

grafana id：9621

在这里插入图片描述

prometheus 配置文件参考

    - job_name: docker-registrykubernetes_sd_configs:- role: endpointsrelabel_configs:- source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name]regex: registry;docker-registry-svcaction: keep- source_labels: [__meta_kubernetes_pod_ip]regex: (.+)target_label: __address__replacement: ${1}:5001- source_labels: [__meta_kubernetes_endpoints_name]action: replacetarget_label: endpoint- source_labels: [__meta_kubernetes_pod_name]action: replacetarget_label: pod- source_labels: [__meta_kubernetes_service_name]action: replacetarget_label: service- source_labels: [__meta_kubernetes_namespace]action: replacetarget_label: namespace

docker registry ui

Github 项目地址：Joxit/docker-registry-ui-2.5.7

相关的变量和参数详见：available-options

---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/name: docker-registry-uiname: docker-registry-ui-svcnamespace: registry
spec:ports:- name: httpport: 8080protocol: TCPtargetPort: 8080selector:app.kubernetes.io/name: docker-registry-uitype: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:labels:app.kubernetes.io/name: docker-registry-uiname: docker-registry-uinamespace: registry
spec:replicas: 1selector:matchLabels:app.kubernetes.io/name: docker-registry-uitemplate:metadata:labels:app.kubernetes.io/name: docker-registry-uispec:containers:- env:- name: SINGLE_REGISTRYvalue: "true"- name: SHOW_CATALOG_NB_TAGSvalue: "true"- name: REGISTRY_SECUREDvalue: "true"- name: NGINX_PROXY_PASS_URLvalue: http://docker-registry-svc.registry.svc.cluster.local:5000- name: NGINX_PROXY_HEADER_Authorizationvalue: $http_authorizationimage: joxit/docker-registry-ui:2.5.7imagePullPolicy: IfNotPresentname: docker-registry-uisecurityContext:fsGroup: 101runAsUser: 101
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: docker-registry-uinamespace: registry
spec:ingressClassName: nginxrules:- host: registry.ui.devops.icuhttp:paths:- backend:service:name: docker-registry-ui-svcport:number: 8080path: /pathType: Prefix