1. 什么是Ingress
官方文档：https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/#what-is-ingress
Ingress 是 kubernetes API 中的标准资源类型之一，主要是k8s官方在维护。
2. Ingress的作用
Ingress 提供从集群外部到集群内服务的 HTTP 和 HTTPS 路由。 流量路由由 Ingress 资源所定义的规则来控制。
ingress 实现的功能是在应用层对客户端请求的 host 名称或请求的 URL 路径把请求转发到指定的 service 资源的规则，即用
于将 kubernetes 集群外部的请求资源转发之集群内部的 service，再被 service 转发之 pod
处理客户端的请求。
说简单点就是另一种对外暴露服务的方式。
Ingress本身其实就是nginx二次开发的一个东西。
3. Ingress-controller
为了让 Ingress 资源工作，集群必须有一个正在运行的 Ingress 控制器。
Ingress 资源需要指定监听地址、请求的 host 和 URL 等配置，然后根据这些规则的匹配机制将客户端的请求进行转发，这种能够为 ingress 配置资源监听并转发流量的组件称为ingress 控制器(ingress controller)，ingress controller 是 kubernetes 的一个附件，类似于dashboard 或者 flannel 一样，需要单独部署。
4. 其他类型的Ingres控制器
https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress-controllers/#additional-controllers
5. 部署Ingress Nginx
官方文档：https://kubernetes.github.io/ingress-nginx/deploy/
官方文档：https://github.com/kubernetes/ingress-nginx/blob/main/docs/deploy/index.md
可以把官方提供的这个yaml下载下面，改改部署也可以
5.1 部署注意事项
官方文档：https://github.com/kubernetes/ingress-nginx?tab=readme-ov-file#supported-versions-table
部署Ingress-Nginx的时候一定要注意和k8s版本兼容性的问题
5.2 部署方式
Ingress-nginx有2种部署方式，一个是deployment，一个是demonset，区别：
（1）deployment方式部署
默认只有一个ingress控制器，在被调度到的节点上监听默认的80和443端口，如果有请求是从其他节点过来的，需要先经过路由转发才能到我ingress控制器所在的这个节点，就增加了一个转发的性能开销。
（2）demonset方式部署
在每个k8s节点上都部署一个ingress控制器，同时也在每个节点上都监听了80和443端口，该方式性能更好。
5.3 部署Ingress Nginx v1.3.0
因为我本地也无法从外网拉取docker镜像，所以改用了改好的1.3.0，我的集群是1.24.0的，刚好1.3.0能用
5.3.1 yaml配置
5.3.1.1 deployment
[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# cat 1.ingress-nginx-controller-v1.3.0_deployment.yaml
apiVersion: v1
kind: Namespace
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxname: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginxnamespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginxnamespace: ingress-nginx
rules:
- apiGroups:- ""resources:- namespacesverbs:- get
- apiGroups:- ""resources:- configmaps- pods- secrets- endpointsverbs:- get- list- watch
- apiGroups:- ""resources:- servicesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingressesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingresses/statusverbs:- update
- apiGroups:- networking.k8s.ioresources:- ingressclassesverbs:- get- list- watch
- apiGroups:- ""resourceNames:- ingress-controller-leaderresources:- configmapsverbs:- get- update
- apiGroups:- ""resources:- configmapsverbs:- create
- apiGroups:- coordination.k8s.ioresourceNames:- ingress-controller-leaderresources:- leasesverbs:- get- update
- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- create
- apiGroups:- ""resources:- eventsverbs:- create- patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admissionnamespace: ingress-nginx
rules:
- apiGroups:- ""resources:- secretsverbs:- get- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx
rules:
- apiGroups:- ""resources:- configmaps- endpoints- nodes- pods- secrets- namespacesverbs:- list- watch
- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- list- watch
- apiGroups:- ""resources:- nodesverbs:- get
- apiGroups:- ""resources:- servicesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingressesverbs:- get- list- watch
- apiGroups:- ""resources:- eventsverbs:- create- patch
- apiGroups:- networking.k8s.ioresources:- ingresses/statusverbs:- update
- apiGroups:- networking.k8s.ioresources:- ingressclassesverbs:- get- list- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission
rules:
- apiGroups:- admissionregistration.k8s.ioresources:- validatingwebhookconfigurationsverbs:- get- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginxnamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx
subjects:
- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admissionnamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx-admission
subjects:
- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx
subjects:
- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx-admission
subjects:
- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: v1
data:allow-snippet-annotations: "true"
kind: ConfigMap
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-controllernamespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-controllernamespace: ingress-nginx
spec:ipFamilies:- IPv4ipFamilyPolicy: SingleStackports:- appProtocol: httpname: httpport: 80protocol: TCPtargetPort: httpnodePort: 50080- appProtocol: http #kubernetes v1.20 stable,appProtocol字段提供了一种为每个Service端口指定应用协议的方式,此字段的取值会被映射到对应的Endpointsname: prometheus-metrics-portport: 10254protocol: TCPtargetPort: 10254 #ingress-nginx-controller内置的指标数据采集端口nodePort: 50254
# - name: metrics-port
# port: 10254
# targetPort: 10254
# nodePort: 50254
# protocol: TCP- appProtocol: httpsname: httpsport: 443protocol: TCPtargetPort: httpsnodePort: 50443selector:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtype: NodePort
---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-controller-admissionnamespace: ingress-nginx
spec:ports:- appProtocol: httpsname: https-webhookport: 443targetPort: webhookselector:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtype: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-controllernamespace: ingress-nginx
spec:minReadySeconds: 0revisionHistoryLimit: 10selector:matchLabels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtemplate:metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxspec:containers:- args:- /nginx-ingress-controller- --election-id=ingress-controller-leader- --controller-class=k8s.io/ingress-nginx- --ingress-class=nginx- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller- --validating-webhook=:8443- --validating-webhook-certificate=/usr/local/certificates/cert- --validating-webhook-key=/usr/local/certificates/keyenv:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: LD_PRELOADvalue: /usr/local/lib/libmimalloc.soimage: registry.cn-hangzhou.aliyuncs.com/zhangshijie/ingress-nginx-controller:v1.3.0imagePullPolicy: IfNotPresentlifecycle:preStop:exec:command:- /wait-shutdownlivenessProbe:failureThreshold: 5httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1name: controllerports:- containerPort: 80name: httpprotocol: TCP- containerPort: 443name: httpsprotocol: TCP- containerPort: 8443name: webhookprotocol: TCPreadinessProbe:failureThreshold: 3httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1resources:requests:cpu: 100mmemory: 90MisecurityContext:allowPrivilegeEscalation: truecapabilities:add:- NET_BIND_SERVICEdrop:- ALLrunAsUser: 101volumeMounts:- mountPath: /usr/local/certificates/name: webhook-certreadOnly: truednsPolicy: ClusterFirstnodeSelector:kubernetes.io/os: linuxserviceAccountName: ingress-nginxterminationGracePeriodSeconds: 300volumes:- name: webhook-certsecret:secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission-createnamespace: ingress-nginx
spec:template:metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission-createspec:containers:- args:- create- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc- --namespace=$(POD_NAMESPACE)- --secret-name=ingress-nginx-admissionenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespaceimage: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0imagePullPolicy: IfNotPresentname: createsecurityContext:allowPrivilegeEscalation: falsenodeSelector:kubernetes.io/os: linuxrestartPolicy: OnFailuresecurityContext:fsGroup: 2000runAsNonRoot: truerunAsUser: 2000serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission-patchnamespace: ingress-nginx
spec:template:metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission-patchspec:containers:- args:- patch- --webhook-name=ingress-nginx-admission- --namespace=$(POD_NAMESPACE)- --patch-mutating=false- --secret-name=ingress-nginx-admission- --patch-failure-policy=Failenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespaceimage: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0imagePullPolicy: IfNotPresentname: patchsecurityContext:allowPrivilegeEscalation: falsenodeSelector:kubernetes.io/os: linuxrestartPolicy: OnFailuresecurityContext:fsGroup: 2000runAsNonRoot: truerunAsUser: 2000serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: nginx
spec:controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:- v1clientConfig:service:name: ingress-nginx-controller-admissionnamespace: ingress-nginxpath: /networking/v1/ingressesfailurePolicy: FailmatchPolicy: Equivalentname: validate.nginx.ingress.kubernetes.iorules:- apiGroups:- networking.k8s.ioapiVersions:- v1operations:- CREATE- UPDATEresources:- ingressessideEffects: None5.3.1.2 demonset
[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# cat 2.ingress-nginx-controller-v1.3.0_daemonset.yaml
apiVersion: v1
kind: Namespace
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxname: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginxnamespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginxnamespace: ingress-nginx
rules:
- apiGroups:- ""resources:- namespacesverbs:- get
- apiGroups:- ""resources:- configmaps- pods- secrets- endpointsverbs:- get- list- watch
- apiGroups:- ""resources:- servicesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingressesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingresses/statusverbs:- update
- apiGroups:- networking.k8s.ioresources:- ingressclassesverbs:- get- list- watch
- apiGroups:- ""resourceNames:- ingress-controller-leaderresources:- configmapsverbs:- get- update
- apiGroups:- ""resources:- configmapsverbs:- create
- apiGroups:- coordination.k8s.ioresourceNames:- ingress-controller-leaderresources:- leasesverbs:- get- update
- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- create
- apiGroups:- ""resources:- eventsverbs:- create- patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admissionnamespace: ingress-nginx
rules:
- apiGroups:- ""resources:- secretsverbs:- get- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx
rules:
- apiGroups:- ""resources:- configmaps- endpoints- nodes- pods- secrets- namespacesverbs:- list- watch
- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- list- watch
- apiGroups:- ""resources:- nodesverbs:- get
- apiGroups:- ""resources:- servicesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingressesverbs:- get- list- watch
- apiGroups:- ""resources:- eventsverbs:- create- patch
- apiGroups:- networking.k8s.ioresources:- ingresses/statusverbs:- update
- apiGroups:- networking.k8s.ioresources:- ingressclassesverbs:- get- list- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission
rules:
- apiGroups:- admissionregistration.k8s.ioresources:- validatingwebhookconfigurationsverbs:- get- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginxnamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx
subjects:
- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admissionnamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx-admission
subjects:
- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx
subjects:
- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx-admission
subjects:
- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: v1
data:allow-snippet-annotations: "true"
kind: ConfigMap
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-controllernamespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-controllernamespace: ingress-nginx
spec:ipFamilies:- IPv4ipFamilyPolicy: SingleStackports:- appProtocol: httpname: httpport: 80protocol: TCPtargetPort: httpnodePort: 30080- appProtocol: http #kubernetes v1.20 stable,appProtocol字段提供了一种为每个Service端口指定应用协议的方式,此字段的取值会被映射到对应的Endpointsname: prometheus-metrics-portport: 10254protocol: TCPtargetPort: 10254 #ingress-nginx-controller内置的指标数据采集端口nodePort: 30254
# - name: metrics-port
# port: 10254
# targetPort: 10254
# nodePort: 50254
# protocol: TCP- appProtocol: httpsname: httpsport: 443protocol: TCPtargetPort: httpsnodePort: 30443selector:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtype: NodePort
---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-controller-admissionnamespace: ingress-nginx
spec:ports:- appProtocol: httpsname: https-webhookport: 443targetPort: webhookselector:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtype: ClusterIP
---
apiVersion: apps/v1
kind: DaemonSet
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-controllernamespace: ingress-nginx
spec:minReadySeconds: 0revisionHistoryLimit: 10selector:matchLabels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtemplate:metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxspec:#hostNetwork: true #使用宿主机网络。使用宿主机网络一定要修改dns策略为ClusterFirstWithHostNet，否则ingress控制器的默认dns策略ClusterFirst会因为pod使用了hostnetwork而退回default模式，从而直接使用宿主机dns配置。#hostPID: true #使用宿主机Pidcontainers:- args:- /nginx-ingress-controller- --election-id=ingress-controller-leader- --controller-class=k8s.io/ingress-nginx- --ingress-class=nginx- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller- --validating-webhook=:8443- --validating-webhook-certificate=/usr/local/certificates/cert- --validating-webhook-key=/usr/local/certificates/keyenv:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: LD_PRELOADvalue: /usr/local/lib/libmimalloc.soimage: registry.cn-hangzhou.aliyuncs.com/zhangshijie/ingress-nginx-controller:v1.3.0imagePullPolicy: IfNotPresentlifecycle:preStop:exec:command:- /wait-shutdownlivenessProbe:failureThreshold: 5httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1name: controllerports:- containerPort: 80name: httpprotocol: TCP- containerPort: 443name: httpsprotocol: TCP- containerPort: 8443name: webhookprotocol: TCPreadinessProbe:failureThreshold: 3httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1resources:requests:cpu: 100mmemory: 90MisecurityContext:allowPrivilegeEscalation: truecapabilities:add:- NET_BIND_SERVICEdrop:- ALLrunAsUser: 101volumeMounts:- mountPath: /usr/local/certificates/name: webhook-certreadOnly: truednsPolicy: ClusterFirsttolerations:- key: "key1"operator: "Equal"value: "value1"effect: "NoSchedule"nodeSelector:kubernetes.io/os: linuxserviceAccountName: ingress-nginxterminationGracePeriodSeconds: 300volumes:- name: webhook-certsecret:secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission-createnamespace: ingress-nginx
spec:template:metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission-createspec:containers:- args:- create- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc- --namespace=$(POD_NAMESPACE)- --secret-name=ingress-nginx-admissionenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespaceimage: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0imagePullPolicy: IfNotPresentname: createsecurityContext:allowPrivilegeEscalation: falsenodeSelector:kubernetes.io/os: linuxrestartPolicy: OnFailuresecurityContext:fsGroup: 2000runAsNonRoot: truerunAsUser: 2000serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission-patchnamespace: ingress-nginx
spec:template:metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission-patchspec:containers:- args:- patch- --webhook-name=ingress-nginx-admission- --namespace=$(POD_NAMESPACE)- --patch-mutating=false- --secret-name=ingress-nginx-admission- --patch-failure-policy=Failenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespaceimage: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0imagePullPolicy: IfNotPresentname: patchsecurityContext:allowPrivilegeEscalation: falsenodeSelector:kubernetes.io/os: linuxrestartPolicy: OnFailuresecurityContext:fsGroup: 2000runAsNonRoot: truerunAsUser: 2000serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: nginx
spec:controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.3.0name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:- v1clientConfig:service:name: ingress-nginx-controller-admissionnamespace: ingress-nginxpath: /networking/v1/ingressesfailurePolicy: FailmatchPolicy: Equivalentname: validate.nginx.ingress.kubernetes.iorules:- apiGroups:- networking.k8s.ioapiVersions:- v1operations:- CREATE- UPDATEresources:- ingressessideEffects: None
5.3.2 demonset方式部署ingress nginx 控制器
[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# kubectl apply -f 2.ingress-nginx-controller-v1.3.0_daemonset.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
daemonset.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# kubectl get po,svc -n ingress-nginx
NAME READY STATUS RESTARTS AGE
pod/ingress-nginx-admission-create-bbxkb 0/1 Completed 0 26m
pod/ingress-nginx-admission-patch-z7zqf 0/1 Completed 0 26m
pod/ingress-nginx-controller-5ch6s 1/1 Running 0 26m
pod/ingress-nginx-controller-d9mtz 1/1 Running 0 26mNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/ingress-nginx-controller NodePort 10.200.238.16 <none> 80:30080/TCP,10254:30254/TCP,443:30443/TCP 26m
service/ingress-nginx-controller-admission ClusterIP 10.200.118.113 <none> 443/TCP 22m
6. 基于域名实现单host及多host的ingress
6.1 部署测试用的web服务
[root@master1 Ingress-case-20230611-backup]# cat tomcat-app1.yaml
kind: Deployment
apiVersion: apps/v1
metadata:labels:app: tomcat-app1-deployment-labelname: tomcat-app1-deployment
spec:replicas: 1selector:matchLabels:app: tomcat-app1-selectortemplate:metadata:labels:app: tomcat-app1-selectorspec:containers:- name: tomcat-app1-containerimage: tomcat:7.0.94-alpine#command: ["/apps/tomcat/bin/run_tomcat.sh"]imagePullPolicy: IfNotPresentports:- containerPort: 8080protocol: TCPname: httpenv:- name: "password"value: "123456"- name: "age"value: "18"resources:limits:cpu: 1memory: "512Mi"requests:cpu: 500mmemory: "512Mi"
---
kind: Service
apiVersion: v1
metadata:labels:app: tomcat-app1-service-labelname: tomcat-app1-service
spec:ports:- name: httpport: 80protocol: TCPtargetPort: 8080selector:app: tomcat-app1-selector[root@master1 Ingress-case-20230611-backup]# kubectl apply -f tomcat-app1.yaml
deployment.apps/tomcat-app1-deployment created
service/tomcat-app1-service created[root@master1 Ingress-case-20230611-backup]# cat tomcat-app2.yaml
kind: Deployment
apiVersion: apps/v1
metadata:labels:app: tomcat-app2-deployment-labelname: tomcat-app2-deployment
spec:replicas: 1selector:matchLabels:app: tomcat-app2-selectortemplate:metadata:labels:app: tomcat-app2-selectorspec:containers:- name: tomcat-app2-containerimage: tomcat:7.0.94-alpine#command: ["/apps/tomcat/bin/run_tomcat.sh"]imagePullPolicy: IfNotPresentports:- containerPort: 8080protocol: TCPname: httpenv:- name: "password"value: "123456"- name: "age"value: "18"resources:limits:cpu: 1memory: "512Mi"requests:cpu: 500mmemory: "512Mi"
---
kind: Service
apiVersion: v1
metadata:labels:app: tomcat-app2-service-labelname: tomcat-app2-service
spec:ports:- name: httpport: 80protocol: TCPtargetPort: 8080selector:app: tomcat-app2-selector[root@master1 Ingress-case-20230611-backup]# kubectl apply -f tomcat-app2.yaml
deployment.apps/tomcat-app2-deployment created
service/tomcat-app2-service created[root@master1 Ingress-case-20230611-backup]# kubectl get po |grep tomcat
tomcat-app1-deployment-749f777dbf-mz9ws 1/1 Running 0 4m10s
tomcat-app2-deployment-6999ff4c89-brmrp 1/1 Running 0 2m48s6.2 环境初始化
6.2.1 tomcat-1
[root@master1 Ingress-case-20230611-backup]# kubectl exec -it tomcat-app1-deployment-749f777dbf-mz9ws -- /bin/bash
bash-4.4# pwd
/usr/local/tomcat/webapps
bash-4.4# mkdir app1
bash-4.4# echo 'pc web test page' > app1/index.jsp
bash-4.4# exit6.2.2 tomcat-2
[root@master1 Ingress-case-20230611-backup]# kubectl exec -it tomcat-app2-deployment-6999ff4c89-brmrp -- /bin/bash
bash-4.4# pwd
/usr/local/tomcat/webapps
bash-4.4# mkdir app2
bash-4.4# echo 'mobile app test page' > app2/index.jsp
bash-4.4# exit6.3 配置单域名访问的ingress规则
6.3.1 配置ingress规则
[root@master1 Ingress-case-20230611-backup]# cat 2.1.ingress_single-mobile.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: mobile.test.com # 用域名作为ingress名称，方便识别namespace: default # 注意nginx规则要和后端pod在同一个ns下annotations: # 注解，类似于指令，相当于配置nginx服务的一些功能kubernetes.io/ingress.class: "nginx" ##指定Ingress Controller的类型nginx.ingress.kubernetes.io/use-regex: "true" ##指定后面rules定义的path可以使用正则表达式nginx.ingress.kubernetes.io/proxy-connect-timeout: "600" ##连接超时时间,默认为5snginx.ingress.kubernetes.io/proxy-send-timeout: "600" # 设置代理服务器向后端服务器发送数据的超时时间，单位为秒，默认60s，如果超时Ingress控制器将终止连接并返回适当的错误。nginx.ingress.kubernetes.io/proxy-read-timeout: "600" ##读取超时时间,默认为60s# 上面的三个proxy超时时间，建议设置长一点时间。nginx.ingress.kubernetes.io/proxy-body-size: "50m" ##客户端上传文件，最大大小，默认为20m#nginx.ingress.kubernetes.io/rewrite-target: / ##URL重写nginx.ingress.kubernetes.io/app-root: /index.html # 定义上下文中 / 必须重定向的应用程序根目录
spec:rules: # 路由规则- host: mobile.test.com # 客户端访问的host域名http:paths:- pathType: Prefix # 前缀匹配，且区分大小写。另外还有Exact精确匹配和ImplementationSpecific。path: "/" # 匹配路径，类似nginx的locationbackend: # 后端配置service:name: tomcat-app2-service # 后端pod的service名称port:number: 80 # service端口[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 2.1.ingress_single-mobile.yaml
ingress.networking.k8s.io/mobile.test.com created
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
mobile.test.com <none> mobile.test.com 80 3s6.3.2 访问测试
6.4 配置多域名访问的ingress规则
6.4.1 配置ingress规则
# 清理环境
[root@master1 Ingress-case-20230611-backup]# kubectl delete -f 2.1.ingress_single-mobile.yaml# 配置并应用新的ingress规则
[root@master1 Ingress-case-20230611-backup]# cat 2.2.ingress_multi-host.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: nginx-webannotations:kubernetes.io/ingress.class: "nginx"nginx.ingress.kubernetes.io/use-regex: "true"nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"nginx.ingress.kubernetes.io/proxy-send-timeout: "600"nginx.ingress.kubernetes.io/proxy-read-timeout: "600"nginx.ingress.kubernetes.io/proxy-body-size: "10m"#nginx.ingress.kubernetes.io/rewrite-target: / ##URL重写nginx.ingress.kubernetes.io/app-root: /index.html
spec:rules:- host: pc.test.com # 域名一http:paths:- pathType: Prefixpath: "/"backend:service:name: tomcat-app1-serviceport:number: 80- host: mobile.test.com # 域名二http:paths:- pathType: Prefixpath: "/"backend:service:name: tomcat-app2-serviceport:[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 2.2.ingress_multi-host.yaml
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-web <none> pc.test.com,mobile.test.com 192.168.10.11,192.168.10.12 80 15m6.4.2 访问测试
6.4.3 清理环境
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-web <none> pc.test.com,mobile.test.com 192.168.10.11,192.168.10.12 80 15m[root@master1 Ingress-case-20230611-backup]# kubectl delete ingress nginx-web
ingress.networking.k8s.io "nginx-web" deleted6.5 配置基于URL访问的ingress规则
就是和nginx location配置差不多的道理，通过不同的URL location匹配并转发到不同的后端service。
6.5.1 ingress规则配置
[root@master1 Ingress-case-20230611-backup]# cat 3.1.ingress-url.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: nginx-webannotations:kubernetes.io/ingress.class: "nginx"nginx.ingress.kubernetes.io/use-regex: "true"nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"nginx.ingress.kubernetes.io/proxy-send-timeout: "600"nginx.ingress.kubernetes.io/proxy-read-timeout: "600"nginx.ingress.kubernetes.io/proxy-body-size: "10m"#nginx.ingress.kubernetes.io/rewrite-target: /nginx.ingress.kubernetes.io/app-root: /index.html
spec:rules:- host: pc.test.comhttp:paths:- pathType: Prefixpath: "/app1"backend:service:name: tomcat-app1-serviceport:number: 80- pathType: Prefixpath: "/app2"backend:service:name: tomcat-app2-serviceport:number: 80[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 3.1.ingress-url.yaml
ingress.networking.k8s.io/nginx-web created
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-web <none> pc.test.com 80 3s6.5.2 访问测试
6.5.3 清理环境
[root@master1 Ingress-case-20230611-backup]# kubectl delete -f 3.1.ingress-url.yaml
ingress.networking.k8s.io "nginx-web" deleted7. 单域名和多域名的https证书配置
7.1 单域名https证书配置
7.1.1 自签证书
7.1.1.1 签发ca证书
[root@master1 ~]# mkdir certs
[root@master1 ~]# cd certs
[root@master1 certs]# openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 3560 -nodes -subj '/CN=pc.test.com'
Generating a 4096 bit RSA private key
...........++
..............................................................................++
writing new private key to 'ca.key'
-----
[root@master1 certs]# ll
total 8
-rw-r--r-- 1 root root 1793 Feb 19 10:37 ca.crt
-rw-r--r-- 1 root root 3272 Feb 19 10:37 ca.key7.1.1.2 签发域名证书-公钥和csr
[root@master1 certs]# openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=pc.test.com'
Generating a 4096 bit RSA private key
............................................................................................................................................................................................................++
.........................................................................................................................................++
writing new private key to 'server.key'
-----
[root@master1 certs]# ll -rt
total 16
-rw-r--r-- 1 root root 3272 Feb 19 10:37 ca.key
-rw-r--r-- 1 root root 1793 Feb 19 10:37 ca.crt
-rw-r--r-- 1 root root 3272 Feb 19 10:38 server.key
-rw-r--r-- 1 root root 1586 Feb 19 10:38 server.csr
[root@master1 certs]#7.1.1.3 签发域名证书-私钥
[root@master1 certs]# openssl x509 -req -sha256 -days 3650 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt
Signature ok
subject=/CN=pc.test.com
Getting CA Private Key
[root@master1 certs]# ll -rt
total 20
-rw-r--r-- 1 root root 3272 Feb 19 10:37 ca.key
-rw-r--r-- 1 root root 1793 Feb 19 10:37 ca.crt
-rw-r--r-- 1 root root 3272 Feb 19 10:38 server.key
-rw-r--r-- 1 root root 1586 Feb 19 10:38 server.csr
-rw-r--r-- 1 root root 1663 Feb 19 10:40 server.crt7.1.2 上传证书到k8s集群中
[root@master1 certs]# kubectl create secret tls tls-secret-pc --cert=server.crt --key=server.key -n default
secret/tls-secret-pc created
[root@master1 certs]# kubectl get secret # 证书上传完毕后，内容就会通过base64加密
NAME TYPE DATA AGE
tls-secret-pc kubernetes.io/tls 2 5s7.1.3 配置ingress规则
[root@master1 Ingress-case-20230611-backup]# cat 4.1.ingress-https-magedu_single-host.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: nginx-webannotations:kubernetes.io/ingress.class: "nginx" ##指定Ingress Controller的类型nginx.ingress.kubernetes.io/ssl-redirect: 'true' #SSL重定向，即将http请求强制重定向至https，等于nginx中的全站https
spec:tls:- hosts:- pc.test.comsecretName: tls-secret-pcrules:- host: pc.test.comhttp:paths:- pathType: Prefixpath: "/"backend:service:name: tomcat-app1-serviceport:number: 80[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 4.1.ingress-https-magedu_single-host.yaml
ingress.networking.k8s.io/nginx-web created
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-web <none> pc.test.com 80, 443 5s7.1.4 访问测试
7.2 多域名https证书配置
这里只展示配置，其他的不再演示
7.2.1 配置ingress规则
[root@master1 Ingress-case-20230611-backup]# cat 4.2.ingress-https-magedu_multi-host.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: nginx-web-mobileannotations:kubernetes.io/ingress.class: "nginx"nginx.ingress.kubernetes.io/ssl-redirect: 'true'
spec:tls: # 就是在tls下配置不同的hosts就行了，指定不同的secret证书- hosts:- mobile.test.comsecretName: tls-secret-mobile- hosts:- pc.test.comsecretName: tls-secret-pcrules:- host: pc.test.comhttp:paths:- pathType: Prefixpath: "/"backend:service:name: tomcat-app1-serviceport:number: 80- host: mobile.test.comhttp:paths:- pathType: Prefixpath: "/"backend:service:name: tomcat-app2-serviceport:number: 808. ingress 证书更新案例
8.1 确认老证书的过期时间
8.2 签发新证书
实际工作中，使用商业证书都是不需要自己手动签发的。
[root@master1 certs-new]# openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=pc.test.com'
Generating a 4096 bit RSA private key
......................................................................................................................++
.........++
writing new private key to 'server.key'
-----
[root@master1 certs-new]# ls
server.csr server.key
[root@master1 certs-new]# openssl x509 -req -sha256 -days 36500 -in server.csr -CA ../certs/ca.crt -CAkey ../certs/ca.key -set_serial 01 -out server.crt # 这里还是基于之前的ca签发新证书，这里的证书有效期我配置了100年 -days 36500
Signature ok
subject=/CN=pc.test.com
Getting CA Private Key
[root@master1 certs-new]# ll -rt
total 12
-rw-r--r-- 1 root root 3268 Feb 19 11:40 server.key
-rw-r--r-- 1 root root 1586 Feb 19 11:40 server.csr
-rw-r--r-- 1 root root 1667 Feb 19 11:41 server.crt
[root@master1 certs-new]#8.3 base64加密新证书内容
8.3.1 加密公钥.crt文件内容
# 加密后的内容：
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVvakNDQW9vQ0FRRXdEUVlKS29aSWh2Y05BUUVMQlFBd0ZqRVVNQklHQTFVRUF3d0xjR011ZEdWemRDNWoKYjIwd0lCY05NalF3TWpFNU1ETTBNVE0xV2hnUE1qRXlOREF4TWpZd016UXhNelZhTUJZeEZEQVNCZ05WQkFNTQpDM0JqTG5SbGMzUXVZMjl0TUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUFzRlFuClovOXlLOVZBYnZiNVQ0bm5oSTNBMXUzMEVnYzFadlZRS1Q3L29uMFpWYnVZTFVNdDc3bHJNMjRRaUtGS0JjaXIKYjFrT0xJNDAvV1lmdVBUMDVIK2J5MlFxR3Zkb2dOWXBza2E5SWsxcnM1bzNVVjltNDBNSnhSOE9IOEJoUThNLwpzZVZGMWp5SXEwb0ZLUHFFSDVkMnhZTXdMekMrT3ArNGNNZUsxU3hoNlNNbm82TXhhcU00aGJlTU5NbmFPY1BjCkVHT3FqZHVMbmtPRXd4UDFmWDZrQkpEaW5WditKYUJwWG8zdHJJaFg4RWIxNGpqL21qemFON1pObm43d3grVGcKTHl3cUhEZWdHVmoxays4V2hqOUdXNWhXL3F1ZXM1L3lDeUdDR1ZYODNxN3NvRWxjR0ovRU8yRnZPTWJQTUJRdwplZjdvTkU5WFhzdXZMZVVMUXNOZ3FvR3RicEpaTlFxLzh1QnBIbUhsNjljQmtuVkVnSlJZMC9pK3VEaGFpQ1lWCkJPZURVb3hFOFZQSWlMMWJ2KzhIZ2lWVm5LL3lmazhsU3J1RldnbzlkTkhDTWFEQ014cE5KYTA5N2xYQ1c5N1cKMGNUNEF6WGdXL3B0VlR6ZktWZUd4VCtGem9jUGlZbk1EeXEwcXVTMXNYeWpjK0xZTjZWWmRPcGRwZzVZczg3Tgp5V25EVUl1OWg4NldRbnJMZDd1ZTY1VU9EWmJjb2QzTVVoMUVNcFNwTHE3bEZwWTdZRmVONklBRWRobnRzWVhECkFzU3ZWdVNSL1hkRThZYVNFYXNhMEYrWVczZWlGQzNobEFzZDZVVG5UK0E4Z09zemhpQVorRE43NDROOVVVdEgKeWlKYXVkTlVxRXA1VjE3V1gzN25oZWJ1SzdaUlRMOWRmcThYSzk4Q0F3RUFBVEFOQmdrcWhraUc5dzBCQVFzRgpBQU9DQWdFQVVzR09qUG5Sb2NOaFRqUjNoa1dQYmNDT3AwenJDbndEY1Y4ZG11QzVGbm5PTUdJc2hZelV5ZHJUCjZsOEFmRlhibUNtZ2xvbjFxUEpPR3lMT3A5YlRRdlQ3bDY4elRSb0p5R1JxdVZCRGoxb1JCUUI3TER3ZUN4THQKU2NGR0JFdi9wWFc3NnVWbXovZTVRRmlIY0pna1F1bmsyR0JvZUUwM2lLR3ZabW9PcnM3b2VMRW1XM3prb2JxbgpiTlV1MFMraHRJcHlnZEYrM1pjL0RVTmsyaWEwS3JFVnVlMWdDL0YveldiNVR5eDBQVEhnNlIwKzRKeERMRlF0Cnh0N3IrNXVQeUg0QVlsWFNwenNnM3VvL1BmNkZtOHA0VlJDcGNCYUlaUCtPRlZzSzN2SFhyODJuNHdXOU8vRCsKNFJ1cnpXWnRGK0hmNHJVY3VpN2gwbmk5d0pUUUVUZDJuTEZxcHM0ZENuNCs1L1FtY3JVUDdkNlk0OExEYktLawp3UWd2NzB1aUp6alhRNjRKa1dMSDhWVTRRdEpGQSs1QkFraHFaRFZEWXoxMEV3QlJvRWNuWFA5MXpaUyszOTVICkZFNjB0Vkx0YmVnSktxb0pUSTJ0WThyWWNhSVBWbjNESi9KbTdLTWpXZGwxK3BKNEhVNzgzRW9sRzMyYkZiNTUKZm9LQzE2T0tkQ2x4ZTZrOCtyQWxBRnV1Z1UxZzBFVXVycjNHeERtbjBaNG5hZEFKcmYvK1p6NXZXUndPWEwwSgpGRFhUTkRVRE9qcnIxTlJ1bHVTOERROEVIZTM4N1dtM2ExMSttOUZVYUhpdHI4U21DZG5lK2RzQkswNUxVVEtoCmJEL3pOQzI3SzJIQUxmaUlrRmZZTFN4TGl1Q1c3QXpoQ3oyeHp2UVdUS0RQRGRaZ1BCMD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQ==
8.3.2 加密私钥.key文件内容
8.3 更新证书
在线更新secrets内容是实时生效的，但是生产环境变更时，一定不要在线编辑，先导出旧的secrets yaml到本地，备份后，在文件中修改应用，这样就算出了问题，也可以第一时间恢复正常。
但是由于secrets创建后，无法通过apply -f 更新内容，所以先导出旧的内容备份，然后edit在线编辑实时生效。
或者备份好后，先delete 旧的secrets，再apply -f创建新的，但是这种方式没有第一种稳妥。
8.3.1 修改对应证书secrets的内容
[root@master1 certs-new]# kubectl get secrets
NAME TYPE DATA AGE
tls-secret-pc kubernetes.io/tls 2 73m
[root@master1 certs-new]# kubectl edit secrets tls-secret-pc
8.3.2 验证证书更新结果
9. 配置ingress控制器日志格式为json
参考：https://github.com/zhangshijle/ingress-files/blob/main/2.ingress-configmap.yaml
[root@master1 Ingress-case-20230611-backup]# kubectl get cm -n ingress-nginx|grep ingress
ingress-controller-leader 0 18h
ingress-nginx-controller 1 18h# 调整内容展示
kind: ConfigMap
apiVersion: v1
metadata:name: nginx-configurationnamespace: ingress-nginxlabels:app.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxannotations:# 下面这个注解的作用记录了最后一次应用的配置信息。kubectl.kubernetes.io/last-applied-configuration: >{"apiVersion":"v1","kind":"ConfigMap","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/part-of":"ingress-nginx"},"name":"nginx-configuration","namespace":"ingress-nginx"}}
data:use-forwarded-headers: "true" # 启用使用转发的HTTP头信息。compute-full-forwarded-for: "true" # 启用计算完整的转发的HTTP头中的"X-Forwarded-For"字段。use-proxy-protocol: 'True' # 启用使用代理协议（Proxy Protocol），但是要后端支持4层才行。real-ip-header: proxy_protocol # 指定真实IP的头信息名称为"proxy_protocol"。set-real-ip-from: 172.26.0.0/16 # 公有云的话要修改为当前kubernetes环境的当前使用的VPC的CIDR格式的网络信息（掩码），然后日志里面会显示是从哪个网段转发过来的请求。私有云不用加。# 日志格式log-format-upstream: '{"nginx_timestamp":"$time_iso8601","tcp_xff":"$proxy_protocol_addr","clientip":"$remote_addr","nginx_host":"$server_addr","host":"$http_host","request":"$request","url":"$request_uri","upstreamhost":"$upstream_addr","status":"$status","body_bytes_sent":"$body_bytes_sent","request_time":"$request_time","upstream_response_time":"$upstream_response_time","xff":"$http_x_forwarded_for","referer":"$http_referer","http_user_agent":"$http_user_agent","request_length":"$request_length","request_method":"$request_method"}'
