Traefik系列

一、入门Traefik系列——基础简介

官方文档

https://doc.traefik.io/traefik/[1]

简介

Traefik是一个为了让部署微服务更加便捷而诞生的现代HTTP反向代理、负载均衡工具。它支持多种后台 (Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd, Zookeeper, BoltDB, Rest API, file…) 来自动化、动态的应用它的配置文件设置。

流量示意图

图片

图片

核心概念

当请求Traefik时,请求首先到entrypoints,然后分析传入的请求,查看他们是否与定义的Routers匹配。如果匹配,则会通过一系列middlewares处理,再到traefikServices上做流量转发,最后请求到kubernetes的services上

图片

  • Providers用来自动发现平台上的服务,可以是编排工具、容器引擎云提供商或者键值存储。Traefik通过查询Providers的API来查询路由的相关信息,一旦检测到变化,就会动态的更新路由。

  • Entrypoints监听传入的流量,是网络的入口点,定义了接受请求的端口(HTTP或者TCP)

  • Routers分析请求(host,path,headers,SSL等),负责将传入的请求连接到可以处理这些请求的服务上去。

  • Middlewares中间件,用来修改请求或者根据请求来做出判断,中间件被附件到路由上,是一种在请求发送到服务之前调整请求的一种方法。

  • Service将请求转发给应用,负责配置如何最终将处理传入请求的实际服务,Traefik的Service介于Middlewares与KubernetesService之间,可以实现加权负载、流量复制等功能。

traefik组件与nginx类比

组件名称功能nginx相同概念
Providers监听路由信息变化,更新路由修改nginx配置,reload服务。
Entrypoints网络入口,监听传入的流量配置文件listen指定监听端口
Routers分析传入的请求,匹配规则配置文件server_name+location
Middlewares中间件,修改请求或响应location配置段中添加的缓存、压缩、请求头等配置
Service请求转发http配置段中的upstream

Nginx-Ingress和traefik区别

Ingress Controller

k8s 是通过一个又一个的 controller 来负责监控、维护集群状态。Ingress Controller 就是监控 Ingress 路由规则的一个变化,然后跟 k8s 的资源操作入口 api-server 进行通信交互。K8s 并没有自带 Ingress Controller,它只是一种标准,具体实现有多种,需要自己单独安装,常用的是 Nginx Ingress Controller 和 Traefik Ingress Controller。Ingress Controller 收到请求,匹配 Ingress 转发规则,匹配到了就转发到后端 Service,而 Service 可能代表的后端 Pod 有多个,选出一个转发到那个 Pod,最终由那个 Pod 处理请求。

图片

与kubernetes交互

nginx-ingress:由于微服务架构以及 Docker 技术和 kubernetes 编排工具最近几年才开始逐渐流行,所以一开始的反向代理服务器比如 nginx、apache 并未提供其支持,所以才会出现 Ingress Controller 这种东西来做 kubernetes 和前端负载均衡器如 nginx 之间做衔接;即 Ingress Controller 的存在就是为了能跟 kubernetes 交互,然后写入nginx 配置,最后reload。使用nginx作为前端负载均衡,通过ingress controller不断的和kubernetes api交互,实时获取后端service,pod等的变化,然后动态更新nginx配置,并刷新使配置生效,达到服务发现的目的。  traefik:traefik本身设计的就能够实时跟kubernetes api交互,感知后端service,pod等的变化,自动更新配置并重载。

traefik优点

  • 不需要安装其他依赖,使用 GO 语言编译可执行文件

  • 支持多种后台,如 Docker, Swarm mode, Kubernetes, Marathon, Consul, Etcd, Rancher, Amazon ECS 等等

  • 支持 REST API

  • 配置文件热重载,可自动监听配置改动、发现新服务,并自动更新无需人工重启

  • 支持熔断、限流功能

  • 支持轮训、负载均衡

  • 提供简洁的 UI 界面

  • 支持 Websocket, HTTP/2, GRPC

  • 自动更新 HTTPS 证书

  • 支持高可用集群模式

Nginx和Traefik横向对比

Nginx IngressTraefik ingress
协议http/https、http2、grpc、tcp/udphttp/https、http2、grpc、tcp、tcp+tls
路由匹配host、pathhost、path、headers、query、path prefix、method
命名空间支持-共用或指定命名空间
部署策略-金丝雀部署、蓝绿部署、灰度部署
upstream探测重试、超时、心跳探测重试、超时、心跳探测、熔断
负载均衡算法RR、会话保持、最小连接、最短时间、一致性hashWRR、动态RR、会话保持
优点简单易用,易接入Golang编写,部署容易,支持众多的后端,内置WebUI
缺点没有解决nginx reload,插件多,但是扩展性能差性能略逊于NGINX,但强于HAProxy

二、入门Traefik系列——部署与配置

helm部署

参考文档

官方文档:https://doc.traefik.io/traefik/getting-started/install-traefik/[1]

gtihub地址:https://github.com/traefik/traefik-helm-chart[2]

必要条件

Kubernetes版本1.14+

Helm版本3+

安装traefik


# 添加repo
[root@k8s-master traefik]# helm repo add traefik https://helm.traefik.io/traefik
# 更新repo仓库资源
[root@k8s-master traefik]# helm repo update
# 查看repo仓库traefik
[root@k8s-master traefik]# helm search repo traefik
NAME            CHART VERSION   APP VERSION     DESCRIPTION                                       
stable/traefik  1.87.7          1.7.26          DEPRECATED - A Traefik based Kubernetes ingress...
traefik/traefik 10.24.3         2.8.5           A Traefik based Kubernetes ingress controller# 创建traefik-v2名称空间
# 创建traefik名称空间
[root@k8s-master traefik]# kubectl create ns traefik
# 安装traefik
[root@k8s-master traefik]# helm install --namespace=traefik traefik traefik/traefik
# 查看helm列表
[root@k8s-master traefik]# helm list -n traefik
NAME    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
traefik traefik         1               2022-09-22 11:35:22.176916449 +0800 CST deployed        traefik-10.24.3 2.8.5 
# 查看pod资源信息
[root@k8s-master traefik]# kubectl get pod -n traefik
NAME                       READY   STATUS    RESTARTS   AGE
traefik-5bfc574f88-vz4zr   1/1     Running   0          65s

端口转发dashboard服务

默认情况下,由于安全考虑,不会公开 Traefik 仪表板。可以通过端口转发实现仪表板访问

kubectl port-forward -n traefik deployment/traefik --address 0.0.0.0 9000:9000
# 可通过网址访问:http://k8s-master:9000/dashboard/
# 如果出现以下报错信息无法访问
Forwarding from 0.0.0.0:9000 -> 9000
Handling connection for 9000
E1229 13:10:40.655810  243434 portforward.go:400] an error occurred forwarding 9000 -> 9000: error forwarding port 9000 to pod de96185927c5a2a6150c67130c14ef0f97f14900144389122570ccfc4b5b3179, uid : unable to do port forwarding: socat not found
Handling connection for 9000
E1229 13:11:06.847006  243434 portforward.go:400] an error occurred forwarding 9000 -> 9000: error forwarding port 9000 to pod de96185927c5a2a6150c67130c14ef0f97f14900144389122570ccfc4b5b3179, uid : unable to do port forwarding: socat not found
Handling connection for 9000# 解决办法在node节点安装socat
yum install socat

域名访问dashboard服务

使用helm部署的traefik默认使用LoadBalancer暴露服务,如果想使用此方式访问,首先要部署MetalLB才能分配到EXTERNAL-IP,


[root@k8s-master traefik]# ls
dashboard-ingress.yaml  dashboard-svc.yaml  dashboard.yaml
[root@k8s-master traefik]# kubectl get svc -n traefik 
NAME      TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                      AGE
traefik   LoadBalancer   10.97.204.137   192.168.10.201   80:31718/TCP,443:31294/TCP   20m

然后配置dashboard的ingress资源:kubectl apply -f dashboard.yaml


apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboardnamespace: traefik
spec:entryPoints:- webroutes:- match: Host(`traefik.test.com`) && (PathPrefix(`/dashboard`) || PathPrefix(`/api`))kind: Ruleservices:- name: api@internalkind: TraefikService

接下来使用集群外部机器访问,添加hosts解析

192.168.10.201 traefik.test.com
# ip 为LoadBalancer的EXTERNAL-IP,域名为ingress配置的域名

图片

image.png

YAML自定义部署

helm虽然实现了一键安装部署,但是查看helm包的value.yaml配置发现总共有500多行配置,当需要修改配置项或者对traefik做一下自定义配置时,并不灵活。如果只是使用traefik的基础功能,推荐使用helm部署。如果想深入研究使用traefik的话,推荐使用自定义方式部署,系列文章后续均使用自定义方式做演示。

环境准备

k8s版本:1.19.16

traefik版本:2.8.7

其中master节点充当边缘节点,安装两块网卡,eth0:k8s集群内网ip  eth1公网ip

官方文档:https://doc.traefik.io/traefik/providers/kubernetes-crd/[4]

官方示例文件:https://github.com/traefik/traefik/blob/master/docs/content/user-guides/crd-acme/index.md[5](示例文件仅提供最基本的配置,且所有配置项通过args传参,仅供参考,并不推荐用于生产)

创建CRD资源

这里要注意你的k8s版本,从k8s 1.16开始废弃apiextensions.k8s.io/v1beta1,1.22完全删除。对于k8s 1.16 以上版本,使用apiextensions.k8s.io/v1

[root@k8s-master traefik]# wget https://raw.githubusercontent.com/traefik/traefik/v2.8/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml
[root@k8s-master traefik]# kubectl apply -f kubernetes-crd-definition-v1.yml 
[root@k8s-master traefik]# kubectl get crd
NAME                                    CREATED AT
ingressroutes.traefik.containo.us       2022-09-22T10:01:07Z
ingressroutetcps.traefik.containo.us    2022-09-22T10:01:07Z
ingressrouteudps.traefik.containo.us    2022-09-22T10:01:07Z
ipaddresspools.metallb.io               2022-09-23T01:14:18Z
l2advertisements.metallb.io             2022-09-23T01:14:18Z
middlewares.traefik.containo.us         2022-09-22T10:01:07Z
middlewaretcps.traefik.containo.us      2022-09-22T10:01:07Z
serverstransports.traefik.containo.us   2022-09-22T10:01:07Z
tlsoptions.traefik.containo.us          2022-09-22T10:01:08Z
tlsstores.traefik.containo.us           2022-09-22T10:01:08Z
traefikservices.traefik.containo.us     2022-09-22T10:01:08Z

创建RBAC资源

[root@k8s-master traefik]# wget https://raw.githubusercontent.com/traefik/traefik/v2.8/docs/content/reference/dynamic-configuration/kubernetes-crd-rbac.yml
dashboard-ingress.yaml  kubernetes-crd-definition-v1.yml
[root@k8s-master traefik]# vim kubernetes-crd-rbac.yml
[root@k8s-master traefik]# kubectl apply -f kubernetes-crd-rbac.yml
[root@k8s-master traefik]# kubectl get clusterrole | grep traefik
traefik-ingress-controller                                             2022-09-23T14:10:06Z
[root@k8s-master traefik]# kubectl get clusterrolebinding | grep traefik
traefik-ingress-controller                             ClusterRole/traefik-ingress-controller                                             2m

创建traefik配置文件

在 Traefik 中有三种方式定义静态配置:在配置文件中、在命令行参数中、通过环境变量传递,由于 Traefik 配置很多,通过 CLI 定义不是很方便,一般时候选择将其配置选项放到配置文件中,然后存入 ConfigMap,将其挂入 traefik 中。参考文档:https://doc.traefik.io/traefik/getting-started/configuration-overview/

创建 traefik-config.yaml 文件


kind: ConfigMap
apiVersion: v1
metadata:name: traefik-config
data:traefik.yaml: |-global:checkNewVersion: false    # 周期性的检查是否有新版本发布sendAnonymousUsage: false # 周期性的匿名发送使用统计信息serversTransport:insecureSkipVerify: true  # Traefik忽略验证代理服务的TLS证书api:insecure: true            # 允许HTTP 方式访问APIdashboard: true           # 启用Dashboarddebug: false              # 启用Debug调试模式metrics:prometheus:               # 配置Prometheus监控指标数据,并使用默认配置addRoutersLabels: true  # 添加routers metricsentryPoint: "metrics"   # 指定metrics监听地址entryPoints:web:address: ":80"          # 配置80端口,并设置入口名称为webforwardedHeaders: insecure: true        # 信任所有的forward headerswebsecure:address: ":443"         # 配置443端口,并设置入口名称为 websecureforwardedHeaders: insecure: truetraefik:address: ":9000"        # 配置9000端口,并设置入口名称为 dashboardmetrics:address: ":9100"        # 配置9100端口,作为metrics收集入口tcpep:address: ":9200"        # 配置9200端口,作为tcp入口udpep:address: ":9300/udp"    # 配置9300端口,作为udp入口providers:kubernetesCRD:            # 启用Kubernetes CRD方式来配置路由规则ingressClass: ""allowCrossNamespace: true   #允许跨namespaceallowEmptyServices: true    #允许空endpoints的servicelog:filePath: "/etc/traefik/logs/traefik.log" # 设置调试日志文件存储路径,如果为空则输出到控制台level: "INFO"             # 设置调试日志级别format: "common"          # 设置调试日志格式accessLog:filePath: "/etc/traefik/logs/access.log" # 设置访问日志文件存储路径,如果为空则输出到控制台format: "common"          # 设置访问调试日志格式bufferingSize: 0          # 设置访问日志缓存行数filters:statusCodes: ["200"]   # 设置只保留指定状态码范围内的访问日志retryAttempts: true     # 设置代理访问重试失败时,保留访问日志minDuration: 20         # 设置保留请求时间超过指定持续时间的访问日志fields:                   # 设置访问日志中的字段是否保留(keep保留、drop不保留)defaultMode: keep       # 设置默认保留访问日志字段names:                  # 针对访问日志特别字段特别配置保留模式ClientUsername: dropStartUTC: drop        # 禁用日志timestamp使用UTCheaders:                # 设置Header中字段是否保留defaultMode: keep     # 设置默认保留Header中字段names:                # 针对Header中特别字段特别配置保留模式#User-Agent: redact # 可以针对指定agentAuthorization: dropContent-Type: keep

创建 Traefik ConfigMap 资源kubectl apply -f traefik-config.yaml

节点设置label标签

模拟实际生产环境,假设集群中master节点安装两块网卡充当边缘节点,需要提前给master节点设置 Label,这样当程序部署时 Pod 会自动调度到设置 Label 的节点上。

master节点网络信息如下:

网卡名称ip用途
ens33192.168.10.10k8s集群内部ip
ens160192.168.93.128公网ip,用于外部访问

给master节点设置标签kubectl label nodes k8s-master IngressProxy=true  查看节点label信息

[root@k8s-master traefik]# kubectl get nodes --show-labels
NAME         STATUS   ROLES    AGE   VERSION    LABELS
k8s-master   Ready    master   28h   v1.19.16   IngressProxy=true,beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-master,kubernetes.io/os=linux,node-role.kubernetes.io/master=
k8s-work1    Ready    <none>   28h   v1.19.16   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-work1,kubernetes.io/os=linux
k8s-work2    Ready    <none>   28h   v1.19.16   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-work2,kubernetes.io/os=linux

Deployment部署traefik

使用DeamonSet或者Deployment均可部署,此处使用Deployment方式部署 Traefik,副本数设置为1,调度至IngressProxy=true的那台master边缘节点,并使用host网络模式,提高网络入口的网络性能

创建 traefik 部署文件 traefik-deployment.yaml


apiVersion: v1
kind: ServiceAccount
metadata:namespace: defaultname: traefik-ingress-controller
---
apiVersion: apps/v1
kind: Deployment
metadata:name: traefik-ingress-controllernamespace: defaultlabels:app: traefik
spec:replicas: 1   # 副本数为1,因为集群只设置一台master为边缘节点selector:matchLabels:app: traefiktemplate:metadata:name: traefiklabels:app: traefikspec:serviceAccountName: traefik-ingress-controllerterminationGracePeriodSeconds: 1containers:- name: traefikimage: traefik:v2.8.7env:- name: KUBERNETES_SERVICE_HOST       # 手动指定k8s api,避免网络组件不稳定。value: "192.168.10.10"- name: KUBERNETES_SERVICE_PORT_HTTPS # API server端口value: "6443"- name: KUBERNETES_SERVICE_PORT       # API server端口value: "6443"- name: TZ                            # 指定时区value: "Asia/Shanghai"ports:- name: webcontainerPort: 80hostPort: 80                      # 将容器端口绑定所在服务器的 80 端口- name: websecurecontainerPort: 443hostPort: 443                     # 将容器端口绑定所在服务器的 443 端口- name: admincontainerPort: 9000               # Traefik Dashboard 端口- name: metricscontainerPort: 9100               # metrics端口- name: tcpepcontainerPort: 9200               # tcp端口- name: udpepcontainerPort: 9300               # udp端口securityContext:                      # 只开放网络权限  capabilities:drop:- ALLadd:- NET_BIND_SERVICEargs:- --configfile=/etc/traefik/config/traefik.yamlvolumeMounts:- mountPath: /etc/traefik/configname: config- mountPath: /etc/traefik/logsname: logdir- mountPath: /etc/localtimename: timezonereadOnly: truevolumes:- name: configconfigMap:name: traefik-config - name: logdirhostPath:path: /data/traefik/logstype: "DirectoryOrCreate"- name: timezone                       #挂载时区文件hostPath:path: /etc/localtimetype: Filetolerations:                             # 设置容忍所有污点,防止节点被设置污点- operator: "Exists"hostNetwork: true                        # 开启host网络,提高网络入口的网络性能nodeSelector:                            # 设置node筛选器,在特定label的节点上启动IngressProxy: "true"                   # 调度至IngressProxy: "true"的节点

部署并查看deployment资源信息

[root@k8s-master traefik]# kubectl apply -f traefik-deployment.yaml
NAME                                          READY   STATUS    RESTARTS   AGE
traefik-ingress-controller-6c79ff9645-hh559   1/1     Running   0          29m
[root@k8s-master traefik]# kubectl get pod 
NAME                                          READY   STATUS    RESTARTS   AGE
traefik-ingress-controller-6c79ff9645-hh559   1/1     Running   0          29m                                                                           14h

创建service资源

创建 traefik服务资源文件 traefik-svc.yaml

# traefik-svc.yaml
apiVersion: v1
kind: Service
metadata:name: traefik
spec:type: NodePort    ## 官网示例为ClusterIP,为方便演示,此处改为NodePortselector:app: traefikports:- name: webprotocol: TCPport: 80targetPort: 80- name: websecureprotocol: TCPport: 443targetPort: 443- name: adminprotocol: TCPport: 9000targetPort: 9000- name: metricsprotocol: TCPport: 9100targetPort: 9100- name: tcpepprotocol: TCPport: 9200targetPort: 9200- name: udpepprotocol: UDPport: 9300targetPort: 9300
部署并查看svc资源信息[root@k8s-master traefik]# kubectl apply -f traefik-svc.yaml
[root@k8s-master traefik]# kubectl get svc
NAME         TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                                                                                   14h
traefik      NodePort       10.102.64.244   <none>           80:31843/TCP,443:30318/TCP,9000:32689/TCP,9100:31527/TCP,9200:32432/TCP,9300:30608/UDP   2m35s

NodePort方式访问dashboard

traefik的dashboard使用nodeport暴露服务,将9000端口映射为32689,现在访问http://192.168.10.10:32689/

图片

dashboard配置http域名访问

Traefik 应用已经部署完成,treafik的Dashboard为svc类型是NodePort,接下来配置域名规则,模拟外部用户通过公网IP访问dashboard应用

Traefik创建路由规则有多种方式,比如:

  • 原生Ingress写法

  • 使用CRD IngressRoute方式

  • 使用GatewayAPI的方式

相较于原生Ingress写法,ingressRoute是2.1以后新增功能,简单来说,他们都支持路径(path)路由和域名(host)HTTP路由,以及HTTPS配置,区别在于IngressRoute需要定义CRD扩展,但是它支持了TCP、UDP路由以及中间件等新特性,强烈推荐使用ingressRoute,GatewayAPI方式后续再详细介绍。

创建 Traefik Dashboard 路由规则文件 traefik-dashboard-route.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboard
spec:entryPoints:- webroutes:- match: Host(`traefik.test.com`)kind: Ruleservices:- name: api@internalkind: TraefikService

创建ingress资源

[root@k8s-master traefik]# kubectl apply -f traefik-dashboard-route.yaml
[root@k8s-master traefik]# kubectl get ingressroute -n traefik 
NAME                      AGE
traefik-dashboard-route   44m
[root@k8s-master traefik]# kubectl get ingressroute
NAME        AGE
dashboard   19m

配置 Hosts,客户端想通过域名访问服务,必须要进行 DNS 解析,由于这里没有 DNS 服务器进行域名解析,所以直接修改客户端端的 hosts 文件,将 Traefik 所在的节点的 公网IP 和自定义 host 绑定。

打开电脑的 Hosts 配置文件,往其加入下面配置:192.168.93.128 traefik.test.com

配置完成后,打开浏览器输入地址:http://traefik.test.com 打开 Traefik Dashboard。

图片

其他配置

强制使用TLS v1.2+

如今,TLS v1.0 和 v1.1 因为存在安全问题,现在已被弃用。为了保障系统安全,所有入口路由都应该强制使用TLS v1.2 或更高版本。

参考文档:https://doc.traefik.io/traefik/user-guides/crd-acme/#force-tls-v12

[root@k8s-master traefik]# cat tlsoption.yml 
---
apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:name: defaultnamespace: default
spec:minVersion: VersionTLS12cipherSuites:- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384   # TLS 1.2- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305    # TLS 1.2- TLS_AES_256_GCM_SHA384                  # TLS 1.3- TLS_CHACHA20_POLY1305_SHA256            # TLS 1.3curvePreferences:- CurveP521- CurveP384sniStrict: true
[root@k8s-master traefik]# kubectl apply -f tlsoption.yml 
tlsoption.traefik.containo.us/default created

日志轮换

官方并没有日志轮换的功能,但是traefik收到USR1信号后会重建日志文件,因此可以通过logrotate实现日志轮换

参考文档:https://doc.traefik.io/traefik/observability/logs/[8]

在/etc/logrotate.d创建下traefik目录

mkdir -p /etc/logrotate.d/traefik

配置logrotate文件

/data/traefik/logs/*.log {dailyrotate 15missingoknotifemptycompressdateextdateyesterdaydateformat .%Y-%m-%dcreate 0644 root rootpostrotatedocker kill --signal="USR1" $(docker ps | grep traefik |grep -v pause| awk '{print $1}')endscript}

添加crontab定时任务

sudo echo "0 0 * * * /usr/sbin/logrotate -f /etc/logrotate.d/traefik/traefikLogrotate >/dev/null 2>&1" > /etc/crontab

多控制器

有的业务场景下可能需要在一个集群中部署多个 traefik,不同的实例控制不同的 IngressRoute 资源对象,要实现该功能有两种方法:

通过 annotations 注解筛选

首先在traefik配置文件中的providers下增加Ingressclass参数,指定具体的值。

参考文档:https://doc.traefik.io/traefik/providers/kubernetes-crd/#ingressclass


kind: ConfigMap
apiVersion: v1
metadata:name: traefik-config
data:traefik.yaml: |-global:checkNewVersion: false    # 周期性的检查是否有新版本发布sendAnonymousUsage: false # 周期性的匿名发送使用统计信息serversTransport:insecureSkipVerify: true  # Traefik忽略验证代理服务的TLS证书api:insecure: true            # 允许HTTP 方式访问APIdashboard: true           # 启用Dashboarddebug: false              # 启用Debug调试模式metrics:prometheus:               # 配置Prometheus监控指标数据,并使用默认配置addRoutersLabels: true  # 添加routers metricsentryPoint: "metrics"   # 指定metrics监听地址entryPoints:web:address: ":80"          # 配置80端口,并设置入口名称为webforwardedHeaders: insecure: true        # 信任所有的forward headerswebsecure:address: ":443"         # 配置443端口,并设置入口名称为 websecureforwardedHeaders: insecure: truetraefik:address: ":9000"        # 配置9000端口,并设置入口名称为 dashboardmetrics:address: ":9100"        # 配置9100端口,作为metrics收集入口tcpep:address: ":9200"        # 配置9200端口,作为tcp入口udpep:address: ":9300/udp"    # 配置9300端口,作为udp入口providers:kubernetesCRD:            # 启用Kubernetes CRD方式来配置路由规则ingressClass: "traefik-v2.8" # 指定traefik的ingressClass实例名称allowCrossNamespace: true   #允许跨namespaceallowEmptyServices: true    #允许空endpoints的servicelog:filePath: "/etc/traefik/logs/traefik.log" # 设置调试日志文件存储路径,如果为空则输出到控制台level: "INFO"             # 设置调试日志级别format: "common"          # 设置调试日志格式accessLog:filePath: "/etc/traefik/logs/access.log" # 设置访问日志文件存储路径,如果为空则输出到控制台format: "common"          # 设置访问调试日志格式bufferingSize: 0          # 设置访问日志缓存行数filters:statusCodes: ["200"]   # 设置只保留指定状态码范围内的访问日志retryAttempts: true     # 设置代理访问重试失败时,保留访问日志minDuration: 20         # 设置保留请求时间超过指定持续时间的访问日志fields:                   # 设置访问日志中的字段是否保留(keep保留、drop不保留)defaultMode: keep       # 设置默认保留访问日志字段names:                  # 针对访问日志特别字段特别配置保留模式ClientUsername: dropStartUTC: drop        # 禁用日志timestamp使用UTCheaders:                # 设置Header中字段是否保留defaultMode: keep     # 设置默认保留Header中字段names:                # 针对Header中特别字段特别配置保留模式#User-Agent: redact # 可以针对指定agentAuthorization: dropContent-Type: keep

 接下来在IngressRoute 资源对象中的annotations参数中添加 kubernetes.io/ingress.class: traefik-v2.8即可


apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboardannotations:kubernetes.io/ingress.class: traefik-v2.8 #  因为静态配置文件指定了ingressclass,所以这里的annotations 要指定,否则访问会404
spec:entryPoints:- webroutes:- match: Host(`traefik.test.com`)kind: Ruleservices:- name: api@internalkind: TraefikService

通过标签选择器进行过滤

首先在traefik配置文件中的providers下增加labelSelector参数,指定具体的标签键值。

参考文档:https://doc.traefik.io/traefik/providers/kubernetes-crd/#labelselector


kind: ConfigMap
apiVersion: v1
metadata:name: traefik-config
data:traefik.yaml: |-global:checkNewVersion: false    # 周期性的检查是否有新版本发布sendAnonymousUsage: false # 周期性的匿名发送使用统计信息serversTransport:insecureSkipVerify: true  # Traefik忽略验证代理服务的TLS证书api:insecure: true            # 允许HTTP 方式访问APIdashboard: true           # 启用Dashboarddebug: false              # 启用Debug调试模式metrics:prometheus:               # 配置Prometheus监控指标数据,并使用默认配置addRoutersLabels: true  # 添加routers metricsentryPoint: "metrics"   # 指定metrics监听地址entryPoints:web:address: ":80"          # 配置80端口,并设置入口名称为webforwardedHeaders: insecure: true        # 信任所有的forward headerswebsecure:address: ":443"         # 配置443端口,并设置入口名称为 websecureforwardedHeaders: insecure: truetraefik:address: ":9000"        # 配置9000端口,并设置入口名称为 dashboardmetrics:address: ":9100"        # 配置9100端口,作为metrics收集入口tcpep:address: ":9200"        # 配置9200端口,作为tcp入口udpep:address: ":9300/udp"    # 配置9300端口,作为udp入口providers:kubernetesCRD:            # 启用Kubernetes CRD方式来配置路由规则# ingressClass: "traefik-v2.8"    # 指定traefik的ingressClass名称labelSelector: "app=traefik-v2.8" # 通过标签选择器指定traefik标签 allowCrossNamespace: true   #允许跨namespaceallowEmptyServices: true    #允许空endpoints的servicelog:filePath: "/etc/traefik/logs/traefik.log" # 设置调试日志文件存储路径,如果为空则输出到控制台level: "INFO"             # 设置调试日志级别format: "common"          # 设置调试日志格式accessLog:filePath: "/etc/traefik/logs/access.log" # 设置访问日志文件存储路径,如果为空则输出到控制台format: "common"          # 设置访问调试日志格式bufferingSize: 0          # 设置访问日志缓存行数filters:statusCodes: ["200"]   # 设置只保留指定状态码范围内的访问日志retryAttempts: true     # 设置代理访问重试失败时,保留访问日志minDuration: 20         # 设置保留请求时间超过指定持续时间的访问日志fields:                   # 设置访问日志中的字段是否保留(keep保留、drop不保留)defaultMode: keep       # 设置默认保留访问日志字段names:                  # 针对访问日志特别字段特别配置保留模式ClientUsername: dropStartUTC: drop        # 禁用日志timestamp使用UTCheaders:                # 设置Header中字段是否保留defaultMode: keep     # 设置默认保留Header中字段names:                # 针对Header中特别字段特别配置保留模式#User-Agent: redact # 可以针对指定agentAuthorization: dropContent-Type: keep

然后在 IngressRoute 资源对象中添加labels标签选择器,选择app: traefik-v2.8这个标签即可

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboardlabels:     # 通过标签选择器,该IngressRoute资源由配置了app=traefik-v2.8的traefik处理app: traefik-v2.8# annotations:# kubernetes.io/ingress.class: traefik-v2.8 #  因为静态配置文件指定了ingressclass,所以这里的annotations 要指定,否则访问会404
spec:entryPoints:- webroutes:- match: Host(`traefik.test.com`)kind: Ruleservices:- name: api@internalkind: TraefikService

Traefik CRD功能总结

traefik通过自定义资源实现了对traefik资源的创建和管理,支持的crd资源类型如下所示:

参考文档:https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/

kind功能
IngressRouteHTTP路由配置
MiddlewareHTTP中间件配置
TraefikServiceHTTP负载均衡/流量复制配置
IngressRouteTCP路由配置
MiddlewareTCPTCP中间件配置
IngressRouteUDPUDP路由配置
TLSOptionsTLS连接参数配置
TLSStoresTLS存储配置
ServersTransporttraefik与后端之间的传输配置

三、入门Traefik系列——路由配置与使用

环境准备

部署myapp1实例


apiVersion: apps/v1
kind: Deployment
metadata:name: myapp1
spec:selector:matchLabels:app: myapp1template:metadata:labels:app: myapp1spec:containers:- name: myapp1image: ikubernetes/myapp:v1resources:limits:memory: "128Mi"cpu: "500m"ports:- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:name: myapp1
spec:type: ClusterIPselector:app: myapp1ports:- port: 80targetPort: 80

部署myapp2实例

apiVersion: apps/v1
kind: Deployment
metadata:name: myapp2
spec:selector:matchLabels:app: myapp2template:metadata:labels:app: myapp2spec:containers:- name: myapp2image: ikubernetes/myapp:v2resources:limits:memory: "128Mi"cpu: "500m"ports:- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:name: myapp2
spec:type: ClusterIPselector:app: myapp2ports:- port: 80targetPort: 80

创建资源并访问测试

[root@k8s-master ingress]# kubectl get pod 
NAME                                          READY   STATUS    RESTARTS   AGE
myapp1-795d947b45-9lsm6                       1/1     Running   0          2m18s
myapp2-6ffd54f76-ljkr9                        1/1     Running   0          66s
[root@k8s-master ingress]# kubectl get svc 
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                  44h
myapp1       ClusterIP   10.104.91.200   <none>        80/TCP                                                                                   2m26s
myapp2       ClusterIP   10.111.245.32   <none>        80/TCP                                                                                   100s
[root@k8s-master ingress]# curl 10.104.91.200
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@k8s-master ingress]# curl 10.111.245.32
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

路由简介

路由功能

参考文档:https://doc.traefik.io/traefik/routing/overview/

图片

当启动Traefik时,需要定义entrypoints,然后通过entrypoints的路由来分析传入的请求,来查看他们是否是一组规则匹配,如果匹配,则路由可能将请求通过一系列的转换过来在发送到服务上去。

traefik支持的匹配规则

规则描述
Headers(keyvalue)检查headers中是否有一个键为key值为value的键值对
HeadersRegexp(keyregexp)检查headers中是否有一个键位key值为正则表达式匹配的键值对
Host(example.com, ...)检查请求的域名是否包含在特定的域名中
HostRegexp(example.com{subdomain:[a-z]+}.example.com, ...)检查请求的域名是否包含在特定的正则表达式域名中
Method(GET, ...)检查请求方法是否为给定的methods(GET、POST、PUT、DELETE、PATCH)中
Path(/path/articles/{cat:[a-z]+}/{id:[0-9]+}, ...)匹配特定的请求路径,它接受一系列文字和正则表达式路径
PathPrefix(/products//articles/{cat:[a-z]+}/{id:[0-9]+})匹配特定的前缀路径,它接受一系列文字和正则表达式前缀路径
Query(foo=barbar=baz)匹配查询字符串参数,接受key=value的键值对
ClientIP(10.0.0.0/16::1)如果请求客户端 IP 是给定的 IP/CIDR 之一,则匹配。它接受 IPv4、IPv6 和网段格式。

路由配置(IngressRoute)

HTTP域名路由

实现目标:集群外部用户通过访问http://myapp1.test.com域名时,将请求代理至myapp1应用。创建ingressrouter规则文件


apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp1
spec:entryPoints:- webroutes:- match: Host(`myapp1.test.com`) # 域名kind: Ruleservices:- name: myapp1  # 与svc的name一致port: 80      # 与svc的port一致

创建资源

[root@k8s-master ingress]# kubectl apply -f myapp1-ingress.yaml 
ingressroute.traefik.containo.us/myapp1 created
[root@k8s-master ingress]# kubectl get ingressroute 
dashboard  myapp1     
[root@k8s-master ingress]# kubectl get ingressroute
NAME        AGE
dashboard   4h26m
myapp1      20s

客户端添加hosts记录192.168.93.128 myapp1.test.com,然后访问验证

图片

HTTPS域名路由(自有证书)

公网服务的话,可以在云厂商那里购买证书。内部服务的话,就直接用 openssl 来创建一个自签名的证书即可,要注意证书文件名称必须是 tls.crt 和 tls.key。接下来演示自签证书的配置。创建自签证书

[root@k8s-master ingress]# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=myapp2.test.com"

建Secret资源来引用证书文件


[root@k8s-master ingress]# kubectl create secret tls myapp2-tls --cert=tls.crt --key=tls.key
secret/myapp2-tls created
[root@k8s-master ingress]# kubectl describe secrets myapp2-tls 
Name:         myapp2-tls
Namespace:    default
Labels:       <none>
Annotations:  <none>Type:  kubernetes.io/tlsData
====
tls.crt:  1131 bytes
tls.key:  1704 bytes

 创建IngressRouter规则文件,集群外部用户通过访问https://myapp2.test.com域名时,将请求代理至myapp2应用。

[root@k8s-master ingress]# cat myapp2-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2
spec:entryPoints:- websecure                    # 监听 websecure 这个入口点,也就是通过 443 端口来访问routes:- match: Host(`myapp2.test.com`)kind: Ruleservices:- name: myapp2port: 80tls:secretName: myapp2-tls         # 指定tls证书名称

创建资源

[root@k8s-master ingress]# kubectl apply -f myapp2-ingress.yaml 
ingressroute.traefik.containo.us/myapp2 created
[root@k8s-master ingress]# kubectl get ingressroute 
dashboard  myapp2     
[root@k8s-master ingress]# kubectl get ingressroute
NAME        AGE
dashboard   5h11m
myapp1      45m
myapp2      2m55s

客户端添加hosts记录192.168.93.128 myapp2.test.com,然后访问验证,由于我们是自签名的证书,所以证书是不受信任的。

图片

HTTPS域名路由(自动生成证书)

参考文档:https://doc.traefik.io/traefik-enterprise/tls/acme/

Traefik除了使用自有证书外,还支持在创建ingress资源时自动请求Let's Encrypt生成证书,并且在证书过期前30天自动续订证书。要使用Let's Encrypt自动生成证书,需要使用ACME。需要在静态配置中定义 "证书解析器",Traefik负责从ACME服务器中检索证书。然后,每个 "路由器 "被配置为启用TLS,并通过tls.certresolver配置选项与一个证书解析器关联。如果使用tlsChallenge,则要求Let's Encrypt到 Traefik 443 端口必须是可达的。如果使用httpChallenge,则要求Let's Encrypt到 Traefik 80端口必须是可达的。如果使用dnsChallenge,只需要配置上 DNS 解析服务商的 API 访问密钥即可校验。

tlsChallenge/httpChallenge

使用tlsChallenge或者httpChallenge的前提条件是traefik所在节点可以正常访问Let's Encrypt网站,国内网络可能访问存在异常,并且配置的ingress域名已经设置了dns的A记录解析,指向traefik所在的节点公网IP地址才可以,否则申请成功证书后无法通过验证。修改traefik配置文件,新增certificatesResolvers配置

# traefik-config.yaml
kind: ConfigMap
apiVersion: v1
metadata:name: traefik-config
data:traefik.yaml: |-global:checkNewVersion: false    ## 周期性的检查是否有新版本发布sendAnonymousUsage: false ## 周期性的匿名发送使用统计信息serversTransport:insecureSkipVerify: true  ## Traefik忽略验证代理服务的TLS证书api:insecure: true            ## 允许HTTP 方式访问APIdashboard: true           ## 启用Dashboarddebug: false              ## 启用Debug调试模式metrics:prometheus:               ## 配置Prometheus监控指标数据,并使用默认配置addRoutersLabels: true  ## 添加routers metricsentryPoint: "metrics"     ## 指定metrics监听地址entryPoints:web:address: ":80"          ## 配置80端口,并设置入口名称为webforwardedHeaders: insecure: true        ## 信任所有的forward headerswebsecure:address: ":443"         ## 配置443端口,并设置入口名称为 websecureforwardedHeaders: insecure: truetraefik:address: ":9000"        ## 配置9000端口,并设置入口名称为 dashboardmetrics:address: ":9100"        ## 配置9100端口,作为metrics收集入口tcpep:address: ":9200"        ## 配置9200端口,作为tcp入口udpep:address: ":9300/udp"    ## 配置9300端口,作为udp入口certificatesResolvers:      ## 开启ACME自动续签证书sample:acme:email: 1554382111@qq.com  # 邮箱配置storage: /etc/traefik/acme/acme.json    # 保存 ACME 证书的位置# tlsChallenge: {}            # tlsChallenge模式续签httpChallenge:entryPoint: web             # httpChallenge模式续签providers:kubernetesCRD:            ## 启用Kubernetes CRD方式来配置路由规则ingressClass: ""allowCrossNamespace: true   ##允许跨namespaceallowEmptyServices: true    ##允许空endpoints的servicekubernetesIngress:        ## 启动Kubernetes Ingress方式来配置路由规则ingressClass: ""log:filePath: "/etc/traefik/logs/traefik.log" ## 设置调试日志文件存储路径,如果为空则输出到控制台level: "DEBUG"             ## 设置调试日志级别format: "common"          ## 设置调试日志格式accessLog:filePath: "/etc/traefik/logs/access.log" ## 设置访问日志文件存储路径,如果为空则输出到控制台format: "common"          ## 设置访问调试日志格式bufferingSize: 0          ## 设置访问日志缓存行数filters:#statusCodes: ["200"]   ## 设置只保留指定状态码范围内的访问日志retryAttempts: true     ## 设置代理访问重试失败时,保留访问日志minDuration: 20         ## 设置保留请求时间超过指定持续时间的访问日志fields:                   ## 设置访问日志中的字段是否保留(keep保留、drop不保留)defaultMode: keep       ## 设置默认保留访问日志字段names:                  ## 针对访问日志特别字段特别配置保留模式ClientUsername: dropStartUTC: drop        ## 禁用日志timestamp使用UTCheaders:                ## 设置Header中字段是否保留defaultMode: keep     ## 设置默认保留Header中字段names:                ## 针对Header中特别字段特别配置保留模式#User-Agent: redact  ## 可以针对指定agentAuthorization: dropContent-Type: keep

修改traefik的deployment资源文件,挂载acme目录存储ACME证书信息

# traefik-deployment.yaml
apiVersion: v1
kind: ServiceAccount
metadata:namespace: defaultname: traefik-ingress-controller
---
apiVersion: apps/v1
kind: Deployment
metadata:name: traefik-ingress-controllernamespace: defaultlabels:app: traefik
spec:replicas: 1   #副本数为1,因为集群只设置一台master为边缘节点selector:matchLabels:app: traefiktemplate:metadata:name: traefiklabels:app: traefikspec:serviceAccountName: traefik-ingress-controllerterminationGracePeriodSeconds: 1containers:- name: traefikimage: traefik:v2.8.7env:- name: KUBERNETES_SERVICE_HOST       ##手动指定k8s api,避免网络组件不稳定。value: "192.168.10.10"- name: KUBERNETES_SERVICE_PORT_HTTPS ## API server端口value: "6443"- name: KUBERNETES_SERVICE_PORT       ## API server端口value: "6443"- name: TZ                            ##指定时区value: "Asia/Shanghai"ports:- name: webcontainerPort: 80hostPort: 80                      ## 将容器端口绑定所在服务器的 80 端口- name: websecurecontainerPort: 443hostPort: 443                     ## 将容器端口绑定所在服务器的 443 端口- name: admincontainerPort: 9000               ## Traefik Dashboard 端口- name: metricscontainerPort: 9100               ## metrics端口- name: tcpepcontainerPort: 9200               ## tcp端口- name: udpepcontainerPort: 9300               ## udp端口securityContext:                      ## 只开放网络权限  capabilities:drop:- ALLadd:- NET_BIND_SERVICEargs:- --configfile=/etc/traefik/config/traefik.yamlvolumeMounts:- mountPath: /etc/traefik/configname: config- mountPath: /etc/traefik/logsname: logdir- mountPath: /etc/localtimename: timezonereadOnly: true- mountPath: /etc/traefik/acmename: acmevolumes:- name: configconfigMap:name: traefik-config - name: logdirhostPath:path: /data/traefik/logstype: "DirectoryOrCreate"- name: timezone                       #挂载时区文件hostPath:path: /etc/localtimetype: File- name: acme                           # 自动续签证书文件hostPath:path: /data/traefik/acmetype: "DirectoryOrCreate"tolerations:                             ## 设置容忍所有污点,防止节点被设置污点- operator: "Exists"hostNetwork: true                        ## 开启host网络,提高网络入口的网络性能nodeSelector:                            ## 设置node筛选器,在特定label的节点上启动IngressProxy: "true"                   ## 调度至IngressProxy: "true"的节点

配置IngressRouter规则

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2
spec:entryPoints:- websecure                    # 监听 websecure 这个入口点,也就是通过 443 端口来访问routes:- match: Host(`myapp2.cuiliangblog.cn`) # 必须是真实存在的域名,且配置了dns解析记录,指向traefik节点所在的公网IPkind: Ruleservices:- name: myapp2port: 80tls:certResolver: sample         # 使用自动生成证书,名字与traefik的certificatesResolvers名称一致

dnsChallenge

dns 校验方式可以生成通配符的证书,只需要配置上 DNS 解析服务商的 API 访问密钥即可校验。每个厂商的配置都略有差异,此处以阿里云为例,其他厂商的配置请查看文档https://go-acme.github.io/lego/dns/[3]修改traefik配置文件,新增dnsChallenge配置

kind: ConfigMap
apiVersion: v1
metadata:name: traefik-config
data:traefik.yaml: |-global:checkNewVersion: false    ## 周期性的检查是否有新版本发布sendAnonymousUsage: false ## 周期性的匿名发送使用统计信息serversTransport:insecureSkipVerify: true  ## Traefik忽略验证代理服务的TLS证书api:insecure: true            ## 允许HTTP 方式访问APIdashboard: true           ## 启用Dashboarddebug: false              ## 启用Debug调试模式metrics:prometheus:               ## 配置Prometheus监控指标数据,并使用默认配置addRoutersLabels: true  ## 添加routers metricsentryPoint: "metrics"     ## 指定metrics监听地址entryPoints:web:address: ":80"          ## 配置80端口,并设置入口名称为webforwardedHeaders: insecure: true        ## 信任所有的forward headerswebsecure:address: ":443"         ## 配置443端口,并设置入口名称为 websecureforwardedHeaders: insecure: truetraefik:address: ":9000"        ## 配置9000端口,并设置入口名称为 dashboardmetrics:address: ":9100"        ## 配置9100端口,作为metrics收集入口tcpep:address: ":9200"        ## 配置9200端口,作为tcp入口udpep:address: ":9300/udp"    ## 配置9300端口,作为udp入口certificatesResolvers:      ## 开启ACME自动续签证书sample:acme:email: 1554382111@qq.com  # 邮箱配置storage: /etc/traefik/acme/acme.json    # 保存 ACME 证书的位置# tlsChallenge: {}              # tls模式续签# httpChallenge:#   entryPoint: web             # http模式续签dnsChallenge:                   # dns模式续签证书provider: alidns              # 云厂商编号       delayBeforeCheck: 0           # ACME 验证之前,会验证 TXT 记录。设定延迟验证时间(以秒为单位)providers:kubernetesCRD:            ## 启用Kubernetes CRD方式来配置路由规则ingressClass: ""allowCrossNamespace: true   ##允许跨namespaceallowEmptyServices: true    ##允许空endpoints的servicekubernetesIngress:        ## 启动Kubernetes Ingress方式来配置路由规则ingressClass: ""log:filePath: "/etc/traefik/logs/traefik.log" ## 设置调试日志文件存储路径,如果为空则输出到控制台level: "DEBUG"             ## 设置调试日志级别format: "common"          ## 设置调试日志格式accessLog:filePath: "/etc/traefik/logs/access.log" ## 设置访问日志文件存储路径,如果为空则输出到控制台format: "common"          ## 设置访问调试日志格式bufferingSize: 0          ## 设置访问日志缓存行数filters:#statusCodes: ["200"]   ## 设置只保留指定状态码范围内的访问日志retryAttempts: true     ## 设置代理访问重试失败时,保留访问日志minDuration: 20         ## 设置保留请求时间超过指定持续时间的访问日志fields:                   ## 设置访问日志中的字段是否保留(keep保留、drop不保留)defaultMode: keep       ## 设置默认保留访问日志字段names:                  ## 针对访问日志特别字段特别配置保留模式ClientUsername: dropStartUTC: drop        ## 禁用日志timestamp使用UTCheaders:                ## 设置Header中字段是否保留defaultMode: keep     ## 设置默认保留Header中字段names:                ## 针对Header中特别字段特别配置保留模式#User-Agent: redact  ## 可以针对指定agentAuthorization: dropContent-Type: keep

登录阿里云后台获取ALICLOUD_ACCESS_KEY、ALICLOUD_SECRET_KEY、ALICLOUD_REGION_ID信息 创建Secret 对象存放密钥信息,记得填写base64编码后的值

apiVersion: v1
kind: Secret
metadata:name: alidns-secret
type: Opaque
data:ALICLOUD_ACCESS_KEY: XXXALICLOUD_SECRET_KEY: XXXALICLOUD_REGION_ID: XXX

修改traefik的deployment资源清单,添加密钥env变量

apiVersion: v1
kind: ServiceAccount
metadata:namespace: defaultname: traefik-ingress-controller
---
apiVersion: apps/v1
kind: Deployment
metadata:name: traefik-ingress-controllernamespace: defaultlabels:app: traefik
spec:replicas: 1   # 副本数为1,因为集群只设置一台master为边缘节点selector:matchLabels:app: traefiktemplate:metadata:name: traefiklabels:app: traefikspec:serviceAccountName: traefik-ingress-controllerterminationGracePeriodSeconds: 1containers:- name: traefikimage: traefik:v2.8.7env:- name: KUBERNETES_SERVICE_HOST       # 手动指定k8s api,避免网络组件不稳定。value: "192.168.10.10"- name: KUBERNETES_SERVICE_PORT_HTTPS # API server端口value: "6443"- name: KUBERNETES_SERVICE_PORT       # API server端口value: "6443"- name: TZ                            # 指定时区value: "Asia/Shanghai"- name: ALICLOUD_ACCESS_KEY           # 阿里云AKvalueFrom:secretKeyRef:name: alidns-secretkey: ALICLOUD_ACCESS_KEY- name: ALICLOUD_SECRET_KEY           # 阿里云SKvalueFrom:secretKeyRef:name: alidns-secretkey: ALICLOUD_SECRET_KEY- name: ALICLOUD_REGION_ID            # 阿里云资源区域编号valueFrom:secretKeyRef:name: alidns-secretkey: ALICLOUD_REGION_IDports:- name: webcontainerPort: 80hostPort: 80                      # 将容器端口绑定所在服务器的 80 端口- name: websecurecontainerPort: 443hostPort: 443                     # 将容器端口绑定所在服务器的 443 端口- name: admincontainerPort: 9000               # Traefik Dashboard 端口- name: metricscontainerPort: 9100               # metrics端口- name: tcpepcontainerPort: 9200               # tcp端口- name: udpepcontainerPort: 9300               # udp端口securityContext:                      # 只开放网络权限  capabilities:drop:- ALLadd:- NET_BIND_SERVICEargs:- --configfile=/etc/traefik/config/traefik.yamlvolumeMounts:- mountPath: /etc/traefik/configname: config- mountPath: /etc/traefik/logsname: logdir- mountPath: /etc/localtimename: timezonereadOnly: true- mountPath: /etc/traefik/acmename: acmevolumes:- name: configconfigMap:name: traefik-config - name: logdirhostPath:path: /data/traefik/logstype: "DirectoryOrCreate"- name: timezone                       #挂载时区文件hostPath:path: /etc/localtimetype: File- name: acme                           # 自动续签证书文件hostPath:path: /data/traefik/acmetype: "DirectoryOrCreate"tolerations:                             # 设置容忍所有污点,防止节点被设置污点- operator: "Exists"hostNetwork: true                        # 开启host网络,提高网络入口的网络性能nodeSelector:                            # 设置node筛选器,在特定label的节点上启动IngressProxy: "true"                   # 调度至IngressProxy: "true"的节点

修改ingressrouter配置

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2
spec:entryPoints:- websecure                    # 监听 websecure 这个入口点,也就是通过 443 端口来访问routes:- match: Host(`myapp2.cuiliangblog.cn`)kind: Ruleservices:- name: myapp2port: 80tls:certResolver: sample         # 使用自动生成证书,名字与traefik的certificatesResolvers名称一致domains:- main: "*.cuiliangblog.cn"  # 不指定的话,默认申请Host域名,可以指定申请通配符域名

然后在阿里云DNS上做解析,重新创建ingress资源时即可触发申请证书。常见错误处理方案

日志关键词原因解决方案
net/http: timeout awaiting response headers或者connect: connection refusedtraefik所在节点无法访问Let's Encrypt申请证书使用工具加速
acme:  error :400/403申请的域名DNS解析记录为配置,或者配置地址不正确,指向了其他IP更改DNS解析配置,指向traefik节点所在的公网IP
acme: error : 429失败次数过多,每个小时只允许请求5次换账号/域名/IP重试或者等一个小时后再试

路由配置(IngressRouteTCP)

TCP路由(不带TLS证书)

首先部署一个简单的redis服务,资源清单文件如下所示:


# redis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: redis
spec:selector:matchLabels:app: redistemplate:metadata:labels:app: redisspec:containers:- name: redisimage: redis:latestresources:limits:memory: "128Mi"cpu: "500m"ports:- containerPort: 6379protocol: TCP
---
apiVersion: v1
kind: Service
metadata:name: redis
spec:selector:app: redisports:- port: 6379targetPort: 6379

创建redis应用

[root@k8s-master ingress]# kubectl apply -f redis.yaml 
deployment.apps/redis created
service/redis created

创建IngressRouter进行对外暴露

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:name: redis
spec:entryPoints:- tcpep											# 指定入口点为tcp端口routes:- match: HostSNI(`*`)         # 由于Traefik中使用TCP路由配置需要SNI,而SNI又是依赖TLS的,所以我们需要配置证书才行,如果没有证书的话,我们可以使用通配符*(适配ip的)进行配置services:- name: redisport: 6379

查看traefik的dashboard页面是否生效

[root@tiaoban ~]# redis-cli -h redis.test.com -p 9200
redis.test.com:9200> set key_a value_a
OK
redis.test.com:9200> get key_a
"value_a"
redis.test.com:9200>

图片

集群外部客户端配置hosts解析192.168.93.128 redis.test.com(域名可以随意填写,只要能解析到traefik所在节点即可),然后通过redis-cli工具访问redis,记得指定tcpep的端口。

如果需要再添加其他tcp路由,需要修改traefik配置,新增entryPoints端口。

TCP路由(带TLS证书)

有时候为了安全要求,tcp传输也需要使用TLS证书加密,redis从6.0开始支持了tls证书通信。

[root@k8s-master ingress]# mkdir redis-ssl
[root@k8s-master ingress]# cd redis-ssl/
[root@k8s-master redis-ssl]# openssl genrsa -out ca.key 4096
[root@k8s-master redis-ssl]# openssl req -x509 -new -nodes -sha256 -key ca.key -days 3650 -subj '/O=Redis Test/CN=Certificate Authority' -out ca.crt
[root@k8s-master redis-ssl]# openssl genrsa -out redis.key 2048
[root@k8s-master redis-ssl]# openssl req -new -sha256 -key redis.key -subj '/O=Redis Test/CN=Server' | openssl x509 -req -sha256 -CA ca.crt -CAkey ca.key -CAserial ca.txt -CAcreateserial -days 365 -out redis.crt
oot@k8s-master redis-ssl]# openssl dhparam -out redis.dh 2048
[root@k8s-master redis-ssl]# ll
总用量 24
-rw-r--r-- 1 root root 1895 9月  25 08:34 ca.crt
-rw------- 1 root root 3243 9月  25 08:34 ca.key
-rw-r--r-- 1 root root   41 9月  25 08:35 ca.txt
-rw-r--r-- 1 root root 1407 9月  25 08:35 redis.crt
-rw-r--r-- 1 root root  424 9月  25 08:35 redis.dh
-rw------- 1 root root 1679 9月  25 08:34 redis.key

创建secret资源,使用tls类型,包含redis.crt和redis.key

[root@k8s-master redis-ssl]# kubectl create secret tls redis-tls --key=redis.key --cert=redis.crt
secret/redis-tls created
[root@k8s-master redis-ssl]# kubectl describe secrets redis-tls 
Name:         redis-tls
Namespace:    default
Labels:       <none>
Annotations:  <none>Type:  kubernetes.io/tlsData
====
tls.crt:  1407 bytes
tls.key:  1679 bytes

创建secret资源,使用generic类型,包含ca.crt

[root@k8s-master redis-ssl]# kubectl create secret generic redis-ca --from-file=ca.crt=ca.crt
secret/redis-ca created
[root@k8s-master redis-ssl]# kubectl describe secrets redis-ca 
Name:         redis-ca
Namespace:    default
Labels:       <none>
Annotations:  <none>Type:  OpaqueData
====
ca.crt:  1895 bytes

修改redis配置,启用tls证书,并挂载证书文件

apiVersion: v1
kind: ConfigMap
metadata:name: redislabels:app: redis
data:redis.conf : |-port 0tls-port 6379tls-cert-file   /etc/tls/tls.crttls-key-file   /etc/tls/tls.keytls-ca-cert-file   /etc/ca/ca.crt
---
apiVersion: apps/v1
kind: Deployment
metadata:name: redis
spec:selector:matchLabels:app: redistemplate:metadata:labels:app: redisspec:containers:- name: redisimage: redis:latestresources:limits:memory: "128Mi"cpu: "500m"ports:- containerPort: 6379protocol: TCPvolumeMounts:- name: configmountPath: /etc/redis- name: tlsmountPath: /etc/tls- name: camountPath: /etc/caargs:- /etc/redis/redis.confvolumes:- name:  configconfigMap:name: redis- name: tlssecret:secretName: redis-tls- name: casecret:secretName: redis-ca
---
apiVersion: v1
kind: Service
metadata:name: redis
spec:selector:app: redisports:- port: 6379targetPort: 6379

创建资源并查看状态

[root@k8s-master ingress]# kubectl apply -f redis.yaml 
configmap/redis created
deployment.apps/redis created
service/redis created
[root@k8s-master ingress]# kubectl get pod
NAME                                          READY   STATUS              RESTARTS   AGE
redis-69974c8b56-rzxb6                        1/1     Running             0          7s

创建IngressRouter资源,指定域名和证书

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:name: redis
spec:entryPoints:- tcpeproutes:- match: HostSNI(`redis.test.com`)services:- name: redisport: 6379tls:secretName: redis-tls

traefik管理页查看

yum install openssl openssl-devel -y
wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make redis-cli BUILD_TLS=yes MALLOC=libc
cp src/redis-cli /usr/local/bin/

图片

yum安装的redis-cli版本为5.0.3,不支持tls,需要编译安装6.0以上版本,并在编译时开启TLS

客户端添加hosts记录192.168.93.128 redis.test.com,直接访问redis,直接报错


[root@tiaoban src]# ./src/redis-cli -h redis.test.com -p 9200127.0.0.1:6379> set key 1
Error: Connection reset by peer

客户端使用证书访问redis测试成功

[root@tiaoban src]# ./redis-cli -h redis.test.com -p 9200 --tls --cert /tmp/redis-ssl/redis.crt --key /tmp/redis-ssl/redis.key --cacert /tmp/redis-ssl/ca.crt
redis.test.com:9200> set key 1
OK

路由配置(IngressRouteUDP)

UDP路由

traefik同样也提供了UDP的支持,以我们最常用的rsyslog服务为例,演示traefik如果配置使用 首先制作一个rsyslog镜像


[root@k8s-master udp]# ls
Dockerfile  rsyslog.conf
# rsyslog配置
[root@k8s-master udp]# cat rsyslog.conf 
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imudp
$UDPServerRun 514
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
$OmitLocalLogging off
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
authpriv.*                                              /var/log/secure
mail.*                                                  -/var/log/maillog
cron.*                                                  /var/log/cron
*.emerg                                                 :omusrmsg:*
uucp,news.crit                                          /var/log/spooler
local7.*                                                /var/log/boot.log
# dockerfile配置
[root@k8s-master udp]# cat Dockerfile 
FROM centos:7
RUN yum -y install rsyslog && rm -rf /etc/rsyslog.d/listen.conf
COPY rsyslog.conf /etc/rsyslog.conf
EXPOSE 514/udp
CMD ["/usr/sbin/rsyslogd", "-dn"]
# 构建镜像
[root@k8s-master udp]# docker build -t rsyslog:v1 .

接下来创建rsyslog的资源清单

[root@k8s-master udp]# cat rsyslog-deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:name: rsyslog
spec:selector:matchLabels:app: rsyslogtemplate:metadata:labels:app: rsyslogspec:containers:- name: rsyslogimage: rsyslog:v1resources:limits:memory: "128Mi"cpu: "500m"ports:- containerPort: 514protocol: UDP
---
apiVersion: v1
kind: Service
metadata:name: rsyslog
spec:selector:app: rsyslogports:- port: 514protocol: UDPtargetPort: 514
部署上面的应用并查看[root@k8s-master udp]# kubectl apply -f rsyslog-deployment.yaml 
deployment.apps/rsyslog created
service/rsyslog created
[root@k8s-master udp]# kubectl get pod
NAME                                          READY   STATUS    RESTARTS   AGE
rsyslog-5dfc9d64b5-d9wjj                      1/1     Running   0          3s

创建IngressRouter资源,代理UDP应用,需要注意的是UDP资源访问时直接通过公网ip+dup的entryPoints端口即可,不需要配置域名

[root@k8s-master udp]# cat rsyslog-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteUDP
metadata:name: rsyslog
spec:entryPoints:- udpeproutes:- services:- name: rsyslogport: 514
[root@k8s-master udp]# kubectl apply -f rsyslog-ingress.yaml 
ingressrouteudp.traefik.containo.us/rsyslog created

查看dashboard的udp信息

图片

集群外部访问udp服务,通过 Traefik 所在节点的公网 IP(192.168.93.128)与 entryPoints端口(9300)来访问 UDP 应用进行测试

[root@tiaoban ~]# logger -n 192.168.93.128 -P 9300 "cuiliang123"
[root@tiaoban ~]# logger -n 192.168.93.128 -P 9300 "hello 123"

查看rsyslog日志,验证请求是否成功

[root@k8s-master ~]# kubectl get pod
NAME                                          READY   STATUS    RESTARTS   AGE
rsyslog-5dfc9d64b5-d9wjj                      1/1     Running   0          6m48s
[root@k8s-master ~]# kubectl exec -it rsyslog-5dfc9d64b5-d9wjj -- bash
[root@rsyslog-5dfc9d64b5-d9wjj /]# tail -n 5 /var/log/messages 
Sep 25 04:06:20 rsyslog-5dfc9d64b5-d9wjj rsyslogd:  [origin software="rsyslogd" swVersion="8.24.0-57.el7_9.3" x-pid="1" x-info="http://www.rsyslog.com"] start
Sep 25 12:12:55 tiaoban root cuiliang123
Sep 25 12:17:19 tiaoban root hello 123

负载均衡配置

traefik可以对http、TCP、UDP实现负载均衡,根据需求创建IngressRoute/IngressRouteTCP/IngressRouteUDP即可,此处以http为例。

http路由多个k8s service配置

创建两个deployment应用与对应的svc

[root@k8s-master udp]# kubectl get pod
NAME                                          READY   STATUS    RESTARTS   AGE
myapp1-795d947b45-9lsm6                       1/1     Running   1          25h
myapp2-6ffd54f76-ljkr9                        1/1     Running   1          25h
[root@k8s-master udp]# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                 2d22h
myapp1       ClusterIP   10.104.91.200   <none>        80/TCP                                                                                   25h
myapp2       ClusterIP   10.111.245.32   <none>        80/TCP

创建IngressRouter资源,配置域名为myapp.test.com,请求流量均摊到两个k8s的service上。

[root@k8s-master ingress]# cat myapp-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myappnamespace: default
spec:entryPoints:- webroutes:- match: Host(`myapp.test.com`)kind: Ruleservices:- name: myapp1namespace: defaultport: 80 - name: myapp2namespace: defaultport: 80
[root@k8s-master ingress]# kubectl apply -f myapp-ingress.yaml 
ingressroute.traefik.containo.us/myapp created

查看dashboard页面路由信息,发现已成功配置代理两个service服务,且权重均为1

图片

解析来访问测试,发现依次循环响应myapp1和myapp2的内容

[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

三、入门Traefik系列——中间件

简介

Traefik Middlewares 是一个处于路由和后端服务之前的中间件,在外部流量进入 Traefik,且路由规则匹配成功后,将流量发送到对应的后端服务前,先将其发给中间件进行一系列处理(类似于过滤器链 Filter,进行一系列处理),例如,添加 Header 头信息、鉴权、流量转发、处理访问路径前缀、IP 白名单等等,经过一个或者多个中间件处理完成后,再发送给后端服务,这个就是中间件的作用。Traefik内置了很多不同功能的Middleware,主要是针对HTTP和TCP,这里挑选几个比较常用的进行演示。

参考文档:https://doc.traefik.io/traefik/middlewares/overview/

重定向

redirectScheme的更多用法参考文档https://doc.traefik.io/traefik/middlewares/http/redirectscheme/

还是以前面的deployment应用与对应的svc为例

[root@k8s-master udp]# kubectl get pod
NAME                                          READY   STATUS    RESTARTS   AGE
myapp2-795d947b45-9lsm6                       1/1     Running   1          25h
[root@k8s-master udp]# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                 2d22h
myapp2       ClusterIP   10.104.91.200   <none>        80/TCP   
[root@k8s-master ingress]# kubectl describe secrets myapp2-tls
Name:         myapp2-tls
Namespace:    default
Labels:       <none>
Annotations:  <none>Type:  kubernetes.io/tlsData
====
tls.crt:  1131 bytes
tls.key:  1704 bytes

创建一个https的IngressRoute

[root@k8s-master middleware]# cat https-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2-tls
spec:entryPoints:- websecureroutes:- match: Host(`myapp2.test.com`)kind: Ruleservices:- name: myapp2port: 80 tls:secretName: myapp2-tls         # 指定tls证书名称
[root@k8s-master middleware]# kubectl apply -f https-ingress.yaml
ingressroute.traefik.containo.us/myapp2-tls created

定义一个强制将http请求跳转到https的中间件。

[root@k8s-master middleware]# cat https-middleware.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: redirect-https-middleware
spec:redirectScheme:scheme: https
[root@k8s-master middleware]# kubectl apply -f https-middleware.yaml 
middleware.traefik.containo.us/redirect-https-middleware created

定义一个http的IngressRoute,并使用中间件

[root@k8s-master middleware]# cat http-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2
spec:entryPoints:- webroutes:- match: Host(`myapp2.test.com`)kind: Ruleservices:- name: myapp2port: 80middlewares:- name: redirect-https-middleware   # 指定使用RedirectScheme中间件,完成http强制跳转至https
[root@k8s-master middleware]# kubectl apply -f http-ingress.yaml 
ingressroute.traefik.containo.us/myapp1 created

访问测试,当用户访问http://myapp.test.com时会强制跳转到https://myapp.test.com

图片

去除请求路径前缀

假设现在有这样一个需求,当访问http://myapp.test.com/v1时,流量调度至myapp1。当访问http://myapp.test.com/v2时,流量调度至myapp2。这种需求是非常常见的,在NGINX中,我们可以配置多个Location来定制规则,使用Traefik也可以这么做。但是定制不同的前缀后,由于应用本身并没有这些前缀,导致请求返回404,这时候我们就需要对请求的path进行处理。

参考文档https://doc.traefik.io/traefik/middlewares/http/stripprefix/

创建一个IngressRoute,并设置两条规则,根据不同的访问路径代理至相对应的service

[root@k8s-master middleware]# cat myapp-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp
spec:entryPoints:- webroutes:- match: Host(`myapp.test.com`) && PathPrefix(`/v1`)kind: Ruleservices:- name: myapp1port: 80- match: Host(`myapp.test.com`) && PathPrefix(`/v2`)kind: Ruleservices:- name: myapp2port: 80 
[root@k8s-master middleware]# kubectl apply -f myapp-ingress.yaml 
ingressroute.traefik.containo.us/myapp created

进行访问测试http://myapp.test.com/v1,虽然traefik配置无误,但是由于myapp1应用并没有v1这个路径,因此返回404页面

图片

图片

接下来定义去除前缀的中间件stripPrefix,指定将请求路径中的v1、v2去除。

[root@k8s-master middleware]# cat prefix-url-middleware.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: prefix-url-middleware
spec:stripPrefix:prefixes:- /v1- /v2
[root@k8s-master middleware]# kubectl apply -f prefix-url-middleware.yaml 
middleware.traefik.containo.us/prefix-url-middleware created

修改上面的ingressRoute,添加刚刚定义的prefix-url-middleware中间件

[root@k8s-master middleware]# cat myapp-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp
spec:entryPoints:- webroutes:- match: Host(`myapp.test.com`) && PathPrefix(`/v1`)kind: Ruleservices:- name: myapp1port: 80 middlewares:- name: prefix-url-middleware- match: Host(`myapp.test.com`) && PathPrefix(`/v2`)kind: Ruleservices:- name: myapp2port: 80 middlewares:- name: prefix-url-middleware
[root@k8s-master middleware]# kubectl apply -f myapp-ingress.yaml 
ingressroute.traefik.containo.us/myapp configured

查看traefik的dashboard,已添加了中间件

图片

接下来进行访问测试

图片

图片

添加IP白名单

为提高安全性,通常情况下一些管理员界面会设置ip访问白名单,只希望个别用户可以访问,例如访问traefik的dashboard的url,这时候就可以使用Traefik中的ipWhiteList中间件来完成。

参考文档https://doc.traefik.io/traefik/middlewares/http/ipwhitelist/

当前traefik的dashboard任何主机都可以访问

[root@tiaoban ~]# curl http://traefik.test.com/dashboard/
<!DOCTYPE html><html><head><title>Traefik</title><meta charset=utf-8><meta name=description content="Traefik UI">……

接下来定义IP访问白名单的中间件ipWhiteList,指定可以访问的ip列表。

[root@k8s-master middleware]# cat ip-white-middleware.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: ip-white-list-middleware
spec:ipWhiteList:sourceRange:- 127.0.0.1/32- 192.168.93.1
[root@k8s-master middleware]# kubectl apply -f ip-white-middleware.yaml 
middleware.traefik.containo.us/ip-white-list-middleware created

修改dashboard的ingressRoute,添加ip白名单中间件

[root@k8s-master middleware]# cat dashboard-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboard
spec:entryPoints:- webroutes:- match: Host(`traefik.test.com`)kind: Ruleservices:- name: api@internalkind: TraefikServicemiddlewares:- name: ip-white-list-middleware
[root@k8s-master middleware]# kubectl apply -f dashboard-ingress.yaml 
ingressroute.traefik.containo.us/dashboard configured

接下来使用白名单之外的ip访问测试

[root@tiaoban ~]# curl traefik.test.com/dashboard/
Forbidden[root@tiaoban ~]# 
[root@tiaoban ~]# curl -I http://traefik.test.com/dashboard/
HTTP/1.1 403 Forbidden
Date: Sun, 25 Sep 2022 13:32:49 GMT
Content-Length: 9
Content-Type: text/plain; charset=utf-8

基础用户认证

通常企业安全要求规范除了要对管理员页面限制访问ip外,还需要添加账号密码认证,而traefik默认没有提供账号密码认证功能,此时就可以通过BasicAuth中间件完成用户认证,只有认证通过的授权用户才可以访问页面。

参考文档:https://doc.traefik.io/traefik/middlewares/http/basicauth/

图片

使用basicAuth认证需要使用htpasswd工具生成密码文件,因此先安装httpd软件包

[root@k8s-master middleware]# dnf install -y httpd

使用htpasswd工具设置用户名密码,生成密钥文件

[root@k8s-master middleware]# htpasswd -bc basic-auth-secret cuiliang 123
Adding password for user cuiliang

 将生成的basic-auth-secret密码文件创建成secret资源

[root@k8s-master middleware]# kubectl create secret generic basic-auth --from-file=basic-auth-secret
secret/basic-auth created
[root@k8s-master middleware]# kubectl get secrets 
NAME                                     TYPE                                  DATA   AGE
basic-auth                               Opaque                                1      8s

接下来创建basicAuth中间件,指定使用刚刚创建的secret资源。

[root@k8s-master middleware]# cat basic-auth-middleware.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: basic-auth-middleware
spec:basicAuth:secret: basic-auth
[root@k8s-master middleware]# kubectl apply -f basic-auth-middleware.yaml 
middleware.traefik.containo.us/basic-auth-middleware created

修改dashboard的ingressRoute,添加basicAuth中间件

[root@k8s-master middleware]# cat basic-auth-middleware.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: basic-auth-middleware
spec:basicAuth:secret: basic-auth
[root@k8s-master middleware]# cat dashboard-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboard
spec:entryPoints:- webroutes:- match: Host(`traefik.test.com`)kind: Ruleservices:- name: api@internalkind: TraefikServicemiddlewares:- name: basic-auth-middleware
[root@k8s-master middleware]# kubectl apply -f basic-auth-middleware.yaml 
middleware.traefik.containo.us/basic-auth-middleware created

客户端访问验证,刷新页面后,弹出用户登录认证页面。

图片

修改请求/响应头信息

为了提高业务的安全性,安全团队会定期进行漏洞扫描,其中有些web漏洞就需要通过修改响应头处理,traefik的Headers中间件不仅可以修改返回客户端的响应头信息,还能修改反向代理后端service服务的请求头信息。

图片

例如对https://myapp2.test.com提高安全策略,强制启用HSTS HSTS:即HTTP严格传输安全响应头,收到该响应头的浏览器会在 63072000s(约 2 年)的时间内,只要访问该网站,即使输入的是 http,浏览器会自动跳转到 https。(HSTS 是浏览器端的跳转,之前的HTTP 重定向到 HTTPS是服务器端的跳转) 

参考文档https://doc.traefik.io/traefik/middlewares/http/headers/

定义响应头中间件Headers,指定响应内容中添加Strict-Transport-Security配置。

[root@k8s-master middleware]# cat hsts-header-middleware.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: hsts-header-middleware
spec:headers:customResponseHeaders:Strict-Transport-Security: 'max-age=63072000'
[root@k8s-master middleware]# kubectl apply -f hsts-header-middleware.yaml 
middleware.traefik.containo.us/hsts-header-middleware created

修改myapp2的ingressRoute,添加headers中间件

[root@k8s-master middleware]# cat myapp2-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2-tls
spec:entryPoints:- web- websecureroutes:- match: Host(`myapp2.test.com`)kind: Ruleservices:- name: myapp2port: 80 middlewares:- name: hsts-header-middlewaretls:secretName: myapp2-tls         # 指定tls证书名称
[root@k8s-master middleware]# kubectl apply -f myapp2-ingress.yaml 
ingressroute.traefik.containo.us/myapp2-tls configured

客户端访问验证,查看响应头信息

图片

限流

在实际生产环境中,流量限制也是经常用到的,它可以用作安全目的,比如可以减慢暴力密码破解的速率。通过将传入请求的速率限制为真实用户的典型值,并标识目标URL地址(通过日志),还可以用来抵御 DDOS 攻击。更常见的情况,该功能被用来保护下游应用服务器不被同时太多用户请求所压垮。

参考文档https://doc.traefik.io/traefik/middlewares/http/ratelimit/

先模拟正常情况,无任何限流措施,对myapp1使用ab工具进行压力测试,一共请求一百次,每次并发10次。测试结果失败的请求为0,总耗时0.412秒

[root@tiaoban ~]# ab -n 100 -c 10  "http://myapp1.test.com/"
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking myapp1.test.com (be patient).....doneServer Software:        nginx/1.12.2
Server Hostname:        myapp1.test.com
Server Port:            80Document Path:          /
Document Length:        65 bytesConcurrency Level:      10
Time taken for tests:   0.412 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      27700 bytes
HTML transferred:       6500 bytes
Requests per second:    242.78 [#/sec] (mean)
Time per request:       41.189 [ms] (mean)
Time per request:       4.119 [ms] (mean, across all concurrent requests)
Transfer rate:          65.68 [Kbytes/sec] receivedConnection Times (ms)min  mean[+/-sd] median   max
Connect:        1    4   4.3      3      16
Processing:     4   20  19.2     15     173
Waiting:        4   20  19.0     14     171
Total:          8   25  19.5     19     175Percentage of the requests served within a certain time (ms)50%     1966%     2775%     3080%     3290%     4395%     5598%     6199%    175100%    175 (longest request)

定义限流中间件RateLimit,指定1s内请求数平均值不大于10个,高峰最大值不大于50个。

[root@k8s-master middleware]# cat rate-limit-middleware.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: rate-limit-middleware
spec:rateLimit:burst: 10average: 50
[root@k8s-master middleware]# kubectl apply -f rate-limit-middleware.yaml 
middleware.traefik.containo.us/rate-limit-middleware created

修改myapp1的ingressRoute,添加RateLimit中间件

[root@k8s-master middleware]# cat myapp1-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp1
spec:entryPoints:- webroutes:- match: Host(`myapp1.test.com`)kind: Ruleservices:- name: myapp1  port: 80   middlewares:- name: rate-limit-middleware
[root@k8s-master middleware]# kubectl apply -f myapp1-ingress.yaml 
ingressroute.traefik.containo.us/myapp1 created

接下来继续使用ab工具进行压力测试,一共请求一百次,每次并发10次。测试结果失败的请求为82次,总耗时0.297秒

[root@tiaoban ~]# ab -n 100 -c 10  "http://myapp1.test.com/"
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking myapp1.test.com (be patient).....doneServer Software:        nginx/1.12.2
Server Hostname:        myapp1.test.com
Server Port:            80Document Path:          /
Document Length:        65 bytesConcurrency Level:      10
Time taken for tests:   0.297 seconds
Complete requests:      100
Failed requests:        82(Connect: 0, Receive: 0, Length: 82, Exceptions: 0)
Non-2xx responses:      82
Total transferred:      20562 bytes
HTML transferred:       2564 bytes
Requests per second:    336.30 [#/sec] (mean)
Time per request:       29.736 [ms] (mean)
Time per request:       2.974 [ms] (mean, across all concurrent requests)
Transfer rate:          67.53 [Kbytes/sec] receivedConnection Times (ms)min  mean[+/-sd] median   max
Connect:        1    3   3.4      2      17
Processing:     2   12  15.9      9     151
Waiting:        2   11  15.7      7     150
Total:          3   15  16.2     11     153Percentage of the requests served within a certain time (ms)50%     1166%     1475%     1880%     2190%     2495%     3498%     4999%    153100%    153 (longest request)

熔断

图片

熔断简介服务熔断的作用类似于我们家用的保险丝,当某服务出现不可用或响应超时的情况时,为了防止整个系统出现雪崩,暂时停止对该服务的调用。熔断器三种状态

  • Closed:关闭状态,所有请求都正常访问。

  • Open:打开状态,所有请求都会被降级。traefik会对请求情况计数,当一定时间内失败请求百分比达到阈值,则触发熔断,断路器会完全打开。

  • Recovering:半开恢复状态,open状态不是永久的,打开后会进入休眠时间。随后断路器会自动进入半开状态。此时会释放部分请求通过,若这些请求都是健康的,则会完全关闭断路器,否则继续保持打开,再次进行休眠计时

服务熔断原理(断路器的原理)统计用户在指定的时间范围(默认10s)之内的请求总数达到指定的数量之后,如果不健康的请求(超时、异常)占总请求数量的百分比(50%)达到了指定的阈值之后,就会触发熔断。触发熔断,断路器就会打开(open),此时所有请求都不能通过。在5s之后,断路器会恢复到半开状态(half open),会允许少量请求通过,如果这些请求都是健康的,那么断路器会回到关闭状态(close).如果这些请求还是失败的请求,断路器还是恢复到打开的状态(open).traefik支持的触发器

  • NetworkErrorRatio:网络错误率

  • ResponseCodeRatio:状态代码比率

  • LatencyAtQuantileMS:分位数的延迟(以毫秒为单位)

参考文档https://doc.traefik.io/traefik/middlewares/http/circuitbreaker/

定义熔断中间件circuitBreaker,指定50% 的请求比例响应时间大于 1MS 时熔断。

[root@k8s-master middleware]# cat circuit-breaker-middleware.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: circuit-breaker-middleware
spec:circuitBreaker:expression: LatencyAtQuantileMS(50.0) > 1
[root@k8s-master middleware]# kubectl apply -f circuit-breaker-middleware.yaml 
middleware.traefik.containo.us/circuit-breaker-middleware created

修改myapp1的ingressRoute,添加circuitBreaker中间件

root@k8s-master middleware]# cat myapp1-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp1
spec:entryPoints:- webroutes:- match: Host(`myapp1.test.com`)kind: Ruleservices:- name: myapp1  port: 80   middlewares:- name: circuit-breaker-middleware
root@k8s-master middleware]# kubectl apply -f myapp1-ingress.yaml 
ingressroute.traefik.containo.us/myapp1 created

继续进行压力测试,一共请求一千次,每次并发100次。触发熔断机制,测试结果失败的请求为999次,总耗时0.938秒。

[root@tiaoban ~]# ab -n 1000 -c 100  "http://myapp1.test.com/"
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking myapp1.test.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requestsServer Software:        nginx/1.12.2
Server Hostname:        myapp1.test.com
Server Port:            80Document Path:          /
Document Length:        65 bytesConcurrency Level:      100
Time taken for tests:   0.938 seconds
Complete requests:      1000
Failed requests:        999(Connect: 0, Receive: 0, Length: 999, Exceptions: 0)
Non-2xx responses:      999
Total transferred:      153124 bytes
HTML transferred:       19046 bytes
Requests per second:    1065.54 [#/sec] (mean)
Time per request:       93.849 [ms] (mean)
Time per request:       0.938 [ms] (mean, across all concurrent requests)
Transfer rate:          159.34 [Kbytes/sec] receivedConnection Times (ms)min  mean[+/-sd] median   max
Connect:        1   12   9.2     11      58
Processing:     2   66  53.3     53     314
Waiting:        1   63  51.7     51     309
Total:          5   79  52.8     66     326Percentage of the requests served within a certain time (ms)50%     6666%     8075%     9680%    10590%    12895%    16898%    28099%    309100%    326 (longest request)

自定义错误页

在实际的业务中,肯定会存在4XX 5XX相关的错误异常,如果每个应用都开发一个单独的错误页,无疑大大增加了开发成本,traefik同样也支持自定义错误页,但是需要注意的是,错误页面不是有traefik存储处理,而是通过定义中间件,将错误的请求重定向到其他的页面。

图片

参考文档:https://doc.traefik.io/traefik/middlewares/http/errorpages/

首先,我们先创建一个应用,使用flask开个一个简单的demo项目。这个web应用的功能是:当请求/时,返回状态码为200,当请求/400时,返回400状态码,当请求/500时,返回500状态码。应用的源代码如下:

  • app.py

    from flask import Flask, abortapp = Flask(__name__)@app.route('/')
    def hello_world():# put application's code herereturn'Hello World!'@app.route('/400')
    def error_404():abort(400)@app.route('/500')
    def error_500():abort(500)if __name__ == '__main__':app.run()
  • templates/index.html

    <!DOCTYPE html>
    <html lang="en"><head><meta charset="UTF-8"><title>flask</title></head><body><h1>hello flask</h1><img src="{{ url_for('static',filename='photo.jpg') }}" alt="photo"></body>
    </html>

为了方便大家测试,已将镜像打包上传至docker hub仓库

docker pull cuiliang0302/request-code:v2.0

接下来我们使用deployment控制器部署这个服务,并创建svc资源

[root@k8s-master middleware]# cat flask.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:name: flask
spec:selector:matchLabels:app: flasktemplate:metadata:labels:app: flaskspec:containers:- name: flaskimage: cuiliang0302/request-code:v1.0resources:limits:memory: "128Mi"cpu: "500m"ports:- containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:name: flask
spec:type: ClusterIPselector:app: flaskports:- port: 5000targetPort: 5000[root@k8s-master middleware]# kubectl apply -f flask.yaml 
deployment.apps/flask created
service/flask created

接下来创建ingressRouter资源

[root@k8s-master middleware]# cat flask-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: flask
spec:entryPoints:- webroutes:- match: Host(`flask.test.com`)kind: Ruleservices:- name: flask  port: 5000 
[root@k8s-master middleware]# kubectl apply -f flask-ingress.yaml 
ingressroute.traefik.containo.us/flask created

使用域名访问验证,先添加hosts解析记录192.168.93.128 flask.test.com,分别请求不同的路径,模拟4XX 5XX错误

[root@k8s-master middleware]# curl -I flask.test.com/
HTTP/1.1 200 OK
Content-Length: 12
Content-Type: text/html; charset=utf-8
Date: Wed, 28 Sep 2022 03:11:03 GMT
Server: Werkzeug/2.2.2 Python/3.10.1[root@k8s-master middleware]# curl -I flask.test.com/400
HTTP/1.1 400 Bad Request
Content-Length: 167
Content-Type: text/html; charset=utf-8
Date: Wed, 28 Sep 2022 03:11:07 GMT
Server: Werkzeug/2.2.2 Python/3.10.1[root@k8s-master middleware]# curl -I flask.test.com/500
HTTP/1.1 500 Internal Server Error
Content-Length: 265
Content-Type: text/html; charset=utf-8
Date: Wed, 28 Sep 2022 03:11:11 GMT
Server: Werkzeug/2.2.2 Python/3.10.1[root@k8s-master middleware]# curl -I flask.test.com/404
HTTP/1.1 404 Not Found
Content-Length: 207
Content-Type: text/html; charset=utf-8
Date: Wed, 28 Sep 2022 03:11:17 GMT
Server: Werkzeug/2.2.2 Python/3.10.1

现在提出一个新的需求,当我访问flask项目时,如果错误码为400,返回myapp1的页面,如果错误码为500,返回myapp2的页面(前提是myapp1和myapp2服务已创建)。我们创建errorpages中间件

[root@k8s-master middleware]# cat error-middleware.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: errors5
spec:errors:status:- "500-599"# query: /{status}.html   # 可以为每个页面定义一个状态码,也可以指定5XX使用统一页面返回query : /                 # 指定返回myapp2的请求路径service:name: myapp2port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: errors4
spec:errors:status:- "400-499"# query: /{status}.html   # 可以为每个页面定义一个状态码,也可以指定5XX使用统一页面返回query : /                 # 指定返回myapp1的请求路径service:name: myapp1port: 80
[root@k8s-master middleware]# kubectl apply -f error-middleware.yaml 
middleware.traefik.containo.us/errors5 created
middleware.traefik.containo.us/errors4 created

接下来修改ingressroute资源,添加错误码中间件

[root@k8s-master middleware]# cat flask-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: flask
spec:entryPoints:- webroutes:- match: Host(`flask.test.com`)kind: Ruleservices:- name: flask  port: 5000middlewares:- name: errors4- name: errors5
[root@k8s-master middleware]# kubectl apply -f error-middleware.yaml 
middleware.traefik.containo.us/errors5 created
middleware.traefik.containo.us/errors4 created

最后进行访问验证

# 测试200状态码
[root@k8s-master middleware]# curl -I flask.test.com/
HTTP/1.1 200 OK
Content-Length: 12
Content-Type: text/html; charset=utf-8
Date: Thu, 06 Oct 2022 00:36:19 GMT
Server: Werkzeug/2.2.2 Python/3.10.1[root@k8s-master middleware]# curl flask.test.com/
<!DOCTYPE html>
<html lang="en">
<head><meta charset="UTF-8"><title>flask</title>
</head>
<body>
<h1>hello flask</h1>
<img src="/static/photo.jpg" alt="photo">
</body>
# 测试400状态码
[root@k8s-master middleware]# curl -I flask.test.com/400
HTTP/1.1 400 Bad Request
Accept-Ranges: bytes
Content-Length: 65
Content-Type: text/html
Date: Thu, 06 Oct 2022 00:36:35 GMT
Etag: "5a98c760-41"
Last-Modified: Fri, 02 Mar 2018 03:39:12 GMT
Server: nginx/1.12.2[root@k8s-master middleware]# curl flask.test.com/400
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
# 测试500状态码
[root@k8s-master middleware]# curl -I flask.test.com/500
HTTP/1.1 500 Internal Server Error
Accept-Ranges: bytes
Content-Length: 65
Content-Type: text/html
Date: Thu, 06 Oct 2022 00:36:46 GMT
Etag: "5a9251f0-41"
Last-Modified: Sun, 25 Feb 2018 06:04:32 GMT
Server: nginx/1.12.2[root@k8s-master middleware]# curl flask.test.com/500
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
# 测试404状态码
[root@k8s-master middleware]# curl -I flask.test.com/404
HTTP/1.1 404 Not Found
Accept-Ranges: bytes
Content-Length: 65
Content-Type: text/html
Date: Thu, 06 Oct 2022 00:37:00 GMT
Etag: "5a98c760-41"
Last-Modified: Fri, 02 Mar 2018 03:39:12 GMT
Server: nginx/1.12.2[root@k8s-master middleware]# curl flask.test.com/404
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

数据压缩

有时候客户端和服务器之间会传输比较大的报文数据,这时候就占用较大的网络带宽和时长。为了节省带宽,加速报文的响应速速,可以将传输的报文数据先进行压缩,然后再进行传输,traefik也同样支持数据压缩。

图片

参考文档:https://doc.traefik.io/traefik/middlewares/http/compress/[10]

traefik默认只对大于1024字节,且请求标头包含Accept-Encoding gzip的资源进行压缩。可以指定排除特定类型不启用压缩或者根据内容大小来决定是否压缩。继续使用上面创建的flask应用,现在创建中间件,使用默认配置策略即可。

[root@k8s-master middleware]# cat compress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: compress
spec:compress: {}
[root@k8s-master middleware]# kubectl apply -f compress.yaml 
middleware.traefik.containo.us/compress created

修改flask的ingressrouter资源,指定数据压缩中间件

[root@k8s-master middleware]# cat flask-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: flask
spec:entryPoints:- webroutes:- match: Host(`flask.test.com`)kind: Ruleservices:- name: flask  port: 5000middlewares:- name: compress
[root@k8s-master middleware]# kubectl apply -f flask-ingress.yaml 
ingressroute.traefik.containo.us/flask created

接下来查看浏览器f12调试信息

图片资源大于1024字节,开启了压缩

图片

html资源小于1024字节,未启用压缩

图片

四、入门Traefik系列——服务配置与使用

简介

图片

traefik的路由规则就可以实现4层和7层的基本负载均衡操作,使用IngressRoute[1]/IngressRouteTCP[2]/IngressRouteUDP[3]资源即可。但是如果想要实现加权轮询、流量复制等高级操作,traefik抽象出了一个TraefikService[4]资源。此时整体流量走向为:外部流量先通过entryPoints端口进入traefik,然后由IngressRoute[5]/IngressRouteTCP[6]/IngressRouteUDP[7]匹配后进入TraefikService[8],在TraefikService[9]这一层实现加权轮循和流量复制,最后将请求转发至kubernetes的service。除此之外traefik还支持7层的粘性会话、健康检查、传递请求头、响应转发、故障转移等操作。

加权轮询(灰度发布)

灰度发布我们有时候也会称为金丝雀发布(Canary),主要就是让一部分测试的服务也参与到线上去,经过测试观察看是否符合上线要求,在traefik中,通过调整生产与测试服务的权重,实现灰度发布的功能。接下来配置加权轮循,仍然请求myapp.test.com,myapp1的负载权重为1,myapp2的负载权重为2。依旧是上面创建的两个deployment应用与对应的svc

[root@k8s-master udp]# kubectl get pod
NAME                                          READY   STATUS    RESTARTS   AGE
myapp1-795d947b45-9lsm6                       1/1     Running   1          25h
myapp2-6ffd54f76-ljkr9                        1/1     Running   1          25h
[root@k8s-master udp]# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                 2d22h
myapp1       ClusterIP   10.104.91.200   <none>        80/TCP                                                                                   25h
myapp2       ClusterIP   10.111.245.32   <none>        80/TCP

创建IngressRouter资源,配置域名为myapp.test.com,注意此时后端service配置TraefikService。

[root@k8s-master ingress]# cat myapp-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myappnamespace: default
spec:entryPoints:- webroutes:- match: Host(`myapp.test.com`)kind: Ruleservices:                     # 加权轮循时,后端service不再是k8s的service,而是traefik的TraefikService- name: wrr     namespace: defaultkind: TraefikService
[root@k8s-master ingress]# kubectl apply -f myapp-ingress.yaml 
ingressroute.traefik.containo.us/myapp created

创建TraefikService资源,名称与IngressRouter的TraefikService保持一致,services后端填写kubernetes的service,并指定权重。

[root@k8s-master ingress]# cat myapp-traefikService.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:name: wrrnamespace: defaultspec:weighted:services:- name: myapp1port: 80weight: 1- name: myapp2port: 80weight: 2
[root@k8s-master ingress]# kubectl apply -f myapp-traefikService.yaml 
traefikservice.traefik.containo.us/wrr unchanged

查看dashboard的配置信息,此时myapp2权重为2,myapp1权重为1。

图片

客户端访问测试,验证无误。

[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

镜像复制(流量复制)

traefik还支持镜像复制功能,是一种可以将流入流量复制并同时将其发送给其他服务的方法,镜像服务可以获得给定百分比的请求同时也会忽略这部分请求的响应,在实际生产中主要用于测试场景以及问题复现bug定位。依旧是上面创建的两个deployment应用与对应的svc

[root@k8s-master udp]# kubectl get pod
NAME                                          READY   STATUS    RESTARTS   AGE
myapp1-795d947b45-9lsm6                       1/1     Running   1          25h
myapp2-6ffd54f76-ljkr9                        1/1     Running   1          25h
[root@k8s-master udp]# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                 2d22h
myapp1       ClusterIP   10.104.91.200   <none>        80/TCP                                                                                   25h
myapp2       ClusterIP   10.111.245.32   <none>        80/TCP

创建IngressRouter资源,配置域名为myapp.test.com,注意此时后端service配置TraefikService。

[root@k8s-master ingress]# cat myapp-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myappnamespace: default
spec:entryPoints:- webroutes:- match: Host(`myapp.test.com`)kind: Ruleservices:                     # 流量复制时,后端service不再是k8s的service,而是traefik的TraefikService- name: mirror   namespace: defaultkind: TraefikService
[root@k8s-master ingress]# kubectl apply -f myapp-ingress.yaml 
ingressroute.traefik.containo.us/myapp created

创建TraefikService资源,名称与IngressRouter的TraefikService保持一致,services后端填写kubernetes的service,并设置复制流量比例。

[root@k8s-master ingress]# cat myapp-traefikService.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:name: mirrornamespace: defaultspec:mirroring:      # 所有流量全部请求到k8s的myapp1name: myapp1port: 80mirrors:      # 同时复制50%的请求到k8s的myapp2  - name: myapp2port: 80percent: 50
[root@k8s-master ingress]# kubectl apply -f myapp-traefikService.yaml 
traefikservice.traefik.containo.us/mirror created

查看dashboard的配置信息,此时myapp2类型为mirroring,比例为50%

图片

客户端访问测试,只响应了myapp1的内容。

[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

查看myapp1和myapp2的日志,发现有请求日志符合预期。

[root@k8s-master ingress]# kubectl logs myapp1-795d947b45-9lsm6 
10.244.0.0 - - [25/Sep/2022:08:45:33 +0000] "GET / HTTP/1.1" 200 65 "-""curl/7.61.1""192.168.93.1"
10.244.0.0 - - [25/Sep/2022:08:45:35 +0000] "GET / HTTP/1.1" 200 65 "-""curl/7.61.1""192.168.93.1"
10.244.0.0 - - [25/Sep/2022:08:45:36 +0000] "GET / HTTP/1.1" 200 65 "-""curl/7.61.1""192.168.93.1"
10.244.0.0 - - [25/Sep/2022:09:45:38 +0000] "GET / HTTP/1.1" 200 65 "-""curl/7.61.1""192.168.93.1"
[root@k8s-master ingress]# kubectl logs myapp2-6ffd54f76-ljkr9 
10.244.0.0 - - [25/Sep/2022:08:45:33 +0000] "GET / HTTP/1.1" 200 65 "-""curl/7.61.1""192.168.93.1"
10.244.0.0 - - [25/Sep/2022:08:45:36 +0000] "GET / HTTP/1.1" 200 65 "-""curl/7.61.1""192.168.93.1"

粘性会话(会话保持)

当我们使用traefik的负载均衡时,默认情况下轮循多个k8s的service服务,如果用户对同一内容的多次请求,可能被转发到了不同的后端服务器。假设用户发出请求被分配至服务器A,保存了一些信息在session中,该用户再次发送请求被分配到服务器B,要用之前保存的信息,若服务器A和B之间没有session粘滞,那么服务器B就拿不到之前的信息,这样会导致一些问题。traefik同样也支持粘性会话,可以让用户在一次会话周期内的所有请求始终转发到一台特定的后端服务器上。依旧是上面创建的两个deployment应用与对应的svc

[root@k8s-master udp]# kubectl get pod
NAME                                          READY   STATUS    RESTARTS   AGE
myapp1-795d947b45-9lsm6                       1/1     Running   1          25h
myapp2-6ffd54f76-ljkr9                        1/1     Running   1          25h
[root@k8s-master udp]# kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                                 2d22h
myapp1       ClusterIP   10.104.91.200   <none>        80/TCP                                                                                   25h
myapp2       ClusterIP   10.111.245.32   <none>        80/TCP

创建IngressRouter资源,配置域名为myapp.test.com,注意此时后端service配置TraefikService。

[root@k8s-master ingress]# cat myapp-ingress.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myappnamespace: default
spec:entryPoints:- webroutes:- match: Host(`myapp.test.com`)kind: Ruleservices:                     # 粘性会话依赖加权轮循,后端service不再是k8s的service,而是traefik的TraefikService- name: wrr  namespace: defaultkind: TraefikService
[root@k8s-master ingress]# kubectl apply -f myapp-ingress.yaml 
ingressroute.traefik.containo.us/myapp created

创建TraefikService资源,名称与IngressRouter的TraefikService保持一致,services后端填写kubernetes的service,并指定权重。

[root@k8s-master ingress]# cat myapp-traefikService.yaml 
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:name: wrrnamespace: defaultspec:weighted:services:- name: myapp1kind: Serviceport: 80weight: 1- name: myapp2kind: Serviceweight: 2port: 80sticky:                 # 开启粘性会话cookie:               # 基于cookie区分客户端name: lvl1          # 指定客户端请求时,包含的cookie名称
[root@k8s-master ingress]# kubectl apply -f myapp-traefikService.yaml 
traefikservice.traefik.containo.us/wrr unchanged

客户端携带cookie信息访问测试。

# lvl1为default-myapp2-80的请求全部由myapp2响应
[root@tiaoban ~]# curl -b "lvl1=default-myapp2-80" http://myapp.test.com
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl -b "lvl1=default-myapp2-80" http://myapp.test.com
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>
# lvl1为default-myapp1-80的请求全部由myapp1响应
[root@tiaoban ~]# curl -b "lvl1=default-myapp1-80" http://myapp.test.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@tiaoban ~]# curl -b "lvl1=default-myapp1-80" http://myapp.test.com
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.rhkb.cn/news/389397.html

如若内容造成侵权/违法违规/事实不符,请联系长河编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

CentOS7.9 利用 KubeKey 扩容 Kubernetes v1.26 Worker 节点实战

转载&#xff1a;CentOS7.9 利用 KubeKey 扩容 Kubernetes v1.26 Worker 节点实战 知识点 定级&#xff1a;入门级 KubeKey 安装部署 KubeSphere 和 Kubernetes KubeKey 定制化部署集群 KubeSphere v3.4.0 功能概览 Kubernetes 基本操作 CentOS 系统内核升级 演示服务器配…

Python爬虫:下载人生格言

Python爬虫:下载人生格言 爬取网页 将这些格言下载存储到本地 代码: import requests #导入requests库&#xff0c;用于提取网页 from lxml import etree#导入lxml库&#xff0c;用于Xpath数据解析#请求头 header{ user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) A…

游戏加速器推荐 网游加速器排行榜

游戏加速器推荐&#xff0c;玩游戏用什么加速器&#xff01;我得给你推荐一款我常用的。首先呢&#xff0c;就是深度加速器&#xff0c;它针对目前手游网游的游戏加速效果特别棒&#xff0c;而且界面也很友好。 另外&#xff0c;还有深度加速器&#xff0c;这款加速器不仅支持国…

【信创】Linux操作系统上使用命令查看设备序列号

原文链接&#xff1a;【信创】Linux操作系统上使用命令查看设备序列号 Hello&#xff0c;大家好啊&#xff01;今天给大家带来一篇关于在信创操作系统上如何查看设备序列号的文章。在进行设备管理、维护和售后服务时&#xff0c;设备序列号是一个非常重要的信息。本文将详细介绍…

HarmonyOS入门

HarmonyOS 系统定义 HarmonyOS 是一款面向万物互联时代的、全新的分布式操作系统。 Harmony提出了基于同一套系统能力、适配多种终端形态的分布式理念&#xff0c;能够支持手机、平板、智能穿戴、智慧屏、车机等多种终端设置&#xff0c;提供全场景&#xff08;移动办公、运动…

simulink绘制bode图

&#x1f3c6;本文收录于《CSDN问答解惑-专业版》专栏&#xff0c;主要记录项目实战过程中的Bug之前因后果及提供真实有效的解决方案&#xff0c;希望能够助你一臂之力&#xff0c;帮你早日登顶实现财富自由&#x1f680;&#xff1b;同时&#xff0c;欢迎大家关注&&收…

【PGCCC】PostgreSQL 14 小版本分析,有那个版本不建议使用#PG中级

以下是对 PostgreSQL 14 各个小版本的详细分析&#xff0c;包括每个版本的主要变化、修复的 bug 和潜在的问题&#xff1a; PostgreSQL 14.0 发布日期&#xff1a;2021 年 9 月 30 日 主要变化&#xff1a; 增加了并行查询的改进&#xff0c;提升了性能。增强了 JSON 数据类…

【人工智能】Transformers之Pipeline(三):文本转音频(text-to-audio/text-to-speech)

​​​​​​​ 一、引言 pipeline&#xff08;管道&#xff09;是huggingface transformers库中一种极简方式使用大模型推理的抽象&#xff0c;将所有大模型分为音频&#xff08;Audio&#xff09;、计算机视觉&#xff08;Computer vision&#xff09;、自然语言处理&#x…

查找算法:线性查找,golang实现

目录 前言 线性查找 代码示例 1. 算法包 2. 线性查找代码 3. 模拟程序 4. 运行程序 循环次数 假如目标值正好在数组中的第一位 假如目标值正好在数组中的第五位 假如目标值正好在数组中的最后一位 假如目标值不在数组中 线性查找的思想 1. 顺序遍历 2. 比较 3.…

计算机毕业设计选题推荐-课程教学辅助系统-Java/Python项目实战

✨作者主页&#xff1a;IT研究室✨ 个人简介&#xff1a;曾从事计算机专业培训教学&#xff0c;擅长Java、Python、微信小程序、Golang、安卓Android等项目实战。接项目定制开发、代码讲解、答辩教学、文档编写、降重等。 ☑文末获取源码☑ 精彩专栏推荐⬇⬇⬇ Java项目 Python…

金牌挑战——奥运知识大比拼

巴黎奥运会乒乓球项目1日结束男、女单打四分之一决赛。在四分之一决赛中&#xff0c;上演惊天逆转&#xff0c;在大比分0:2、2:3落后的逆势下力挽狂澜&#xff0c;4:3艰难战胜对手&#xff0c;进入四强。祝贺&#xff01; 为此&#xff0c;我立马出了一个知识竞赛&#xff0c;…

vmware安装银河麒麟V10高级服务器操作系统

vmware安装银河麒麟V10高级服务器操作系统 1、下载银河麒麟V10镜像2、VMware安装银河麒麟V10高级服务器操作系统2.1、新建虚拟机2.2、安装虚拟机 3、配置银河麒麟V10高级服务器操作系统3.1、安装vmware tools3.2、配置静态IP地址 和 dns3.3、查看磁盘分区3.4、查看系统版本、内…

正则表达式与文本处理

目录 一、正则表达式 1、正则表达式定义 1.1正则表达式的概念及作用 1.2、正则表达式的工具 1.3、正则表达式的组成 2、基础正则表达式 3、扩展正则表达式 4、元字符操作 4.1、查找特定字符 4.2、利用中括号“[]”来查找集合字符 4.3、查找行首“^”与行尾字符“$”…

[C++探索]初始化列表,static成员,友元函数,内部类,匿名对象

&#x1f496;&#x1f496;&#x1f496;欢迎来到我的博客&#xff0c;我是anmory&#x1f496;&#x1f496;&#x1f496; 又和大家见面了 欢迎来到C探索系列 作为一个程序员你不能不掌握的知识 先来自我推荐一波 个人网站欢迎访问以及捐款 推荐阅读 如何低成本搭建个人网站…

二百五十二、OceanBase——Linux上安装OceanBase数据库(二):新用户配置ssh免密登录

一、目的 在OBD页面上部署OceanBase数据库时发现&#xff0c;需要把新用户也要配置ssh免密登录 二、前提 root用户已经设置免密登录 三、配置步骤 1 切换到新用户obadmin [roothurys23 ~]# su obadmin 2 执行命令生成秘钥文件 [obadminhurys23 oceanbase]$ ssh-keygen …

SSH实现电脑VScode免密登录到虚拟机其原理

在网上想看一下这个原理。发现写的还是比较乱&#xff0c;所以自己总结了一份方便回顾 SSH免密登录的原理主要基于非对称密钥加密技术&#xff0c;比较常用的是RSA算法。 以下是SSH免密登录的详细步骤和原理&#xff1a; 1. 生成密钥对 在客户端上生成一对密钥&#xff0c;…

初识Docker及管理Docker

Docker部署 初识DockerDocker是什么Docker的核心概念镜像容器仓库 容器优点容器在内核中支持2种重要技术&#xff1a;Docker容器与虚拟机的区别 安装Docker源码安装yum安装检查Docker Docker 镜像操作配置镜像加速器&#xff08;阿里系&#xff09;搜索镜像获取镜像查看镜像信息…

【综合案例】使用DevEco Studio编写B站视频卡片

效果展示 知识点 层叠布局 介绍&#xff1a;层叠布局具有较强的 组件层叠 能力。 使用场景&#xff1a;卡片层叠效果 特点&#xff1a;层叠操作 更简洁&#xff0c;编码效率更高。【绝对定位的优势是更灵活】 Stack容器内的子元素顺序是先写的在最下面&#xff0c;即从下到上依…

Linux服务器CPU使用率或CPU负载较高问题的排查及解决方案

本文主要介绍当Linux系统ECS实例CPU使用率或CPU负载较高时&#xff0c;如何排查分析及常见案例说明。 操作场景 在您使用ECS实例过程中&#xff0c;可能会遇到实例CPU使用率或CPU负载持续较高的情况&#xff0c;您可以按照以下步骤排查定位具体问题。 找到影响CPU使用率或CPU…

每日期刊分享

检索:知网 G4 刊期:现收25年3-4月刊版面&#xff0c;预计25年4-5月出刊: 收稿范围:收小学到高中的教育教学稿件