网站后台建设计划书,品牌高端网站建设,网站建设qinnet,温州敎玩具网站建设一、入门Traefik系列——基础简介 官方文档
https://doc.traefik.io/traefik/[1]
简介
Traefik是一个为了让部署微服务更加便捷而诞生的现代HTTP反向代理、负载均衡工具。它支持多种后台 (Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd, Zookeeper, BoltDB, Re…一、入门Traefik系列——基础简介 官方文档
https://doc.traefik.io/traefik/[1]
简介
Traefik是一个为了让部署微服务更加便捷而诞生的现代HTTP反向代理、负载均衡工具。它支持多种后台 (Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd, Zookeeper, BoltDB, Rest API, file…) 来自动化、动态的应用它的配置文件设置。
流量示意图 核心概念
当请求Traefik时请求首先到entrypoints然后分析传入的请求查看他们是否与定义的Routers匹配。如果匹配则会通过一系列middlewares处理再到traefikServices上做流量转发最后请求到kubernetes的services上。 Providers用来自动发现平台上的服务可以是编排工具、容器引擎云提供商或者键值存储。Traefik通过查询Providers的API来查询路由的相关信息一旦检测到变化就会动态的更新路由。 Entrypoints监听传入的流量是网络的入口点定义了接受请求的端口(HTTP或者TCP) Routers分析请求(host,path,headers,SSL等)负责将传入的请求连接到可以处理这些请求的服务上去。 Middlewares中间件用来修改请求或者根据请求来做出判断中间件被附件到路由上是一种在请求发送到服务之前调整请求的一种方法。 Service将请求转发给应用负责配置如何最终将处理传入请求的实际服务Traefik的Service介于Middlewares与KubernetesService之间可以实现加权负载、流量复制等功能。
traefik组件与nginx类比
组件名称功能nginx相同概念Providers监听路由信息变化更新路由修改nginx配置reload服务。Entrypoints网络入口监听传入的流量配置文件listen指定监听端口Routers分析传入的请求匹配规则配置文件server_namelocationMiddlewares中间件修改请求或响应location配置段中添加的缓存、压缩、请求头等配置Service请求转发http配置段中的upstream
Nginx-Ingress和traefik区别
Ingress Controller
k8s 是通过一个又一个的 controller 来负责监控、维护集群状态。Ingress Controller 就是监控 Ingress 路由规则的一个变化然后跟 k8s 的资源操作入口 api-server 进行通信交互。K8s 并没有自带 Ingress Controller它只是一种标准具体实现有多种需要自己单独安装常用的是 Nginx Ingress Controller 和 Traefik Ingress Controller。Ingress Controller 收到请求匹配 Ingress 转发规则匹配到了就转发到后端 Service而 Service 可能代表的后端 Pod 有多个选出一个转发到那个 Pod最终由那个 Pod 处理请求。 与kubernetes交互
nginx-ingress由于微服务架构以及 Docker 技术和 kubernetes 编排工具最近几年才开始逐渐流行所以一开始的反向代理服务器比如 nginx、apache 并未提供其支持所以才会出现 Ingress Controller 这种东西来做 kubernetes 和前端负载均衡器如 nginx 之间做衔接即 Ingress Controller 的存在就是为了能跟 kubernetes 交互然后写入nginx 配置最后reload。使用nginx作为前端负载均衡通过ingress controller不断的和kubernetes api交互实时获取后端servicepod等的变化然后动态更新nginx配置并刷新使配置生效达到服务发现的目的。 traefiktraefik本身设计的就能够实时跟kubernetes api交互感知后端servicepod等的变化自动更新配置并重载。
traefik优点 不需要安装其他依赖使用 GO 语言编译可执行文件 支持多种后台如 Docker, Swarm mode, Kubernetes, Marathon, Consul, Etcd, Rancher, Amazon ECS 等等 支持 REST API 配置文件热重载可自动监听配置改动、发现新服务并自动更新无需人工重启 支持熔断、限流功能 支持轮训、负载均衡 提供简洁的 UI 界面 支持 Websocket, HTTP/2, GRPC 自动更新 HTTPS 证书 支持高可用集群模式
Nginx和Traefik横向对比
Nginx IngressTraefik ingress协议http/https、http2、grpc、tcp/udphttp/https、http2、grpc、tcp、tcptls路由匹配host、pathhost、path、headers、query、path prefix、method命名空间支持-共用或指定命名空间部署策略-金丝雀部署、蓝绿部署、灰度部署upstream探测重试、超时、心跳探测重试、超时、心跳探测、熔断负载均衡算法RR、会话保持、最小连接、最短时间、一致性hashWRR、动态RR、会话保持优点简单易用易接入Golang编写部署容易支持众多的后端内置WebUI缺点没有解决nginx reload插件多但是扩展性能差性能略逊于NGINX但强于HAProxy
二、入门Traefik系列——部署与配置
helm部署
参考文档
官方文档https://doc.traefik.io/traefik/getting-started/install-traefik/[1]
gtihub地址https://github.com/traefik/traefik-helm-chart[2]
必要条件
Kubernetes版本1.14
Helm版本3
安装traefik # 添加repo
[rootk8s-master traefik]# helm repo add traefik https://helm.traefik.io/traefik
# 更新repo仓库资源
[rootk8s-master traefik]# helm repo update
# 查看repo仓库traefik
[rootk8s-master traefik]# helm search repo traefik
NAME CHART VERSION APP VERSION DESCRIPTION
stable/traefik 1.87.7 1.7.26 DEPRECATED - A Traefik based Kubernetes ingress...
traefik/traefik 10.24.3 2.8.5 A Traefik based Kubernetes ingress controller# 创建traefik-v2名称空间
# 创建traefik名称空间
[rootk8s-master traefik]# kubectl create ns traefik
# 安装traefik
[rootk8s-master traefik]# helm install --namespacetraefik traefik traefik/traefik
# 查看helm列表
[rootk8s-master traefik]# helm list -n traefik
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
traefik traefik 1 2022-09-22 11:35:22.176916449 0800 CST deployed traefik-10.24.3 2.8.5
# 查看pod资源信息
[rootk8s-master traefik]# kubectl get pod -n traefik
NAME READY STATUS RESTARTS AGE
traefik-5bfc574f88-vz4zr 1/1 Running 0 65s
端口转发dashboard服务
默认情况下由于安全考虑不会公开 Traefik 仪表板。可以通过端口转发实现仪表板访问
kubectl port-forward -n traefik deployment/traefik --address 0.0.0.0 9000:9000
# 可通过网址访问http://k8s-master:9000/dashboard/
# 如果出现以下报错信息无法访问
Forwarding from 0.0.0.0:9000 - 9000
Handling connection for 9000
E1229 13:10:40.655810 243434 portforward.go:400] an error occurred forwarding 9000 - 9000: error forwarding port 9000 to pod de96185927c5a2a6150c67130c14ef0f97f14900144389122570ccfc4b5b3179, uid : unable to do port forwarding: socat not found
Handling connection for 9000
E1229 13:11:06.847006 243434 portforward.go:400] an error occurred forwarding 9000 - 9000: error forwarding port 9000 to pod de96185927c5a2a6150c67130c14ef0f97f14900144389122570ccfc4b5b3179, uid : unable to do port forwarding: socat not found
Handling connection for 9000# 解决办法在node节点安装socat
yum install socat
域名访问dashboard服务
使用helm部署的traefik默认使用LoadBalancer暴露服务如果想使用此方式访问首先要部署MetalLB才能分配到EXTERNAL-IP [rootk8s-master traefik]# ls
dashboard-ingress.yaml dashboard-svc.yaml dashboard.yaml
[rootk8s-master traefik]# kubectl get svc -n traefik
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik LoadBalancer 10.97.204.137 192.168.10.201 80:31718/TCP,443:31294/TCP 20m
然后配置dashboard的ingress资源kubectl apply -f dashboard.yaml apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboardnamespace: traefik
spec:entryPoints:- webroutes:- match: Host(traefik.test.com) (PathPrefix(/dashboard) || PathPrefix(/api))kind: Ruleservices:- name: apiinternalkind: TraefikService
接下来使用集群外部机器访问添加hosts解析
192.168.10.201 traefik.test.com
# ip 为LoadBalancer的EXTERNAL-IP域名为ingress配置的域名 image.png
YAML自定义部署
helm虽然实现了一键安装部署但是查看helm包的value.yaml配置发现总共有500多行配置当需要修改配置项或者对traefik做一下自定义配置时并不灵活。如果只是使用traefik的基础功能推荐使用helm部署。如果想深入研究使用traefik的话推荐使用自定义方式部署系列文章后续均使用自定义方式做演示。
环境准备
k8s版本1.19.16
traefik版本2.8.7
其中master节点充当边缘节点安装两块网卡eth0k8s集群内网ip eth1公网ip
官方文档https://doc.traefik.io/traefik/providers/kubernetes-crd/[4]
官方示例文件https://github.com/traefik/traefik/blob/master/docs/content/user-guides/crd-acme/index.md[5](示例文件仅提供最基本的配置且所有配置项通过args传参仅供参考并不推荐用于生产)
创建CRD资源 这里要注意你的k8s版本从k8s 1.16开始废弃apiextensions.k8s.io/v1beta11.22完全删除。对于k8s 1.16 以上版本使用apiextensions.k8s.io/v1 [rootk8s-master traefik]# wget https://raw.githubusercontent.com/traefik/traefik/v2.8/docs/content/reference/dynamic-configuration/kubernetes-crd-definition-v1.yml
[rootk8s-master traefik]# kubectl apply -f kubernetes-crd-definition-v1.yml
[rootk8s-master traefik]# kubectl get crd
NAME CREATED AT
ingressroutes.traefik.containo.us 2022-09-22T10:01:07Z
ingressroutetcps.traefik.containo.us 2022-09-22T10:01:07Z
ingressrouteudps.traefik.containo.us 2022-09-22T10:01:07Z
ipaddresspools.metallb.io 2022-09-23T01:14:18Z
l2advertisements.metallb.io 2022-09-23T01:14:18Z
middlewares.traefik.containo.us 2022-09-22T10:01:07Z
middlewaretcps.traefik.containo.us 2022-09-22T10:01:07Z
serverstransports.traefik.containo.us 2022-09-22T10:01:07Z
tlsoptions.traefik.containo.us 2022-09-22T10:01:08Z
tlsstores.traefik.containo.us 2022-09-22T10:01:08Z
traefikservices.traefik.containo.us 2022-09-22T10:01:08Z
创建RBAC资源
[rootk8s-master traefik]# wget https://raw.githubusercontent.com/traefik/traefik/v2.8/docs/content/reference/dynamic-configuration/kubernetes-crd-rbac.yml
dashboard-ingress.yaml kubernetes-crd-definition-v1.yml
[rootk8s-master traefik]# vim kubernetes-crd-rbac.yml
[rootk8s-master traefik]# kubectl apply -f kubernetes-crd-rbac.yml
[rootk8s-master traefik]# kubectl get clusterrole | grep traefik
traefik-ingress-controller 2022-09-23T14:10:06Z
[rootk8s-master traefik]# kubectl get clusterrolebinding | grep traefik
traefik-ingress-controller ClusterRole/traefik-ingress-controller 2m
创建traefik配置文件 在 Traefik 中有三种方式定义静态配置在配置文件中、在命令行参数中、通过环境变量传递由于 Traefik 配置很多通过 CLI 定义不是很方便一般时候选择将其配置选项放到配置文件中然后存入 ConfigMap将其挂入 traefik 中。参考文档https://doc.traefik.io/traefik/getting-started/configuration-overview/ 创建 traefik-config.yaml 文件 kind: ConfigMap
apiVersion: v1
metadata:name: traefik-config
data:traefik.yaml: |-global:checkNewVersion: false # 周期性的检查是否有新版本发布sendAnonymousUsage: false # 周期性的匿名发送使用统计信息serversTransport:insecureSkipVerify: true # Traefik忽略验证代理服务的TLS证书api:insecure: true # 允许HTTP 方式访问APIdashboard: true # 启用Dashboarddebug: false # 启用Debug调试模式metrics:prometheus: # 配置Prometheus监控指标数据并使用默认配置addRoutersLabels: true # 添加routers metricsentryPoint: metrics # 指定metrics监听地址entryPoints:web:address: :80 # 配置80端口并设置入口名称为webforwardedHeaders: insecure: true # 信任所有的forward headerswebsecure:address: :443 # 配置443端口并设置入口名称为 websecureforwardedHeaders: insecure: truetraefik:address: :9000 # 配置9000端口并设置入口名称为 dashboardmetrics:address: :9100 # 配置9100端口作为metrics收集入口tcpep:address: :9200 # 配置9200端口作为tcp入口udpep:address: :9300/udp # 配置9300端口作为udp入口providers:kubernetesCRD: # 启用Kubernetes CRD方式来配置路由规则ingressClass: allowCrossNamespace: true #允许跨namespaceallowEmptyServices: true #允许空endpoints的servicelog:filePath: /etc/traefik/logs/traefik.log # 设置调试日志文件存储路径如果为空则输出到控制台level: INFO # 设置调试日志级别format: common # 设置调试日志格式accessLog:filePath: /etc/traefik/logs/access.log # 设置访问日志文件存储路径如果为空则输出到控制台format: common # 设置访问调试日志格式bufferingSize: 0 # 设置访问日志缓存行数filters:statusCodes: [200] # 设置只保留指定状态码范围内的访问日志retryAttempts: true # 设置代理访问重试失败时保留访问日志minDuration: 20 # 设置保留请求时间超过指定持续时间的访问日志fields: # 设置访问日志中的字段是否保留keep保留、drop不保留defaultMode: keep # 设置默认保留访问日志字段names: # 针对访问日志特别字段特别配置保留模式ClientUsername: dropStartUTC: drop # 禁用日志timestamp使用UTCheaders: # 设置Header中字段是否保留defaultMode: keep # 设置默认保留Header中字段names: # 针对Header中特别字段特别配置保留模式#User-Agent: redact # 可以针对指定agentAuthorization: dropContent-Type: keep
创建 Traefik ConfigMap 资源kubectl apply -f traefik-config.yaml 节点设置label标签 模拟实际生产环境假设集群中master节点安装两块网卡充当边缘节点需要提前给master节点设置 Label这样当程序部署时 Pod 会自动调度到设置 Label 的节点上。 master节点网络信息如下
网卡名称ip用途ens33192.168.10.10k8s集群内部ipens160192.168.93.128公网ip用于外部访问
给master节点设置标签kubectl label nodes k8s-master IngressProxytrue 查看节点label信息
[rootk8s-master traefik]# kubectl get nodes --show-labels
NAME STATUS ROLES AGE VERSION LABELS
k8s-master Ready master 28h v1.19.16 IngressProxytrue,beta.kubernetes.io/archamd64,beta.kubernetes.io/oslinux,kubernetes.io/archamd64,kubernetes.io/hostnamek8s-master,kubernetes.io/oslinux,node-role.kubernetes.io/master
k8s-work1 Ready none 28h v1.19.16 beta.kubernetes.io/archamd64,beta.kubernetes.io/oslinux,kubernetes.io/archamd64,kubernetes.io/hostnamek8s-work1,kubernetes.io/oslinux
k8s-work2 Ready none 28h v1.19.16 beta.kubernetes.io/archamd64,beta.kubernetes.io/oslinux,kubernetes.io/archamd64,kubernetes.io/hostnamek8s-work2,kubernetes.io/oslinux
Deployment部署traefik 使用DeamonSet或者Deployment均可部署此处使用Deployment方式部署 Traefik副本数设置为1调度至IngressProxytrue的那台master边缘节点并使用host网络模式提高网络入口的网络性能 创建 traefik 部署文件 traefik-deployment.yaml apiVersion: v1
kind: ServiceAccount
metadata:namespace: defaultname: traefik-ingress-controller
---
apiVersion: apps/v1
kind: Deployment
metadata:name: traefik-ingress-controllernamespace: defaultlabels:app: traefik
spec:replicas: 1 # 副本数为1因为集群只设置一台master为边缘节点selector:matchLabels:app: traefiktemplate:metadata:name: traefiklabels:app: traefikspec:serviceAccountName: traefik-ingress-controllerterminationGracePeriodSeconds: 1containers:- name: traefikimage: traefik:v2.8.7env:- name: KUBERNETES_SERVICE_HOST # 手动指定k8s api,避免网络组件不稳定。value: 192.168.10.10- name: KUBERNETES_SERVICE_PORT_HTTPS # API server端口value: 6443- name: KUBERNETES_SERVICE_PORT # API server端口value: 6443- name: TZ # 指定时区value: Asia/Shanghaiports:- name: webcontainerPort: 80hostPort: 80 # 将容器端口绑定所在服务器的 80 端口- name: websecurecontainerPort: 443hostPort: 443 # 将容器端口绑定所在服务器的 443 端口- name: admincontainerPort: 9000 # Traefik Dashboard 端口- name: metricscontainerPort: 9100 # metrics端口- name: tcpepcontainerPort: 9200 # tcp端口- name: udpepcontainerPort: 9300 # udp端口securityContext: # 只开放网络权限 capabilities:drop:- ALLadd:- NET_BIND_SERVICEargs:- --configfile/etc/traefik/config/traefik.yamlvolumeMounts:- mountPath: /etc/traefik/configname: config- mountPath: /etc/traefik/logsname: logdir- mountPath: /etc/localtimename: timezonereadOnly: truevolumes:- name: configconfigMap:name: traefik-config - name: logdirhostPath:path: /data/traefik/logstype: DirectoryOrCreate- name: timezone #挂载时区文件hostPath:path: /etc/localtimetype: Filetolerations: # 设置容忍所有污点防止节点被设置污点- operator: ExistshostNetwork: true # 开启host网络提高网络入口的网络性能nodeSelector: # 设置node筛选器在特定label的节点上启动IngressProxy: true # 调度至IngressProxy: true的节点
部署并查看deployment资源信息
[rootk8s-master traefik]# kubectl apply -f traefik-deployment.yaml
NAME READY STATUS RESTARTS AGE
traefik-ingress-controller-6c79ff9645-hh559 1/1 Running 0 29m
[rootk8s-master traefik]# kubectl get pod
NAME READY STATUS RESTARTS AGE
traefik-ingress-controller-6c79ff9645-hh559 1/1 Running 0 29m 14h
创建service资源
创建 traefik服务资源文件 traefik-svc.yaml
# traefik-svc.yaml
apiVersion: v1
kind: Service
metadata:name: traefik
spec:type: NodePort ## 官网示例为ClusterIP,为方便演示此处改为NodePortselector:app: traefikports:- name: webprotocol: TCPport: 80targetPort: 80- name: websecureprotocol: TCPport: 443targetPort: 443- name: adminprotocol: TCPport: 9000targetPort: 9000- name: metricsprotocol: TCPport: 9100targetPort: 9100- name: tcpepprotocol: TCPport: 9200targetPort: 9200- name: udpepprotocol: UDPport: 9300targetPort: 9300
部署并查看svc资源信息[rootk8s-master traefik]# kubectl apply -f traefik-svc.yaml
[rootk8s-master traefik]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) 14h
traefik NodePort 10.102.64.244 none 80:31843/TCP,443:30318/TCP,9000:32689/TCP,9100:31527/TCP,9200:32432/TCP,9300:30608/UDP 2m35s
NodePort方式访问dashboard
traefik的dashboard使用nodeport暴露服务将9000端口映射为32689现在访问http://192.168.10.10:32689/ dashboard配置http域名访问 Traefik 应用已经部署完成treafik的Dashboard为svc类型是NodePort接下来配置域名规则模拟外部用户通过公网IP访问dashboard应用 Traefik创建路由规则有多种方式比如 原生Ingress写法 使用CRD IngressRoute方式 使用GatewayAPI的方式
相较于原生Ingress写法ingressRoute是2.1以后新增功能简单来说他们都支持路径(path)路由和域名(host)HTTP路由以及HTTPS配置区别在于IngressRoute需要定义CRD扩展但是它支持了TCP、UDP路由以及中间件等新特性强烈推荐使用ingressRouteGatewayAPI方式后续再详细介绍。
创建 Traefik Dashboard 路由规则文件 traefik-dashboard-route.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboard
spec:entryPoints:- webroutes:- match: Host(traefik.test.com)kind: Ruleservices:- name: apiinternalkind: TraefikService
创建ingress资源
[rootk8s-master traefik]# kubectl apply -f traefik-dashboard-route.yaml
[rootk8s-master traefik]# kubectl get ingressroute -n traefik
NAME AGE
traefik-dashboard-route 44m
[rootk8s-master traefik]# kubectl get ingressroute
NAME AGE
dashboard 19m
配置 Hosts客户端想通过域名访问服务必须要进行 DNS 解析由于这里没有 DNS 服务器进行域名解析所以直接修改客户端端的 hosts 文件将 Traefik 所在的节点的 公网IP 和自定义 host 绑定。
打开电脑的 Hosts 配置文件往其加入下面配置192.168.93.128 traefik.test.com
配置完成后打开浏览器输入地址http://traefik.test.com 打开 Traefik Dashboard。 其他配置
强制使用TLS v1.2 如今TLS v1.0 和 v1.1 因为存在安全问题现在已被弃用。为了保障系统安全所有入口路由都应该强制使用TLS v1.2 或更高版本。 参考文档https://doc.traefik.io/traefik/user-guides/crd-acme/#force-tls-v12
[rootk8s-master traefik]# cat tlsoption.yml
---
apiVersion: traefik.containo.us/v1alpha1
kind: TLSOption
metadata:name: defaultnamespace: default
spec:minVersion: VersionTLS12cipherSuites:- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 # TLS 1.2- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 # TLS 1.2- TLS_AES_256_GCM_SHA384 # TLS 1.3- TLS_CHACHA20_POLY1305_SHA256 # TLS 1.3curvePreferences:- CurveP521- CurveP384sniStrict: true
[rootk8s-master traefik]# kubectl apply -f tlsoption.yml
tlsoption.traefik.containo.us/default created
日志轮换 官方并没有日志轮换的功能但是traefik收到USR1信号后会重建日志文件因此可以通过logrotate实现日志轮换 参考文档https://doc.traefik.io/traefik/observability/logs/[8]
在/etc/logrotate.d创建下traefik目录
mkdir -p /etc/logrotate.d/traefik
配置logrotate文件
/data/traefik/logs/*.log {dailyrotate 15missingoknotifemptycompressdateextdateyesterdaydateformat .%Y-%m-%dcreate 0644 root rootpostrotatedocker kill --signalUSR1 $(docker ps | grep traefik |grep -v pause| awk {print $1})endscript}
添加crontab定时任务
sudo echo 0 0 * * * /usr/sbin/logrotate -f /etc/logrotate.d/traefik/traefikLogrotate /dev/null 21 /etc/crontab
多控制器
有的业务场景下可能需要在一个集群中部署多个 traefik不同的实例控制不同的 IngressRoute 资源对象要实现该功能有两种方法
通过 annotations 注解筛选
首先在traefik配置文件中的providers下增加Ingressclass参数指定具体的值。
参考文档https://doc.traefik.io/traefik/providers/kubernetes-crd/#ingressclass kind: ConfigMap
apiVersion: v1
metadata:name: traefik-config
data:traefik.yaml: |-global:checkNewVersion: false # 周期性的检查是否有新版本发布sendAnonymousUsage: false # 周期性的匿名发送使用统计信息serversTransport:insecureSkipVerify: true # Traefik忽略验证代理服务的TLS证书api:insecure: true # 允许HTTP 方式访问APIdashboard: true # 启用Dashboarddebug: false # 启用Debug调试模式metrics:prometheus: # 配置Prometheus监控指标数据并使用默认配置addRoutersLabels: true # 添加routers metricsentryPoint: metrics # 指定metrics监听地址entryPoints:web:address: :80 # 配置80端口并设置入口名称为webforwardedHeaders: insecure: true # 信任所有的forward headerswebsecure:address: :443 # 配置443端口并设置入口名称为 websecureforwardedHeaders: insecure: truetraefik:address: :9000 # 配置9000端口并设置入口名称为 dashboardmetrics:address: :9100 # 配置9100端口作为metrics收集入口tcpep:address: :9200 # 配置9200端口作为tcp入口udpep:address: :9300/udp # 配置9300端口作为udp入口providers:kubernetesCRD: # 启用Kubernetes CRD方式来配置路由规则ingressClass: traefik-v2.8 # 指定traefik的ingressClass实例名称allowCrossNamespace: true #允许跨namespaceallowEmptyServices: true #允许空endpoints的servicelog:filePath: /etc/traefik/logs/traefik.log # 设置调试日志文件存储路径如果为空则输出到控制台level: INFO # 设置调试日志级别format: common # 设置调试日志格式accessLog:filePath: /etc/traefik/logs/access.log # 设置访问日志文件存储路径如果为空则输出到控制台format: common # 设置访问调试日志格式bufferingSize: 0 # 设置访问日志缓存行数filters:statusCodes: [200] # 设置只保留指定状态码范围内的访问日志retryAttempts: true # 设置代理访问重试失败时保留访问日志minDuration: 20 # 设置保留请求时间超过指定持续时间的访问日志fields: # 设置访问日志中的字段是否保留keep保留、drop不保留defaultMode: keep # 设置默认保留访问日志字段names: # 针对访问日志特别字段特别配置保留模式ClientUsername: dropStartUTC: drop # 禁用日志timestamp使用UTCheaders: # 设置Header中字段是否保留defaultMode: keep # 设置默认保留Header中字段names: # 针对Header中特别字段特别配置保留模式#User-Agent: redact # 可以针对指定agentAuthorization: dropContent-Type: keep 接下来在IngressRoute 资源对象中的annotations参数中添加 kubernetes.io/ingress.class: traefik-v2.8即可 apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboardannotations:kubernetes.io/ingress.class: traefik-v2.8 # 因为静态配置文件指定了ingressclass所以这里的annotations 要指定否则访问会404
spec:entryPoints:- webroutes:- match: Host(traefik.test.com)kind: Ruleservices:- name: apiinternalkind: TraefikService
通过标签选择器进行过滤
首先在traefik配置文件中的providers下增加labelSelector参数指定具体的标签键值。
参考文档https://doc.traefik.io/traefik/providers/kubernetes-crd/#labelselector kind: ConfigMap
apiVersion: v1
metadata:name: traefik-config
data:traefik.yaml: |-global:checkNewVersion: false # 周期性的检查是否有新版本发布sendAnonymousUsage: false # 周期性的匿名发送使用统计信息serversTransport:insecureSkipVerify: true # Traefik忽略验证代理服务的TLS证书api:insecure: true # 允许HTTP 方式访问APIdashboard: true # 启用Dashboarddebug: false # 启用Debug调试模式metrics:prometheus: # 配置Prometheus监控指标数据并使用默认配置addRoutersLabels: true # 添加routers metricsentryPoint: metrics # 指定metrics监听地址entryPoints:web:address: :80 # 配置80端口并设置入口名称为webforwardedHeaders: insecure: true # 信任所有的forward headerswebsecure:address: :443 # 配置443端口并设置入口名称为 websecureforwardedHeaders: insecure: truetraefik:address: :9000 # 配置9000端口并设置入口名称为 dashboardmetrics:address: :9100 # 配置9100端口作为metrics收集入口tcpep:address: :9200 # 配置9200端口作为tcp入口udpep:address: :9300/udp # 配置9300端口作为udp入口providers:kubernetesCRD: # 启用Kubernetes CRD方式来配置路由规则# ingressClass: traefik-v2.8 # 指定traefik的ingressClass名称labelSelector: apptraefik-v2.8 # 通过标签选择器指定traefik标签 allowCrossNamespace: true #允许跨namespaceallowEmptyServices: true #允许空endpoints的servicelog:filePath: /etc/traefik/logs/traefik.log # 设置调试日志文件存储路径如果为空则输出到控制台level: INFO # 设置调试日志级别format: common # 设置调试日志格式accessLog:filePath: /etc/traefik/logs/access.log # 设置访问日志文件存储路径如果为空则输出到控制台format: common # 设置访问调试日志格式bufferingSize: 0 # 设置访问日志缓存行数filters:statusCodes: [200] # 设置只保留指定状态码范围内的访问日志retryAttempts: true # 设置代理访问重试失败时保留访问日志minDuration: 20 # 设置保留请求时间超过指定持续时间的访问日志fields: # 设置访问日志中的字段是否保留keep保留、drop不保留defaultMode: keep # 设置默认保留访问日志字段names: # 针对访问日志特别字段特别配置保留模式ClientUsername: dropStartUTC: drop # 禁用日志timestamp使用UTCheaders: # 设置Header中字段是否保留defaultMode: keep # 设置默认保留Header中字段names: # 针对Header中特别字段特别配置保留模式#User-Agent: redact # 可以针对指定agentAuthorization: dropContent-Type: keep
然后在 IngressRoute 资源对象中添加labels标签选择器选择app: traefik-v2.8这个标签即可
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboardlabels: # 通过标签选择器该IngressRoute资源由配置了apptraefik-v2.8的traefik处理app: traefik-v2.8# annotations:# kubernetes.io/ingress.class: traefik-v2.8 # 因为静态配置文件指定了ingressclass所以这里的annotations 要指定否则访问会404
spec:entryPoints:- webroutes:- match: Host(traefik.test.com)kind: Ruleservices:- name: apiinternalkind: TraefikService
Traefik CRD功能总结
traefik通过自定义资源实现了对traefik资源的创建和管理支持的crd资源类型如下所示
参考文档https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/
kind功能IngressRouteHTTP路由配置MiddlewareHTTP中间件配置TraefikServiceHTTP负载均衡/流量复制配置IngressRouteTCP路由配置MiddlewareTCPTCP中间件配置IngressRouteUDPUDP路由配置TLSOptionsTLS连接参数配置TLSStoresTLS存储配置ServersTransporttraefik与后端之间的传输配置
三、入门Traefik系列——路由配置与使用
环境准备
部署myapp1实例 apiVersion: apps/v1
kind: Deployment
metadata:name: myapp1
spec:selector:matchLabels:app: myapp1template:metadata:labels:app: myapp1spec:containers:- name: myapp1image: ikubernetes/myapp:v1resources:limits:memory: 128Micpu: 500mports:- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:name: myapp1
spec:type: ClusterIPselector:app: myapp1ports:- port: 80targetPort: 80
部署myapp2实例
apiVersion: apps/v1
kind: Deployment
metadata:name: myapp2
spec:selector:matchLabels:app: myapp2template:metadata:labels:app: myapp2spec:containers:- name: myapp2image: ikubernetes/myapp:v2resources:limits:memory: 128Micpu: 500mports:- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:name: myapp2
spec:type: ClusterIPselector:app: myapp2ports:- port: 80targetPort: 80
创建资源并访问测试
[rootk8s-master ingress]# kubectl get pod
NAME READY STATUS RESTARTS AGE
myapp1-795d947b45-9lsm6 1/1 Running 0 2m18s
myapp2-6ffd54f76-ljkr9 1/1 Running 0 66s
[rootk8s-master ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) 44h
myapp1 ClusterIP 10.104.91.200 none 80/TCP 2m26s
myapp2 ClusterIP 10.111.245.32 none 80/TCP 100s
[rootk8s-master ingress]# curl 10.104.91.200
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
[rootk8s-master ingress]# curl 10.111.245.32
Hello MyApp | Version: v2 | a hrefhostname.htmlPod Name/a
路由简介
路由功能
参考文档https://doc.traefik.io/traefik/routing/overview/ 当启动Traefik时需要定义entrypoints然后通过entrypoints的路由来分析传入的请求来查看他们是否是一组规则匹配如果匹配则路由可能将请求通过一系列的转换过来在发送到服务上去。
traefik支持的匹配规则
规则描述Headers(key, value)检查headers中是否有一个键为key值为value的键值对HeadersRegexp(key, regexp)检查headers中是否有一个键位key值为正则表达式匹配的键值对Host(example.com, ...)检查请求的域名是否包含在特定的域名中HostRegexp(example.com, {subdomain:[a-z]}.example.com, ...)检查请求的域名是否包含在特定的正则表达式域名中Method(GET, ...)检查请求方法是否为给定的methods(GET、POST、PUT、DELETE、PATCH)中Path(/path, /articles/{cat:[a-z]}/{id:[0-9]}, ...)匹配特定的请求路径它接受一系列文字和正则表达式路径PathPrefix(/products/, /articles/{cat:[a-z]}/{id:[0-9]})匹配特定的前缀路径它接受一系列文字和正则表达式前缀路径Query(foobar, barbaz)匹配查询字符串参数接受keyvalue的键值对ClientIP(10.0.0.0/16, ::1)如果请求客户端 IP 是给定的 IP/CIDR 之一则匹配。它接受 IPv4、IPv6 和网段格式。
路由配置IngressRoute
HTTP域名路由
实现目标集群外部用户通过访问http://myapp1.test.com域名时将请求代理至myapp1应用。创建ingressrouter规则文件 apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp1
spec:entryPoints:- webroutes:- match: Host(myapp1.test.com) # 域名kind: Ruleservices:- name: myapp1 # 与svc的name一致port: 80 # 与svc的port一致
创建资源
[rootk8s-master ingress]# kubectl apply -f myapp1-ingress.yaml
ingressroute.traefik.containo.us/myapp1 created
[rootk8s-master ingress]# kubectl get ingressroute
dashboard myapp1
[rootk8s-master ingress]# kubectl get ingressroute
NAME AGE
dashboard 4h26m
myapp1 20s
客户端添加hosts记录192.168.93.128 myapp1.test.com然后访问验证 HTTPS域名路由(自有证书)
公网服务的话可以在云厂商那里购买证书。内部服务的话就直接用 openssl 来创建一个自签名的证书即可要注意证书文件名称必须是 tls.crt 和 tls.key。接下来演示自签证书的配置。创建自签证书
[rootk8s-master ingress]# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj /CNmyapp2.test.com
建Secret资源来引用证书文件 [rootk8s-master ingress]# kubectl create secret tls myapp2-tls --certtls.crt --keytls.key
secret/myapp2-tls created
[rootk8s-master ingress]# kubectl describe secrets myapp2-tls
Name: myapp2-tls
Namespace: default
Labels: none
Annotations: noneType: kubernetes.io/tlsDatatls.crt: 1131 bytes
tls.key: 1704 bytes 创建IngressRouter规则文件集群外部用户通过访问https://myapp2.test.com域名时将请求代理至myapp2应用。
[rootk8s-master ingress]# cat myapp2-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2
spec:entryPoints:- websecure # 监听 websecure 这个入口点也就是通过 443 端口来访问routes:- match: Host(myapp2.test.com)kind: Ruleservices:- name: myapp2port: 80tls:secretName: myapp2-tls # 指定tls证书名称
创建资源
[rootk8s-master ingress]# kubectl apply -f myapp2-ingress.yaml
ingressroute.traefik.containo.us/myapp2 created
[rootk8s-master ingress]# kubectl get ingressroute
dashboard myapp2
[rootk8s-master ingress]# kubectl get ingressroute
NAME AGE
dashboard 5h11m
myapp1 45m
myapp2 2m55s
客户端添加hosts记录192.168.93.128 myapp2.test.com然后访问验证由于我们是自签名的证书所以证书是不受信任的。 HTTPS域名路由(自动生成证书) 参考文档https://doc.traefik.io/traefik-enterprise/tls/acme/ Traefik除了使用自有证书外还支持在创建ingress资源时自动请求Lets Encrypt生成证书并且在证书过期前30天自动续订证书。要使用Lets Encrypt自动生成证书需要使用ACME。需要在静态配置中定义 证书解析器Traefik负责从ACME服务器中检索证书。然后每个 路由器 被配置为启用TLS并通过tls.certresolver配置选项与一个证书解析器关联。如果使用tlsChallenge则要求Lets Encrypt到 Traefik 443 端口必须是可达的。如果使用httpChallenge则要求Lets Encrypt到 Traefik 80端口必须是可达的。如果使用dnsChallenge只需要配置上 DNS 解析服务商的 API 访问密钥即可校验。
tlsChallenge/httpChallenge
使用tlsChallenge或者httpChallenge的前提条件是traefik所在节点可以正常访问Lets Encrypt网站国内网络可能访问存在异常并且配置的ingress域名已经设置了dns的A记录解析指向traefik所在的节点公网IP地址才可以否则申请成功证书后无法通过验证。修改traefik配置文件新增certificatesResolvers配置
# traefik-config.yaml
kind: ConfigMap
apiVersion: v1
metadata:name: traefik-config
data:traefik.yaml: |-global:checkNewVersion: false ## 周期性的检查是否有新版本发布sendAnonymousUsage: false ## 周期性的匿名发送使用统计信息serversTransport:insecureSkipVerify: true ## Traefik忽略验证代理服务的TLS证书api:insecure: true ## 允许HTTP 方式访问APIdashboard: true ## 启用Dashboarddebug: false ## 启用Debug调试模式metrics:prometheus: ## 配置Prometheus监控指标数据并使用默认配置addRoutersLabels: true ## 添加routers metricsentryPoint: metrics ## 指定metrics监听地址entryPoints:web:address: :80 ## 配置80端口并设置入口名称为webforwardedHeaders: insecure: true ## 信任所有的forward headerswebsecure:address: :443 ## 配置443端口并设置入口名称为 websecureforwardedHeaders: insecure: truetraefik:address: :9000 ## 配置9000端口并设置入口名称为 dashboardmetrics:address: :9100 ## 配置9100端口作为metrics收集入口tcpep:address: :9200 ## 配置9200端口作为tcp入口udpep:address: :9300/udp ## 配置9300端口作为udp入口certificatesResolvers: ## 开启ACME自动续签证书sample:acme:email: 1554382111qq.com # 邮箱配置storage: /etc/traefik/acme/acme.json # 保存 ACME 证书的位置# tlsChallenge: {} # tlsChallenge模式续签httpChallenge:entryPoint: web # httpChallenge模式续签providers:kubernetesCRD: ## 启用Kubernetes CRD方式来配置路由规则ingressClass: allowCrossNamespace: true ##允许跨namespaceallowEmptyServices: true ##允许空endpoints的servicekubernetesIngress: ## 启动Kubernetes Ingress方式来配置路由规则ingressClass: log:filePath: /etc/traefik/logs/traefik.log ## 设置调试日志文件存储路径如果为空则输出到控制台level: DEBUG ## 设置调试日志级别format: common ## 设置调试日志格式accessLog:filePath: /etc/traefik/logs/access.log ## 设置访问日志文件存储路径如果为空则输出到控制台format: common ## 设置访问调试日志格式bufferingSize: 0 ## 设置访问日志缓存行数filters:#statusCodes: [200] ## 设置只保留指定状态码范围内的访问日志retryAttempts: true ## 设置代理访问重试失败时保留访问日志minDuration: 20 ## 设置保留请求时间超过指定持续时间的访问日志fields: ## 设置访问日志中的字段是否保留keep保留、drop不保留defaultMode: keep ## 设置默认保留访问日志字段names: ## 针对访问日志特别字段特别配置保留模式ClientUsername: dropStartUTC: drop ## 禁用日志timestamp使用UTCheaders: ## 设置Header中字段是否保留defaultMode: keep ## 设置默认保留Header中字段names: ## 针对Header中特别字段特别配置保留模式#User-Agent: redact ## 可以针对指定agentAuthorization: dropContent-Type: keep
修改traefik的deployment资源文件挂载acme目录存储ACME证书信息
# traefik-deployment.yaml
apiVersion: v1
kind: ServiceAccount
metadata:namespace: defaultname: traefik-ingress-controller
---
apiVersion: apps/v1
kind: Deployment
metadata:name: traefik-ingress-controllernamespace: defaultlabels:app: traefik
spec:replicas: 1 #副本数为1因为集群只设置一台master为边缘节点selector:matchLabels:app: traefiktemplate:metadata:name: traefiklabels:app: traefikspec:serviceAccountName: traefik-ingress-controllerterminationGracePeriodSeconds: 1containers:- name: traefikimage: traefik:v2.8.7env:- name: KUBERNETES_SERVICE_HOST ##手动指定k8s api,避免网络组件不稳定。value: 192.168.10.10- name: KUBERNETES_SERVICE_PORT_HTTPS ## API server端口value: 6443- name: KUBERNETES_SERVICE_PORT ## API server端口value: 6443- name: TZ ##指定时区value: Asia/Shanghaiports:- name: webcontainerPort: 80hostPort: 80 ## 将容器端口绑定所在服务器的 80 端口- name: websecurecontainerPort: 443hostPort: 443 ## 将容器端口绑定所在服务器的 443 端口- name: admincontainerPort: 9000 ## Traefik Dashboard 端口- name: metricscontainerPort: 9100 ## metrics端口- name: tcpepcontainerPort: 9200 ## tcp端口- name: udpepcontainerPort: 9300 ## udp端口securityContext: ## 只开放网络权限 capabilities:drop:- ALLadd:- NET_BIND_SERVICEargs:- --configfile/etc/traefik/config/traefik.yamlvolumeMounts:- mountPath: /etc/traefik/configname: config- mountPath: /etc/traefik/logsname: logdir- mountPath: /etc/localtimename: timezonereadOnly: true- mountPath: /etc/traefik/acmename: acmevolumes:- name: configconfigMap:name: traefik-config - name: logdirhostPath:path: /data/traefik/logstype: DirectoryOrCreate- name: timezone #挂载时区文件hostPath:path: /etc/localtimetype: File- name: acme # 自动续签证书文件hostPath:path: /data/traefik/acmetype: DirectoryOrCreatetolerations: ## 设置容忍所有污点防止节点被设置污点- operator: ExistshostNetwork: true ## 开启host网络提高网络入口的网络性能nodeSelector: ## 设置node筛选器在特定label的节点上启动IngressProxy: true ## 调度至IngressProxy: true的节点
配置IngressRouter规则
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2
spec:entryPoints:- websecure # 监听 websecure 这个入口点也就是通过 443 端口来访问routes:- match: Host(myapp2.cuiliangblog.cn) # 必须是真实存在的域名且配置了dns解析记录指向traefik节点所在的公网IPkind: Ruleservices:- name: myapp2port: 80tls:certResolver: sample # 使用自动生成证书名字与traefik的certificatesResolvers名称一致
dnsChallenge
dns 校验方式可以生成通配符的证书只需要配置上 DNS 解析服务商的 API 访问密钥即可校验。每个厂商的配置都略有差异此处以阿里云为例其他厂商的配置请查看文档https://go-acme.github.io/lego/dns/[3]修改traefik配置文件新增dnsChallenge配置
kind: ConfigMap
apiVersion: v1
metadata:name: traefik-config
data:traefik.yaml: |-global:checkNewVersion: false ## 周期性的检查是否有新版本发布sendAnonymousUsage: false ## 周期性的匿名发送使用统计信息serversTransport:insecureSkipVerify: true ## Traefik忽略验证代理服务的TLS证书api:insecure: true ## 允许HTTP 方式访问APIdashboard: true ## 启用Dashboarddebug: false ## 启用Debug调试模式metrics:prometheus: ## 配置Prometheus监控指标数据并使用默认配置addRoutersLabels: true ## 添加routers metricsentryPoint: metrics ## 指定metrics监听地址entryPoints:web:address: :80 ## 配置80端口并设置入口名称为webforwardedHeaders: insecure: true ## 信任所有的forward headerswebsecure:address: :443 ## 配置443端口并设置入口名称为 websecureforwardedHeaders: insecure: truetraefik:address: :9000 ## 配置9000端口并设置入口名称为 dashboardmetrics:address: :9100 ## 配置9100端口作为metrics收集入口tcpep:address: :9200 ## 配置9200端口作为tcp入口udpep:address: :9300/udp ## 配置9300端口作为udp入口certificatesResolvers: ## 开启ACME自动续签证书sample:acme:email: 1554382111qq.com # 邮箱配置storage: /etc/traefik/acme/acme.json # 保存 ACME 证书的位置# tlsChallenge: {} # tls模式续签# httpChallenge:# entryPoint: web # http模式续签dnsChallenge: # dns模式续签证书provider: alidns # 云厂商编号 delayBeforeCheck: 0 # ACME 验证之前会验证 TXT 记录。设定延迟验证时间(以秒为单位)providers:kubernetesCRD: ## 启用Kubernetes CRD方式来配置路由规则ingressClass: allowCrossNamespace: true ##允许跨namespaceallowEmptyServices: true ##允许空endpoints的servicekubernetesIngress: ## 启动Kubernetes Ingress方式来配置路由规则ingressClass: log:filePath: /etc/traefik/logs/traefik.log ## 设置调试日志文件存储路径如果为空则输出到控制台level: DEBUG ## 设置调试日志级别format: common ## 设置调试日志格式accessLog:filePath: /etc/traefik/logs/access.log ## 设置访问日志文件存储路径如果为空则输出到控制台format: common ## 设置访问调试日志格式bufferingSize: 0 ## 设置访问日志缓存行数filters:#statusCodes: [200] ## 设置只保留指定状态码范围内的访问日志retryAttempts: true ## 设置代理访问重试失败时保留访问日志minDuration: 20 ## 设置保留请求时间超过指定持续时间的访问日志fields: ## 设置访问日志中的字段是否保留keep保留、drop不保留defaultMode: keep ## 设置默认保留访问日志字段names: ## 针对访问日志特别字段特别配置保留模式ClientUsername: dropStartUTC: drop ## 禁用日志timestamp使用UTCheaders: ## 设置Header中字段是否保留defaultMode: keep ## 设置默认保留Header中字段names: ## 针对Header中特别字段特别配置保留模式#User-Agent: redact ## 可以针对指定agentAuthorization: dropContent-Type: keep
登录阿里云后台获取ALICLOUD_ACCESS_KEY、ALICLOUD_SECRET_KEY、ALICLOUD_REGION_ID信息 创建Secret 对象存放密钥信息记得填写base64编码后的值
apiVersion: v1
kind: Secret
metadata:name: alidns-secret
type: Opaque
data:ALICLOUD_ACCESS_KEY: XXXALICLOUD_SECRET_KEY: XXXALICLOUD_REGION_ID: XXX
修改traefik的deployment资源清单添加密钥env变量
apiVersion: v1
kind: ServiceAccount
metadata:namespace: defaultname: traefik-ingress-controller
---
apiVersion: apps/v1
kind: Deployment
metadata:name: traefik-ingress-controllernamespace: defaultlabels:app: traefik
spec:replicas: 1 # 副本数为1因为集群只设置一台master为边缘节点selector:matchLabels:app: traefiktemplate:metadata:name: traefiklabels:app: traefikspec:serviceAccountName: traefik-ingress-controllerterminationGracePeriodSeconds: 1containers:- name: traefikimage: traefik:v2.8.7env:- name: KUBERNETES_SERVICE_HOST # 手动指定k8s api,避免网络组件不稳定。value: 192.168.10.10- name: KUBERNETES_SERVICE_PORT_HTTPS # API server端口value: 6443- name: KUBERNETES_SERVICE_PORT # API server端口value: 6443- name: TZ # 指定时区value: Asia/Shanghai- name: ALICLOUD_ACCESS_KEY # 阿里云AKvalueFrom:secretKeyRef:name: alidns-secretkey: ALICLOUD_ACCESS_KEY- name: ALICLOUD_SECRET_KEY # 阿里云SKvalueFrom:secretKeyRef:name: alidns-secretkey: ALICLOUD_SECRET_KEY- name: ALICLOUD_REGION_ID # 阿里云资源区域编号valueFrom:secretKeyRef:name: alidns-secretkey: ALICLOUD_REGION_IDports:- name: webcontainerPort: 80hostPort: 80 # 将容器端口绑定所在服务器的 80 端口- name: websecurecontainerPort: 443hostPort: 443 # 将容器端口绑定所在服务器的 443 端口- name: admincontainerPort: 9000 # Traefik Dashboard 端口- name: metricscontainerPort: 9100 # metrics端口- name: tcpepcontainerPort: 9200 # tcp端口- name: udpepcontainerPort: 9300 # udp端口securityContext: # 只开放网络权限 capabilities:drop:- ALLadd:- NET_BIND_SERVICEargs:- --configfile/etc/traefik/config/traefik.yamlvolumeMounts:- mountPath: /etc/traefik/configname: config- mountPath: /etc/traefik/logsname: logdir- mountPath: /etc/localtimename: timezonereadOnly: true- mountPath: /etc/traefik/acmename: acmevolumes:- name: configconfigMap:name: traefik-config - name: logdirhostPath:path: /data/traefik/logstype: DirectoryOrCreate- name: timezone #挂载时区文件hostPath:path: /etc/localtimetype: File- name: acme # 自动续签证书文件hostPath:path: /data/traefik/acmetype: DirectoryOrCreatetolerations: # 设置容忍所有污点防止节点被设置污点- operator: ExistshostNetwork: true # 开启host网络提高网络入口的网络性能nodeSelector: # 设置node筛选器在特定label的节点上启动IngressProxy: true # 调度至IngressProxy: true的节点
修改ingressrouter配置
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2
spec:entryPoints:- websecure # 监听 websecure 这个入口点也就是通过 443 端口来访问routes:- match: Host(myapp2.cuiliangblog.cn)kind: Ruleservices:- name: myapp2port: 80tls:certResolver: sample # 使用自动生成证书名字与traefik的certificatesResolvers名称一致domains:- main: *.cuiliangblog.cn # 不指定的话默认申请Host域名可以指定申请通配符域名
然后在阿里云DNS上做解析重新创建ingress资源时即可触发申请证书。常见错误处理方案
日志关键词原因解决方案net/http: timeout awaiting response headers或者connect: connection refusedtraefik所在节点无法访问Lets Encrypt申请证书使用工具加速acme: error :400/403申请的域名DNS解析记录为配置或者配置地址不正确指向了其他IP更改DNS解析配置指向traefik节点所在的公网IPacme: error : 429失败次数过多每个小时只允许请求5次换账号/域名/IP重试或者等一个小时后再试
路由配置IngressRouteTCP
TCP路由(不带TLS证书)
首先部署一个简单的redis服务资源清单文件如下所示 # redis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: redis
spec:selector:matchLabels:app: redistemplate:metadata:labels:app: redisspec:containers:- name: redisimage: redis:latestresources:limits:memory: 128Micpu: 500mports:- containerPort: 6379protocol: TCP
---
apiVersion: v1
kind: Service
metadata:name: redis
spec:selector:app: redisports:- port: 6379targetPort: 6379
创建redis应用
[rootk8s-master ingress]# kubectl apply -f redis.yaml
deployment.apps/redis created
service/redis created
创建IngressRouter进行对外暴露
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:name: redis
spec:entryPoints:- tcpep # 指定入口点为tcp端口routes:- match: HostSNI(*) # 由于Traefik中使用TCP路由配置需要SNI而SNI又是依赖TLS的所以我们需要配置证书才行如果没有证书的话我们可以使用通配符*(适配ip的)进行配置services:- name: redisport: 6379
查看traefik的dashboard页面是否生效
[roottiaoban ~]# redis-cli -h redis.test.com -p 9200
redis.test.com:9200 set key_a value_a
OK
redis.test.com:9200 get key_a
value_a
redis.test.com:9200 集群外部客户端配置hosts解析192.168.93.128 redis.test.com域名可以随意填写只要能解析到traefik所在节点即可然后通过redis-cli工具访问redis记得指定tcpep的端口。
如果需要再添加其他tcp路由需要修改traefik配置新增entryPoints端口。
TCP路由(带TLS证书)
有时候为了安全要求tcp传输也需要使用TLS证书加密redis从6.0开始支持了tls证书通信。
[rootk8s-master ingress]# mkdir redis-ssl
[rootk8s-master ingress]# cd redis-ssl/
[rootk8s-master redis-ssl]# openssl genrsa -out ca.key 4096
[rootk8s-master redis-ssl]# openssl req -x509 -new -nodes -sha256 -key ca.key -days 3650 -subj /ORedis Test/CNCertificate Authority -out ca.crt
[rootk8s-master redis-ssl]# openssl genrsa -out redis.key 2048
[rootk8s-master redis-ssl]# openssl req -new -sha256 -key redis.key -subj /ORedis Test/CNServer | openssl x509 -req -sha256 -CA ca.crt -CAkey ca.key -CAserial ca.txt -CAcreateserial -days 365 -out redis.crt
ootk8s-master redis-ssl]# openssl dhparam -out redis.dh 2048
[rootk8s-master redis-ssl]# ll
总用量 24
-rw-r--r-- 1 root root 1895 9月 25 08:34 ca.crt
-rw------- 1 root root 3243 9月 25 08:34 ca.key
-rw-r--r-- 1 root root 41 9月 25 08:35 ca.txt
-rw-r--r-- 1 root root 1407 9月 25 08:35 redis.crt
-rw-r--r-- 1 root root 424 9月 25 08:35 redis.dh
-rw------- 1 root root 1679 9月 25 08:34 redis.key
创建secret资源使用tls类型包含redis.crt和redis.key
[rootk8s-master redis-ssl]# kubectl create secret tls redis-tls --keyredis.key --certredis.crt
secret/redis-tls created
[rootk8s-master redis-ssl]# kubectl describe secrets redis-tls
Name: redis-tls
Namespace: default
Labels: none
Annotations: noneType: kubernetes.io/tlsDatatls.crt: 1407 bytes
tls.key: 1679 bytes
创建secret资源使用generic类型包含ca.crt
[rootk8s-master redis-ssl]# kubectl create secret generic redis-ca --from-fileca.crtca.crt
secret/redis-ca created
[rootk8s-master redis-ssl]# kubectl describe secrets redis-ca
Name: redis-ca
Namespace: default
Labels: none
Annotations: noneType: OpaqueDataca.crt: 1895 bytes
修改redis配置启用tls证书并挂载证书文件
apiVersion: v1
kind: ConfigMap
metadata:name: redislabels:app: redis
data:redis.conf : |-port 0tls-port 6379tls-cert-file /etc/tls/tls.crttls-key-file /etc/tls/tls.keytls-ca-cert-file /etc/ca/ca.crt
---
apiVersion: apps/v1
kind: Deployment
metadata:name: redis
spec:selector:matchLabels:app: redistemplate:metadata:labels:app: redisspec:containers:- name: redisimage: redis:latestresources:limits:memory: 128Micpu: 500mports:- containerPort: 6379protocol: TCPvolumeMounts:- name: configmountPath: /etc/redis- name: tlsmountPath: /etc/tls- name: camountPath: /etc/caargs:- /etc/redis/redis.confvolumes:- name: configconfigMap:name: redis- name: tlssecret:secretName: redis-tls- name: casecret:secretName: redis-ca
---
apiVersion: v1
kind: Service
metadata:name: redis
spec:selector:app: redisports:- port: 6379targetPort: 6379
创建资源并查看状态
[rootk8s-master ingress]# kubectl apply -f redis.yaml
configmap/redis created
deployment.apps/redis created
service/redis created
[rootk8s-master ingress]# kubectl get pod
NAME READY STATUS RESTARTS AGE
redis-69974c8b56-rzxb6 1/1 Running 0 7s
创建IngressRouter资源指定域名和证书
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:name: redis
spec:entryPoints:- tcpeproutes:- match: HostSNI(redis.test.com)services:- name: redisport: 6379tls:secretName: redis-tls
traefik管理页查看
yum install openssl openssl-devel -y
wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make redis-cli BUILD_TLSyes MALLOClibc
cp src/redis-cli /usr/local/bin/ yum安装的redis-cli版本为5.0.3不支持tls需要编译安装6.0以上版本并在编译时开启TLS
客户端添加hosts记录192.168.93.128 redis.test.com直接访问redis直接报错 [roottiaoban src]# ./src/redis-cli -h redis.test.com -p 9200127.0.0.1:6379 set key 1
Error: Connection reset by peer
客户端使用证书访问redis测试成功
[roottiaoban src]# ./redis-cli -h redis.test.com -p 9200 --tls --cert /tmp/redis-ssl/redis.crt --key /tmp/redis-ssl/redis.key --cacert /tmp/redis-ssl/ca.crt
redis.test.com:9200 set key 1
OK
路由配置IngressRouteUDP
UDP路由
traefik同样也提供了UDP的支持以我们最常用的rsyslog服务为例演示traefik如果配置使用 首先制作一个rsyslog镜像 [rootk8s-master udp]# ls
Dockerfile rsyslog.conf
# rsyslog配置
[rootk8s-master udp]# cat rsyslog.conf
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imudp
$UDPServerRun 514
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$IncludeConfig /etc/rsyslog.d/*.conf
$OmitLocalLogging off
*.info;mail.none;authpriv.none;cron.none /var/log/messages
authpriv.* /var/log/secure
mail.* -/var/log/maillog
cron.* /var/log/cron
*.emerg :omusrmsg:*
uucp,news.crit /var/log/spooler
local7.* /var/log/boot.log
# dockerfile配置
[rootk8s-master udp]# cat Dockerfile
FROM centos:7
RUN yum -y install rsyslog rm -rf /etc/rsyslog.d/listen.conf
COPY rsyslog.conf /etc/rsyslog.conf
EXPOSE 514/udp
CMD [/usr/sbin/rsyslogd, -dn]
# 构建镜像
[rootk8s-master udp]# docker build -t rsyslog:v1 .
接下来创建rsyslog的资源清单
[rootk8s-master udp]# cat rsyslog-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: rsyslog
spec:selector:matchLabels:app: rsyslogtemplate:metadata:labels:app: rsyslogspec:containers:- name: rsyslogimage: rsyslog:v1resources:limits:memory: 128Micpu: 500mports:- containerPort: 514protocol: UDP
---
apiVersion: v1
kind: Service
metadata:name: rsyslog
spec:selector:app: rsyslogports:- port: 514protocol: UDPtargetPort: 514
部署上面的应用并查看[rootk8s-master udp]# kubectl apply -f rsyslog-deployment.yaml
deployment.apps/rsyslog created
service/rsyslog created
[rootk8s-master udp]# kubectl get pod
NAME READY STATUS RESTARTS AGE
rsyslog-5dfc9d64b5-d9wjj 1/1 Running 0 3s
创建IngressRouter资源代理UDP应用需要注意的是UDP资源访问时直接通过公网ipdup的entryPoints端口即可不需要配置域名
[rootk8s-master udp]# cat rsyslog-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteUDP
metadata:name: rsyslog
spec:entryPoints:- udpeproutes:- services:- name: rsyslogport: 514
[rootk8s-master udp]# kubectl apply -f rsyslog-ingress.yaml
ingressrouteudp.traefik.containo.us/rsyslog created
查看dashboard的udp信息 集群外部访问udp服务通过 Traefik 所在节点的公网 IP192.168.93.128与 entryPoints端口9300来访问 UDP 应用进行测试
[roottiaoban ~]# logger -n 192.168.93.128 -P 9300 cuiliang123
[roottiaoban ~]# logger -n 192.168.93.128 -P 9300 hello 123
查看rsyslog日志验证请求是否成功
[rootk8s-master ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
rsyslog-5dfc9d64b5-d9wjj 1/1 Running 0 6m48s
[rootk8s-master ~]# kubectl exec -it rsyslog-5dfc9d64b5-d9wjj -- bash
[rootrsyslog-5dfc9d64b5-d9wjj /]# tail -n 5 /var/log/messages
Sep 25 04:06:20 rsyslog-5dfc9d64b5-d9wjj rsyslogd: [origin softwarersyslogd swVersion8.24.0-57.el7_9.3 x-pid1 x-infohttp://www.rsyslog.com] start
Sep 25 12:12:55 tiaoban root cuiliang123
Sep 25 12:17:19 tiaoban root hello 123
负载均衡配置
traefik可以对http、TCP、UDP实现负载均衡根据需求创建IngressRoute/IngressRouteTCP/IngressRouteUDP即可此处以http为例。
http路由多个k8s service配置
创建两个deployment应用与对应的svc
[rootk8s-master udp]# kubectl get pod
NAME READY STATUS RESTARTS AGE
myapp1-795d947b45-9lsm6 1/1 Running 1 25h
myapp2-6ffd54f76-ljkr9 1/1 Running 1 25h
[rootk8s-master udp]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) 2d22h
myapp1 ClusterIP 10.104.91.200 none 80/TCP 25h
myapp2 ClusterIP 10.111.245.32 none 80/TCP
创建IngressRouter资源配置域名为myapp.test.com请求流量均摊到两个k8s的service上。
[rootk8s-master ingress]# cat myapp-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myappnamespace: default
spec:entryPoints:- webroutes:- match: Host(myapp.test.com)kind: Ruleservices:- name: myapp1namespace: defaultport: 80 - name: myapp2namespace: defaultport: 80
[rootk8s-master ingress]# kubectl apply -f myapp-ingress.yaml
ingressroute.traefik.containo.us/myapp created
查看dashboard页面路由信息发现已成功配置代理两个service服务且权重均为1 解析来访问测试发现依次循环响应myapp1和myapp2的内容
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | a hrefhostname.htmlPod Name/a
三、入门Traefik系列——中间件
简介
Traefik Middlewares 是一个处于路由和后端服务之前的中间件在外部流量进入 Traefik且路由规则匹配成功后将流量发送到对应的后端服务前先将其发给中间件进行一系列处理类似于过滤器链 Filter进行一系列处理例如添加 Header 头信息、鉴权、流量转发、处理访问路径前缀、IP 白名单等等经过一个或者多个中间件处理完成后再发送给后端服务这个就是中间件的作用。Traefik内置了很多不同功能的Middleware主要是针对HTTP和TCP这里挑选几个比较常用的进行演示。
参考文档https://doc.traefik.io/traefik/middlewares/overview/
重定向
redirectScheme的更多用法参考文档https://doc.traefik.io/traefik/middlewares/http/redirectscheme/
还是以前面的deployment应用与对应的svc为例
[rootk8s-master udp]# kubectl get pod
NAME READY STATUS RESTARTS AGE
myapp2-795d947b45-9lsm6 1/1 Running 1 25h
[rootk8s-master udp]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) 2d22h
myapp2 ClusterIP 10.104.91.200 none 80/TCP
[rootk8s-master ingress]# kubectl describe secrets myapp2-tls
Name: myapp2-tls
Namespace: default
Labels: none
Annotations: noneType: kubernetes.io/tlsDatatls.crt: 1131 bytes
tls.key: 1704 bytes
创建一个https的IngressRoute
[rootk8s-master middleware]# cat https-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2-tls
spec:entryPoints:- websecureroutes:- match: Host(myapp2.test.com)kind: Ruleservices:- name: myapp2port: 80 tls:secretName: myapp2-tls # 指定tls证书名称
[rootk8s-master middleware]# kubectl apply -f https-ingress.yaml
ingressroute.traefik.containo.us/myapp2-tls created
定义一个强制将http请求跳转到https的中间件。
[rootk8s-master middleware]# cat https-middleware.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: redirect-https-middleware
spec:redirectScheme:scheme: https
[rootk8s-master middleware]# kubectl apply -f https-middleware.yaml
middleware.traefik.containo.us/redirect-https-middleware created
定义一个http的IngressRoute并使用中间件
[rootk8s-master middleware]# cat http-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2
spec:entryPoints:- webroutes:- match: Host(myapp2.test.com)kind: Ruleservices:- name: myapp2port: 80middlewares:- name: redirect-https-middleware # 指定使用RedirectScheme中间件完成http强制跳转至https
[rootk8s-master middleware]# kubectl apply -f http-ingress.yaml
ingressroute.traefik.containo.us/myapp1 created
访问测试当用户访问http://myapp.test.com时会强制跳转到https://myapp.test.com 去除请求路径前缀
假设现在有这样一个需求当访问http://myapp.test.com/v1时流量调度至myapp1。当访问http://myapp.test.com/v2时流量调度至myapp2。这种需求是非常常见的在NGINX中我们可以配置多个Location来定制规则使用Traefik也可以这么做。但是定制不同的前缀后由于应用本身并没有这些前缀导致请求返回404这时候我们就需要对请求的path进行处理。
参考文档https://doc.traefik.io/traefik/middlewares/http/stripprefix/
创建一个IngressRoute并设置两条规则根据不同的访问路径代理至相对应的service
[rootk8s-master middleware]# cat myapp-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp
spec:entryPoints:- webroutes:- match: Host(myapp.test.com) PathPrefix(/v1)kind: Ruleservices:- name: myapp1port: 80- match: Host(myapp.test.com) PathPrefix(/v2)kind: Ruleservices:- name: myapp2port: 80
[rootk8s-master middleware]# kubectl apply -f myapp-ingress.yaml
ingressroute.traefik.containo.us/myapp created
进行访问测试http://myapp.test.com/v1虽然traefik配置无误但是由于myapp1应用并没有v1这个路径因此返回404页面 接下来定义去除前缀的中间件stripPrefix指定将请求路径中的v1、v2去除。
[rootk8s-master middleware]# cat prefix-url-middleware.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: prefix-url-middleware
spec:stripPrefix:prefixes:- /v1- /v2
[rootk8s-master middleware]# kubectl apply -f prefix-url-middleware.yaml
middleware.traefik.containo.us/prefix-url-middleware created
修改上面的ingressRoute添加刚刚定义的prefix-url-middleware中间件
[rootk8s-master middleware]# cat myapp-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp
spec:entryPoints:- webroutes:- match: Host(myapp.test.com) PathPrefix(/v1)kind: Ruleservices:- name: myapp1port: 80 middlewares:- name: prefix-url-middleware- match: Host(myapp.test.com) PathPrefix(/v2)kind: Ruleservices:- name: myapp2port: 80 middlewares:- name: prefix-url-middleware
[rootk8s-master middleware]# kubectl apply -f myapp-ingress.yaml
ingressroute.traefik.containo.us/myapp configured
查看traefik的dashboard已添加了中间件 接下来进行访问测试 添加IP白名单
为提高安全性通常情况下一些管理员界面会设置ip访问白名单只希望个别用户可以访问例如访问traefik的dashboard的url这时候就可以使用Traefik中的ipWhiteList中间件来完成。
参考文档https://doc.traefik.io/traefik/middlewares/http/ipwhitelist/
当前traefik的dashboard任何主机都可以访问
[roottiaoban ~]# curl http://traefik.test.com/dashboard/
!DOCTYPE htmlhtmlheadtitleTraefik/titlemeta charsetutf-8meta namedescription contentTraefik UI……接下来定义IP访问白名单的中间件ipWhiteList指定可以访问的ip列表。
[rootk8s-master middleware]# cat ip-white-middleware.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: ip-white-list-middleware
spec:ipWhiteList:sourceRange:- 127.0.0.1/32- 192.168.93.1
[rootk8s-master middleware]# kubectl apply -f ip-white-middleware.yaml
middleware.traefik.containo.us/ip-white-list-middleware created
修改dashboard的ingressRoute添加ip白名单中间件
[rootk8s-master middleware]# cat dashboard-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboard
spec:entryPoints:- webroutes:- match: Host(traefik.test.com)kind: Ruleservices:- name: apiinternalkind: TraefikServicemiddlewares:- name: ip-white-list-middleware
[rootk8s-master middleware]# kubectl apply -f dashboard-ingress.yaml
ingressroute.traefik.containo.us/dashboard configured
接下来使用白名单之外的ip访问测试
[roottiaoban ~]# curl traefik.test.com/dashboard/
Forbidden[roottiaoban ~]#
[roottiaoban ~]# curl -I http://traefik.test.com/dashboard/
HTTP/1.1 403 Forbidden
Date: Sun, 25 Sep 2022 13:32:49 GMT
Content-Length: 9
Content-Type: text/plain; charsetutf-8
基础用户认证
通常企业安全要求规范除了要对管理员页面限制访问ip外还需要添加账号密码认证而traefik默认没有提供账号密码认证功能此时就可以通过BasicAuth中间件完成用户认证只有认证通过的授权用户才可以访问页面。
参考文档https://doc.traefik.io/traefik/middlewares/http/basicauth/ 使用basicAuth认证需要使用htpasswd工具生成密码文件因此先安装httpd软件包
[rootk8s-master middleware]# dnf install -y httpd
使用htpasswd工具设置用户名密码生成密钥文件
[rootk8s-master middleware]# htpasswd -bc basic-auth-secret cuiliang 123
Adding password for user cuiliang 将生成的basic-auth-secret密码文件创建成secret资源
[rootk8s-master middleware]# kubectl create secret generic basic-auth --from-filebasic-auth-secret
secret/basic-auth created
[rootk8s-master middleware]# kubectl get secrets
NAME TYPE DATA AGE
basic-auth Opaque 1 8s
接下来创建basicAuth中间件指定使用刚刚创建的secret资源。
[rootk8s-master middleware]# cat basic-auth-middleware.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: basic-auth-middleware
spec:basicAuth:secret: basic-auth
[rootk8s-master middleware]# kubectl apply -f basic-auth-middleware.yaml
middleware.traefik.containo.us/basic-auth-middleware created
修改dashboard的ingressRoute添加basicAuth中间件
[rootk8s-master middleware]# cat basic-auth-middleware.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: basic-auth-middleware
spec:basicAuth:secret: basic-auth
[rootk8s-master middleware]# cat dashboard-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: dashboard
spec:entryPoints:- webroutes:- match: Host(traefik.test.com)kind: Ruleservices:- name: apiinternalkind: TraefikServicemiddlewares:- name: basic-auth-middleware
[rootk8s-master middleware]# kubectl apply -f basic-auth-middleware.yaml
middleware.traefik.containo.us/basic-auth-middleware created
客户端访问验证刷新页面后弹出用户登录认证页面。 修改请求/响应头信息
为了提高业务的安全性安全团队会定期进行漏洞扫描其中有些web漏洞就需要通过修改响应头处理traefik的Headers中间件不仅可以修改返回客户端的响应头信息还能修改反向代理后端service服务的请求头信息。 例如对https://myapp2.test.com提高安全策略强制启用HSTS HSTS即HTTP严格传输安全响应头收到该响应头的浏览器会在 63072000s约 2 年的时间内只要访问该网站即使输入的是 http浏览器会自动跳转到 https。HSTS 是浏览器端的跳转之前的HTTP 重定向到 HTTPS是服务器端的跳转
参考文档https://doc.traefik.io/traefik/middlewares/http/headers/
定义响应头中间件Headers指定响应内容中添加Strict-Transport-Security配置。
[rootk8s-master middleware]# cat hsts-header-middleware.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: hsts-header-middleware
spec:headers:customResponseHeaders:Strict-Transport-Security: max-age63072000
[rootk8s-master middleware]# kubectl apply -f hsts-header-middleware.yaml
middleware.traefik.containo.us/hsts-header-middleware created
修改myapp2的ingressRoute添加headers中间件
[rootk8s-master middleware]# cat myapp2-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp2-tls
spec:entryPoints:- web- websecureroutes:- match: Host(myapp2.test.com)kind: Ruleservices:- name: myapp2port: 80 middlewares:- name: hsts-header-middlewaretls:secretName: myapp2-tls # 指定tls证书名称
[rootk8s-master middleware]# kubectl apply -f myapp2-ingress.yaml
ingressroute.traefik.containo.us/myapp2-tls configured
客户端访问验证查看响应头信息 限流
在实际生产环境中流量限制也是经常用到的它可以用作安全目的比如可以减慢暴力密码破解的速率。通过将传入请求的速率限制为真实用户的典型值并标识目标URL地址(通过日志)还可以用来抵御 DDOS 攻击。更常见的情况该功能被用来保护下游应用服务器不被同时太多用户请求所压垮。
参考文档https://doc.traefik.io/traefik/middlewares/http/ratelimit/
先模拟正常情况无任何限流措施对myapp1使用ab工具进行压力测试一共请求一百次每次并发10次。测试结果失败的请求为0总耗时0.412秒
[roottiaoban ~]# ab -n 100 -c 10 http://myapp1.test.com/
This is ApacheBench, Version 2.3 $Revision: 1843412 $
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking myapp1.test.com (be patient).....doneServer Software: nginx/1.12.2
Server Hostname: myapp1.test.com
Server Port: 80Document Path: /
Document Length: 65 bytesConcurrency Level: 10
Time taken for tests: 0.412 seconds
Complete requests: 100
Failed requests: 0
Total transferred: 27700 bytes
HTML transferred: 6500 bytes
Requests per second: 242.78 [#/sec] (mean)
Time per request: 41.189 [ms] (mean)
Time per request: 4.119 [ms] (mean, across all concurrent requests)
Transfer rate: 65.68 [Kbytes/sec] receivedConnection Times (ms)min mean[/-sd] median max
Connect: 1 4 4.3 3 16
Processing: 4 20 19.2 15 173
Waiting: 4 20 19.0 14 171
Total: 8 25 19.5 19 175Percentage of the requests served within a certain time (ms)50% 1966% 2775% 3080% 3290% 4395% 5598% 6199% 175100% 175 (longest request)
定义限流中间件RateLimit指定1s内请求数平均值不大于10个高峰最大值不大于50个。
[rootk8s-master middleware]# cat rate-limit-middleware.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: rate-limit-middleware
spec:rateLimit:burst: 10average: 50
[rootk8s-master middleware]# kubectl apply -f rate-limit-middleware.yaml
middleware.traefik.containo.us/rate-limit-middleware created
修改myapp1的ingressRoute添加RateLimit中间件
[rootk8s-master middleware]# cat myapp1-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp1
spec:entryPoints:- webroutes:- match: Host(myapp1.test.com)kind: Ruleservices:- name: myapp1 port: 80 middlewares:- name: rate-limit-middleware
[rootk8s-master middleware]# kubectl apply -f myapp1-ingress.yaml
ingressroute.traefik.containo.us/myapp1 created
接下来继续使用ab工具进行压力测试一共请求一百次每次并发10次。测试结果失败的请求为82次总耗时0.297秒
[roottiaoban ~]# ab -n 100 -c 10 http://myapp1.test.com/
This is ApacheBench, Version 2.3 $Revision: 1843412 $
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking myapp1.test.com (be patient).....doneServer Software: nginx/1.12.2
Server Hostname: myapp1.test.com
Server Port: 80Document Path: /
Document Length: 65 bytesConcurrency Level: 10
Time taken for tests: 0.297 seconds
Complete requests: 100
Failed requests: 82(Connect: 0, Receive: 0, Length: 82, Exceptions: 0)
Non-2xx responses: 82
Total transferred: 20562 bytes
HTML transferred: 2564 bytes
Requests per second: 336.30 [#/sec] (mean)
Time per request: 29.736 [ms] (mean)
Time per request: 2.974 [ms] (mean, across all concurrent requests)
Transfer rate: 67.53 [Kbytes/sec] receivedConnection Times (ms)min mean[/-sd] median max
Connect: 1 3 3.4 2 17
Processing: 2 12 15.9 9 151
Waiting: 2 11 15.7 7 150
Total: 3 15 16.2 11 153Percentage of the requests served within a certain time (ms)50% 1166% 1475% 1880% 2190% 2495% 3498% 4999% 153100% 153 (longest request)
熔断 熔断简介服务熔断的作用类似于我们家用的保险丝当某服务出现不可用或响应超时的情况时为了防止整个系统出现雪崩暂时停止对该服务的调用。熔断器三种状态 Closed关闭状态所有请求都正常访问。 Open打开状态所有请求都会被降级。traefik会对请求情况计数当一定时间内失败请求百分比达到阈值则触发熔断断路器会完全打开。 Recovering半开恢复状态open状态不是永久的打开后会进入休眠时间。随后断路器会自动进入半开状态。此时会释放部分请求通过若这些请求都是健康的则会完全关闭断路器否则继续保持打开再次进行休眠计时
服务熔断原理(断路器的原理)统计用户在指定的时间范围默认10s之内的请求总数达到指定的数量之后如果不健康的请求(超时、异常)占总请求数量的百分比50%达到了指定的阈值之后就会触发熔断。触发熔断断路器就会打开(open),此时所有请求都不能通过。在5s之后断路器会恢复到半开状态(half open)会允许少量请求通过如果这些请求都是健康的那么断路器会回到关闭状态(close).如果这些请求还是失败的请求,断路器还是恢复到打开的状态(open).traefik支持的触发器 NetworkErrorRatio网络错误率 ResponseCodeRatio状态代码比率 LatencyAtQuantileMS分位数的延迟以毫秒为单位
参考文档https://doc.traefik.io/traefik/middlewares/http/circuitbreaker/
定义熔断中间件circuitBreaker指定50% 的请求比例响应时间大于 1MS 时熔断。
[rootk8s-master middleware]# cat circuit-breaker-middleware.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: circuit-breaker-middleware
spec:circuitBreaker:expression: LatencyAtQuantileMS(50.0) 1
[rootk8s-master middleware]# kubectl apply -f circuit-breaker-middleware.yaml
middleware.traefik.containo.us/circuit-breaker-middleware created
修改myapp1的ingressRoute添加circuitBreaker中间件
rootk8s-master middleware]# cat myapp1-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myapp1
spec:entryPoints:- webroutes:- match: Host(myapp1.test.com)kind: Ruleservices:- name: myapp1 port: 80 middlewares:- name: circuit-breaker-middleware
rootk8s-master middleware]# kubectl apply -f myapp1-ingress.yaml
ingressroute.traefik.containo.us/myapp1 created
继续进行压力测试一共请求一千次每次并发100次。触发熔断机制测试结果失败的请求为999次总耗时0.938秒。
[roottiaoban ~]# ab -n 1000 -c 100 http://myapp1.test.com/
This is ApacheBench, Version 2.3 $Revision: 1843412 $
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking myapp1.test.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requestsServer Software: nginx/1.12.2
Server Hostname: myapp1.test.com
Server Port: 80Document Path: /
Document Length: 65 bytesConcurrency Level: 100
Time taken for tests: 0.938 seconds
Complete requests: 1000
Failed requests: 999(Connect: 0, Receive: 0, Length: 999, Exceptions: 0)
Non-2xx responses: 999
Total transferred: 153124 bytes
HTML transferred: 19046 bytes
Requests per second: 1065.54 [#/sec] (mean)
Time per request: 93.849 [ms] (mean)
Time per request: 0.938 [ms] (mean, across all concurrent requests)
Transfer rate: 159.34 [Kbytes/sec] receivedConnection Times (ms)min mean[/-sd] median max
Connect: 1 12 9.2 11 58
Processing: 2 66 53.3 53 314
Waiting: 1 63 51.7 51 309
Total: 5 79 52.8 66 326Percentage of the requests served within a certain time (ms)50% 6666% 8075% 9680% 10590% 12895% 16898% 28099% 309100% 326 (longest request)
自定义错误页
在实际的业务中肯定会存在4XX 5XX相关的错误异常如果每个应用都开发一个单独的错误页无疑大大增加了开发成本traefik同样也支持自定义错误页但是需要注意的是错误页面不是有traefik存储处理而是通过定义中间件将错误的请求重定向到其他的页面。 参考文档https://doc.traefik.io/traefik/middlewares/http/errorpages/
首先我们先创建一个应用使用flask开个一个简单的demo项目。这个web应用的功能是当请求/时返回状态码为200当请求/400时返回400状态码当请求/500时返回500状态码。应用的源代码如下 app.py from flask import Flask, abortapp Flask(__name__)app.route(/)
def hello_world():# put applications code herereturnHello World!app.route(/400)
def error_404():abort(400)app.route(/500)
def error_500():abort(500)if __name__ __main__:app.run() templates/index.html !DOCTYPE html
html langenheadmeta charsetUTF-8titleflask/title/headbodyh1hello flask/h1img src{{ url_for(static,filenamephoto.jpg) }} altphoto/body
/html
为了方便大家测试已将镜像打包上传至docker hub仓库
docker pull cuiliang0302/request-code:v2.0
接下来我们使用deployment控制器部署这个服务并创建svc资源
[rootk8s-master middleware]# cat flask.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: flask
spec:selector:matchLabels:app: flasktemplate:metadata:labels:app: flaskspec:containers:- name: flaskimage: cuiliang0302/request-code:v1.0resources:limits:memory: 128Micpu: 500mports:- containerPort: 5000
---
apiVersion: v1
kind: Service
metadata:name: flask
spec:type: ClusterIPselector:app: flaskports:- port: 5000targetPort: 5000[rootk8s-master middleware]# kubectl apply -f flask.yaml
deployment.apps/flask created
service/flask created
接下来创建ingressRouter资源
[rootk8s-master middleware]# cat flask-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: flask
spec:entryPoints:- webroutes:- match: Host(flask.test.com)kind: Ruleservices:- name: flask port: 5000
[rootk8s-master middleware]# kubectl apply -f flask-ingress.yaml
ingressroute.traefik.containo.us/flask created
使用域名访问验证先添加hosts解析记录192.168.93.128 flask.test.com分别请求不同的路径模拟4XX 5XX错误
[rootk8s-master middleware]# curl -I flask.test.com/
HTTP/1.1 200 OK
Content-Length: 12
Content-Type: text/html; charsetutf-8
Date: Wed, 28 Sep 2022 03:11:03 GMT
Server: Werkzeug/2.2.2 Python/3.10.1[rootk8s-master middleware]# curl -I flask.test.com/400
HTTP/1.1 400 Bad Request
Content-Length: 167
Content-Type: text/html; charsetutf-8
Date: Wed, 28 Sep 2022 03:11:07 GMT
Server: Werkzeug/2.2.2 Python/3.10.1[rootk8s-master middleware]# curl -I flask.test.com/500
HTTP/1.1 500 Internal Server Error
Content-Length: 265
Content-Type: text/html; charsetutf-8
Date: Wed, 28 Sep 2022 03:11:11 GMT
Server: Werkzeug/2.2.2 Python/3.10.1[rootk8s-master middleware]# curl -I flask.test.com/404
HTTP/1.1 404 Not Found
Content-Length: 207
Content-Type: text/html; charsetutf-8
Date: Wed, 28 Sep 2022 03:11:17 GMT
Server: Werkzeug/2.2.2 Python/3.10.1现在提出一个新的需求当我访问flask项目时如果错误码为400返回myapp1的页面如果错误码为500返回myapp2的页面(前提是myapp1和myapp2服务已创建)。我们创建errorpages中间件
[rootk8s-master middleware]# cat error-middleware.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: errors5
spec:errors:status:- 500-599# query: /{status}.html # 可以为每个页面定义一个状态码也可以指定5XX使用统一页面返回query : / # 指定返回myapp2的请求路径service:name: myapp2port: 80
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: errors4
spec:errors:status:- 400-499# query: /{status}.html # 可以为每个页面定义一个状态码也可以指定5XX使用统一页面返回query : / # 指定返回myapp1的请求路径service:name: myapp1port: 80
[rootk8s-master middleware]# kubectl apply -f error-middleware.yaml
middleware.traefik.containo.us/errors5 created
middleware.traefik.containo.us/errors4 created接下来修改ingressroute资源添加错误码中间件
[rootk8s-master middleware]# cat flask-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: flask
spec:entryPoints:- webroutes:- match: Host(flask.test.com)kind: Ruleservices:- name: flask port: 5000middlewares:- name: errors4- name: errors5
[rootk8s-master middleware]# kubectl apply -f error-middleware.yaml
middleware.traefik.containo.us/errors5 created
middleware.traefik.containo.us/errors4 created最后进行访问验证
# 测试200状态码
[rootk8s-master middleware]# curl -I flask.test.com/
HTTP/1.1 200 OK
Content-Length: 12
Content-Type: text/html; charsetutf-8
Date: Thu, 06 Oct 2022 00:36:19 GMT
Server: Werkzeug/2.2.2 Python/3.10.1[rootk8s-master middleware]# curl flask.test.com/
!DOCTYPE html
html langen
headmeta charsetUTF-8titleflask/title
/head
body
h1hello flask/h1
img src/static/photo.jpg altphoto
/body
# 测试400状态码
[rootk8s-master middleware]# curl -I flask.test.com/400
HTTP/1.1 400 Bad Request
Accept-Ranges: bytes
Content-Length: 65
Content-Type: text/html
Date: Thu, 06 Oct 2022 00:36:35 GMT
Etag: 5a98c760-41
Last-Modified: Fri, 02 Mar 2018 03:39:12 GMT
Server: nginx/1.12.2[rootk8s-master middleware]# curl flask.test.com/400
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
# 测试500状态码
[rootk8s-master middleware]# curl -I flask.test.com/500
HTTP/1.1 500 Internal Server Error
Accept-Ranges: bytes
Content-Length: 65
Content-Type: text/html
Date: Thu, 06 Oct 2022 00:36:46 GMT
Etag: 5a9251f0-41
Last-Modified: Sun, 25 Feb 2018 06:04:32 GMT
Server: nginx/1.12.2[rootk8s-master middleware]# curl flask.test.com/500
Hello MyApp | Version: v2 | a hrefhostname.htmlPod Name/a
# 测试404状态码
[rootk8s-master middleware]# curl -I flask.test.com/404
HTTP/1.1 404 Not Found
Accept-Ranges: bytes
Content-Length: 65
Content-Type: text/html
Date: Thu, 06 Oct 2022 00:37:00 GMT
Etag: 5a98c760-41
Last-Modified: Fri, 02 Mar 2018 03:39:12 GMT
Server: nginx/1.12.2[rootk8s-master middleware]# curl flask.test.com/404
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
数据压缩
有时候客户端和服务器之间会传输比较大的报文数据这时候就占用较大的网络带宽和时长。为了节省带宽加速报文的响应速速可以将传输的报文数据先进行压缩然后再进行传输traefik也同样支持数据压缩。 参考文档https://doc.traefik.io/traefik/middlewares/http/compress/[10]
traefik默认只对大于1024字节且请求标头包含Accept-Encoding gzip的资源进行压缩。可以指定排除特定类型不启用压缩或者根据内容大小来决定是否压缩。继续使用上面创建的flask应用现在创建中间件使用默认配置策略即可。
[rootk8s-master middleware]# cat compress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:name: compress
spec:compress: {}
[rootk8s-master middleware]# kubectl apply -f compress.yaml
middleware.traefik.containo.us/compress created修改flask的ingressrouter资源指定数据压缩中间件
[rootk8s-master middleware]# cat flask-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: flask
spec:entryPoints:- webroutes:- match: Host(flask.test.com)kind: Ruleservices:- name: flask port: 5000middlewares:- name: compress
[rootk8s-master middleware]# kubectl apply -f flask-ingress.yaml
ingressroute.traefik.containo.us/flask created
接下来查看浏览器f12调试信息
图片资源大于1024字节开启了压缩 html资源小于1024字节未启用压缩 四、入门Traefik系列——服务配置与使用
简介 traefik的路由规则就可以实现4层和7层的基本负载均衡操作使用IngressRoute[1]/IngressRouteTCP[2]/IngressRouteUDP[3]资源即可。但是如果想要实现加权轮询、流量复制等高级操作traefik抽象出了一个TraefikService[4]资源。此时整体流量走向为外部流量先通过entryPoints端口进入traefik然后由IngressRoute[5]/IngressRouteTCP[6]/IngressRouteUDP[7]匹配后进入TraefikService[8]在TraefikService[9]这一层实现加权轮循和流量复制最后将请求转发至kubernetes的service。除此之外traefik还支持7层的粘性会话、健康检查、传递请求头、响应转发、故障转移等操作。
加权轮询(灰度发布)
灰度发布我们有时候也会称为金丝雀发布Canary主要就是让一部分测试的服务也参与到线上去经过测试观察看是否符合上线要求在traefik中通过调整生产与测试服务的权重实现灰度发布的功能。接下来配置加权轮循仍然请求myapp.test.commyapp1的负载权重为1myapp2的负载权重为2。依旧是上面创建的两个deployment应用与对应的svc
[rootk8s-master udp]# kubectl get pod
NAME READY STATUS RESTARTS AGE
myapp1-795d947b45-9lsm6 1/1 Running 1 25h
myapp2-6ffd54f76-ljkr9 1/1 Running 1 25h
[rootk8s-master udp]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) 2d22h
myapp1 ClusterIP 10.104.91.200 none 80/TCP 25h
myapp2 ClusterIP 10.111.245.32 none 80/TCP创建IngressRouter资源配置域名为myapp.test.com注意此时后端service配置TraefikService。
[rootk8s-master ingress]# cat myapp-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myappnamespace: default
spec:entryPoints:- webroutes:- match: Host(myapp.test.com)kind: Ruleservices: # 加权轮循时后端service不再是k8s的service而是traefik的TraefikService- name: wrr namespace: defaultkind: TraefikService
[rootk8s-master ingress]# kubectl apply -f myapp-ingress.yaml
ingressroute.traefik.containo.us/myapp created
创建TraefikService资源名称与IngressRouter的TraefikService保持一致services后端填写kubernetes的service并指定权重。
[rootk8s-master ingress]# cat myapp-traefikService.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:name: wrrnamespace: defaultspec:weighted:services:- name: myapp1port: 80weight: 1- name: myapp2port: 80weight: 2
[rootk8s-master ingress]# kubectl apply -f myapp-traefikService.yaml
traefikservice.traefik.containo.us/wrr unchanged查看dashboard的配置信息此时myapp2权重为2myapp1权重为1。 客户端访问测试验证无误。
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v2 | a hrefhostname.htmlPod Name/a镜像复制(流量复制)
traefik还支持镜像复制功能是一种可以将流入流量复制并同时将其发送给其他服务的方法镜像服务可以获得给定百分比的请求同时也会忽略这部分请求的响应在实际生产中主要用于测试场景以及问题复现bug定位。依旧是上面创建的两个deployment应用与对应的svc
[rootk8s-master udp]# kubectl get pod
NAME READY STATUS RESTARTS AGE
myapp1-795d947b45-9lsm6 1/1 Running 1 25h
myapp2-6ffd54f76-ljkr9 1/1 Running 1 25h
[rootk8s-master udp]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) 2d22h
myapp1 ClusterIP 10.104.91.200 none 80/TCP 25h
myapp2 ClusterIP 10.111.245.32 none 80/TCP创建IngressRouter资源配置域名为myapp.test.com注意此时后端service配置TraefikService。
[rootk8s-master ingress]# cat myapp-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myappnamespace: default
spec:entryPoints:- webroutes:- match: Host(myapp.test.com)kind: Ruleservices: # 流量复制时后端service不再是k8s的service而是traefik的TraefikService- name: mirror namespace: defaultkind: TraefikService
[rootk8s-master ingress]# kubectl apply -f myapp-ingress.yaml
ingressroute.traefik.containo.us/myapp created
创建TraefikService资源名称与IngressRouter的TraefikService保持一致services后端填写kubernetes的service并设置复制流量比例。
[rootk8s-master ingress]# cat myapp-traefikService.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:name: mirrornamespace: defaultspec:mirroring: # 所有流量全部请求到k8s的myapp1name: myapp1port: 80mirrors: # 同时复制50%的请求到k8s的myapp2 - name: myapp2port: 80percent: 50
[rootk8s-master ingress]# kubectl apply -f myapp-traefikService.yaml
traefikservice.traefik.containo.us/mirror created查看dashboard的配置信息此时myapp2类型为mirroring比例为50% 客户端访问测试只响应了myapp1的内容。
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl myapp.test.com
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a查看myapp1和myapp2的日志发现有请求日志符合预期。
[rootk8s-master ingress]# kubectl logs myapp1-795d947b45-9lsm6
10.244.0.0 - - [25/Sep/2022:08:45:33 0000] GET / HTTP/1.1 200 65 -curl/7.61.1192.168.93.1
10.244.0.0 - - [25/Sep/2022:08:45:35 0000] GET / HTTP/1.1 200 65 -curl/7.61.1192.168.93.1
10.244.0.0 - - [25/Sep/2022:08:45:36 0000] GET / HTTP/1.1 200 65 -curl/7.61.1192.168.93.1
10.244.0.0 - - [25/Sep/2022:09:45:38 0000] GET / HTTP/1.1 200 65 -curl/7.61.1192.168.93.1
[rootk8s-master ingress]# kubectl logs myapp2-6ffd54f76-ljkr9
10.244.0.0 - - [25/Sep/2022:08:45:33 0000] GET / HTTP/1.1 200 65 -curl/7.61.1192.168.93.1
10.244.0.0 - - [25/Sep/2022:08:45:36 0000] GET / HTTP/1.1 200 65 -curl/7.61.1192.168.93.1粘性会话(会话保持)
当我们使用traefik的负载均衡时默认情况下轮循多个k8s的service服务如果用户对同一内容的多次请求可能被转发到了不同的后端服务器。假设用户发出请求被分配至服务器A保存了一些信息在session中该用户再次发送请求被分配到服务器B要用之前保存的信息若服务器A和B之间没有session粘滞那么服务器B就拿不到之前的信息这样会导致一些问题。traefik同样也支持粘性会话可以让用户在一次会话周期内的所有请求始终转发到一台特定的后端服务器上。依旧是上面创建的两个deployment应用与对应的svc
[rootk8s-master udp]# kubectl get pod
NAME READY STATUS RESTARTS AGE
myapp1-795d947b45-9lsm6 1/1 Running 1 25h
myapp2-6ffd54f76-ljkr9 1/1 Running 1 25h
[rootk8s-master udp]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) 2d22h
myapp1 ClusterIP 10.104.91.200 none 80/TCP 25h
myapp2 ClusterIP 10.111.245.32 none 80/TCP创建IngressRouter资源配置域名为myapp.test.com注意此时后端service配置TraefikService。
[rootk8s-master ingress]# cat myapp-ingress.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:name: myappnamespace: default
spec:entryPoints:- webroutes:- match: Host(myapp.test.com)kind: Ruleservices: # 粘性会话依赖加权轮循后端service不再是k8s的service而是traefik的TraefikService- name: wrr namespace: defaultkind: TraefikService
[rootk8s-master ingress]# kubectl apply -f myapp-ingress.yaml
ingressroute.traefik.containo.us/myapp created创建TraefikService资源名称与IngressRouter的TraefikService保持一致services后端填写kubernetes的service并指定权重。
[rootk8s-master ingress]# cat myapp-traefikService.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:name: wrrnamespace: defaultspec:weighted:services:- name: myapp1kind: Serviceport: 80weight: 1- name: myapp2kind: Serviceweight: 2port: 80sticky: # 开启粘性会话cookie: # 基于cookie区分客户端name: lvl1 # 指定客户端请求时包含的cookie名称
[rootk8s-master ingress]# kubectl apply -f myapp-traefikService.yaml
traefikservice.traefik.containo.us/wrr unchanged客户端携带cookie信息访问测试。
# lvl1为default-myapp2-80的请求全部由myapp2响应
[roottiaoban ~]# curl -b lvl1default-myapp2-80 http://myapp.test.com
Hello MyApp | Version: v2 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl -b lvl1default-myapp2-80 http://myapp.test.com
Hello MyApp | Version: v2 | a hrefhostname.htmlPod Name/a
# lvl1为default-myapp1-80的请求全部由myapp1响应
[roottiaoban ~]# curl -b lvl1default-myapp1-80 http://myapp.test.com
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
[roottiaoban ~]# curl -b lvl1default-myapp1-80 http://myapp.test.com
Hello MyApp | Version: v1 | a hrefhostname.htmlPod Name/a
