K8s flink-operator 例子

1.参考官网：

https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-stable/docs/try-flink-kubernetes-operator/quick-start/

2.首先环境具备 k8s、helm

我的环境 k8s 1.30 最新版本了

[root@k8s-master ~]# kubectl get no -owide
NAME         STATUS   ROLES           AGE    VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION                CONTAINER-RUNTIME
k8s-master   Ready    control-plane   156d   v1.30.0   192.168.56.211   <none>        CentOS Linux 7 (Core)   5.4.274-1.el7.elrepo.x86_64   containerd://1.6.9
k8s-node1    Ready    <none>          156d   v1.30.0   192.168.56.212   <none>        CentOS Linux 7 (Core)   5.4.274-1.el7.elrepo.x86_64   containerd://1.6.9
k8s-node2    Ready    <none>          156d   v1.30.0   192.168.56.213   <none>        CentOS Linux 7 (Core)   5.4.274-1.el7.elrepo.x86_64   containerd://1.6.9
[root@k8s-master ~]# kubectl get all -A
NAMESPACE     NAME                                            READY   STATUS    RESTARTS      AGE
default       pod/flink-kubernetes-operator-f56d6bf6f-skcnn   2/2     Running   0             23m
kube-system   pod/calico-kube-controllers-6df7596dbd-4scr5    1/1     Running   0             22m
kube-system   pod/calico-node-8b5p2                           1/1     Running   6 (30m ago)   156d
kube-system   pod/calico-node-p8bhd                           1/1     Running   6 (29m ago)   156d
kube-system   pod/calico-node-x4dr2                           1/1     Running   7 (15m ago)   156d
kube-system   pod/coredns-7b5944fdcf-ms42h                    1/1     Running   0             22m
kube-system   pod/coredns-7b5944fdcf-rndst                    1/1     Running   0             22m
kube-system   pod/etcd-k8s-master                             1/1     Running   7 (15m ago)   156d
kube-system   pod/kube-apiserver-k8s-master                   1/1     Running   7 (15m ago)   156d
kube-system   pod/kube-controller-manager-k8s-master          1/1     Running   9 (15m ago)   156d
kube-system   pod/kube-proxy-dr4lp                            1/1     Running   6 (29m ago)   156d
kube-system   pod/kube-proxy-nsvhl                            1/1     Running   6 (30m ago)   156d
kube-system   pod/kube-proxy-sgkfs                            1/1     Running   7 (15m ago)   156d
kube-system   pod/kube-scheduler-k8s-master                   1/1     Running   7 (15m ago)   156dNAMESPACE     NAME                                     TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE
default       service/flink-operator-webhook-service   ClusterIP   10.97.88.103   <none>        443/TCP                  6d23h
default       service/kubernetes                       ClusterIP   10.96.0.1      <none>        443/TCP                  156d
kube-system   service/kube-dns                         ClusterIP   10.96.0.10     <none>        53/UDP,53/TCP,9153/TCP   156dNAMESPACE     NAME                         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system   daemonset.apps/calico-node   3         3         3       3            3           kubernetes.io/os=linux   156d
kube-system   daemonset.apps/kube-proxy    3         3         3       3            3           kubernetes.io/os=linux   156dNAMESPACE     NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
default       deployment.apps/flink-kubernetes-operator   1/1     1            1           6d23h
kube-system   deployment.apps/calico-kube-controllers     1/1     1            1           156d
kube-system   deployment.apps/coredns                     2/2     2            2           156dNAMESPACE     NAME                                                  DESIRED   CURRENT   READY   AGE
default       replicaset.apps/flink-kubernetes-operator-f56d6bf6f   1         1         1       6d23h
kube-system   replicaset.apps/calico-kube-controllers-6df7596dbd    1         1         1       156d
kube-system   replicaset.apps/coredns-7b5944fdcf                    2         2         2       156d
[root@k8s-master ~]#

3.在 Kubernetes 集群上安装证书管理器以添加 Webhook 组件（每个 Kubernetes 集群只需要一次）

kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.8.2/cert-manager.yaml

如果证书管理器安装因任何原因失败，可以通过传递 --set webhook.create=false给操作员的 helm install 命令来禁用 Webhook。

4.Helm 图表部署选定的稳定 Flink Kubernetes Operator 版本：

要查找稳定版本列表，请访问Downloads | Apache Flink

Helm 图表默认指向ghcr.io/apache/flink-kubernetes-operator镜像存储库。如果您遇到连接问题或者更喜欢使用 Dockerhub，则可以--set image.repository=apache/flink-kubernetes-operator在安装过程中使用。

执行命令：

helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator  --set image.repository=apache/flink-kubernetes-operator

helm repo add flink-operator-repo https://downloads.apache.org/flink/flink-kubernetes-operator-1.9.0/
helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator

[root@k8s-master ~]# helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator
NAME: flink-kubernetes-operator
LAST DEPLOYED: Wed Sep 25 19:43:46 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
[root@k8s-master ~]# helm list
NAME                            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                           APP VERSION
flink-kubernetes-operator       default         1               2024-09-25 19:43:46.63808568 +0800 CST  deployed        flink-kubernetes-operator-1.9.0 1.9.0
[root@k8s-master ~]# kubectl get all -A
NAMESPACE      NAME                                             READY   STATUS    RESTARTS      AGE
cert-manager   pod/cert-manager-cainjector-5dbdc949c4-gjfln     1/1     Running   0             9m45s
cert-manager   pod/cert-manager-d68cffc95-rz7nr                 1/1     Running   0             9m45s
cert-manager   pod/cert-manager-webhook-759ddb6555-4p7n6        1/1     Running   0             9m45s
default        pod/flink-kubernetes-operator-68cbbd7779-l5bbr   2/2     Running   0             38s
kube-system    pod/calico-kube-controllers-6df7596dbd-4scr5     1/1     Running   0             34m
kube-system    pod/calico-node-8b5p2                            1/1     Running   6 (41m ago)   156d
kube-system    pod/calico-node-p8bhd                            1/1     Running   6 (41m ago)   156d
kube-system    pod/calico-node-x4dr2                            1/1     Running   7 (27m ago)   156d
kube-system    pod/coredns-7b5944fdcf-ms42h                     1/1     Running   0             34m
kube-system    pod/coredns-7b5944fdcf-rndst                     1/1     Running   0             34m
kube-system    pod/etcd-k8s-master                              1/1     Running   7 (27m ago)   156d
kube-system    pod/kube-apiserver-k8s-master                    1/1     Running   7 (27m ago)   156d
kube-system    pod/kube-controller-manager-k8s-master           1/1     Running   9 (27m ago)   156d
kube-system    pod/kube-proxy-dr4lp                             1/1     Running   6 (41m ago)   156d
kube-system    pod/kube-proxy-nsvhl                             1/1     Running   6 (41m ago)   156d
kube-system    pod/kube-proxy-sgkfs                             1/1     Running   7 (27m ago)   156d
kube-system    pod/kube-scheduler-k8s-master                    1/1     Running   7 (27m ago)   156dNAMESPACE      NAME                                     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
cert-manager   service/cert-manager                     ClusterIP   10.108.134.142   <none>        9402/TCP                 9m46s
cert-manager   service/cert-manager-webhook             ClusterIP   10.107.201.217   <none>        443/TCP                  9m46s
default        service/flink-operator-webhook-service   ClusterIP   10.100.57.92     <none>        443/TCP                  38s
default        service/kubernetes                       ClusterIP   10.96.0.1        <none>        443/TCP                  156d
kube-system    service/kube-dns                         ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP   156dNAMESPACE     NAME                         DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
kube-system   daemonset.apps/calico-node   3         3         3       3            3           kubernetes.io/os=linux   156d
kube-system   daemonset.apps/kube-proxy    3         3         3       3            3           kubernetes.io/os=linux   156dNAMESPACE      NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
cert-manager   deployment.apps/cert-manager                1/1     1            1           9m46s
cert-manager   deployment.apps/cert-manager-cainjector     1/1     1            1           9m46s
cert-manager   deployment.apps/cert-manager-webhook        1/1     1            1           9m45s
default        deployment.apps/flink-kubernetes-operator   1/1     1            1           38s
kube-system    deployment.apps/calico-kube-controllers     1/1     1            1           156d
kube-system    deployment.apps/coredns                     2/2     2            2           156dNAMESPACE      NAME                                                   DESIRED   CURRENT   READY   AGE
cert-manager   replicaset.apps/cert-manager-cainjector-5dbdc949c4     1         1         1       9m46s
cert-manager   replicaset.apps/cert-manager-d68cffc95                 1         1         1       9m46s
cert-manager   replicaset.apps/cert-manager-webhook-759ddb6555        1         1         1       9m45s
default        replicaset.apps/flink-kubernetes-operator-68cbbd7779   1         1         1       38s
kube-system    replicaset.apps/calico-kube-controllers-6df7596dbd     1         1         1       156d
kube-system    replicaset.apps/coredns-7b5944fdcf                     2         2         2       156d
[root@k8s-master ~]#

5.提交 Flink 作业 #

一旦操作员如上一步所示运行，您就可以提交 Flink 作业了：

kubectl create -f https://raw.githubusercontent.com/apache/flink-kubernetes-operator/release-1.9/examples/basic.yaml

您可以跟踪您的作业日志，在成功启动后（在新环境中可能需要一分钟，几秒钟后），您可以：

kubectl logs -f deploy/basic-example

要公开 Flink 仪表板，您可以添加端口转发规则或查看入口配置选项：

 kubectl port-forward svc/basic-example-rest 8081:8081 --address 192.168.56.211
或者是：kubectl port-forward svc/basic-example-rest 8081:8081 --address 0.0.0.0

现在可以通过 192.168.56.211:8081 访问 Flink 仪表板。