K8s版本

kubectl get nodes

jenkins版本：
最新版

安装Jenkins

安装nfs服务，可以选择自己的任意一台机器，我选择的是k8s的控制节点k8smaster1
如果已经安装过nfs，这个步骤可以忽略

yum install nfs-utils -y
systemctl start nfs
systemctl enable nfs

在k8smaster1上创建一个nfs共享目录

mkdir /data/v2  -p
vim /etc/exports

/data/v1 *(rw,no_root_squash)
/data/v2 *(rw,no_root_squash)

使配置文件生效

exportfs -arv

在kubernetes中部署jenkins

创建名称空间

kubectl create namespace jenkins-k8s

创建pv,上传pv.yaml

cat pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:name: jenkins-k8s-pv
spec:capacity:storage: 10GiaccessModes:- ReadWriteManynfs:server: 192.168.40.180path: /data/v2

kubectl apply -f pv.yaml
kubectl get pv

创建pvc

cat pvc.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:name: jenkins-k8s-pvcnamespace: jenkins-k8s
spec:resources:requests:storage: 10GiaccessModes:- ReadWriteMany

kubectl apply -f pvc.yaml
kubectl get pvc -n jenkins-k8s

创建一个sa账号

kubectl create sa jenkins-k8s-sa -n jenkins-k8s

把上面的sa账号做rbac授权

kubectl create clusterrolebinding jenkins-k8s-sa-cluster -n jenkins-k8s  --clusterrole=cluster-admin --serviceaccount=jenkins-k8s:jenkins-k8s-sa

通过deployment部署jenkins

在k8snode节点执行
jenkins安装的时候，每次镜像都要下载最新的，安装最新版jenkins，如何下载最新的镜像？

docker pull jenkins/jenkins:2.394
docker save -o jenkins2.394  jenkins/jenkins:2.394
ctr -n=k8s.io images import jenkins2.394

docker load -i jenkins-slave-latest.tar.gz
ctr -n=k8s.io images import jenkins-slave-latest.tar.gz

备注：jenkins-slave-latest.tar.gz这个里面封装的镜像是jenkins-slave-latest:v1，这个jenkins-slave-latest:v1镜像制作方法如下：

cd /root/slave
cat dockerfile

FROM jenkins/jnlp-slave:4.13.3-1-jdk11
USER root
# 安装Docker
RUN apt-get update && apt-get install -y \docker.io
# 将当前用户加入docker用户组
RUN usermod -aG docker jenkins`在这里插入代码片`
RUN curl -LO https://dl.k8s.io/release/stable.txt
RUN curl -LO https://dl.k8s.io/release/$(cat stable.txt)/bin/linux/amd64/kubectl
RUN chmod +x kubectl
RUN mv kubectl /usr/local/bin/
ENV DOCKER_HOST unix:///var/run/docker.sock

docker build -t=jenkins-slave-latest:v1 .
docker save -o jenkins-slave-latest.tar.gz  jenkins-slave-latest:v1

更新资源清单文件

cat jenkins-deployment.yaml

kind: Deployment
apiVersion: apps/v1
metadata:name: jenkinsnamespace: jenkins-k8s
spec:replicas: 1selector:matchLabels:app: jenkinstemplate:metadata:labels:app: jenkinsspec:serviceAccount: jenkins-k8s-sacontainers:- name: jenkinsimage:  jenkins/jenkins:2.361.4imagePullPolicy: IfNotPresentports:- containerPort: 8080name: webprotocol: TCP- containerPort: 50000name: agentprotocol: TCPresources:limits:cpu: 2000mmemory: 2Girequests:cpu: 500mmemory: 512MilivenessProbe:httpGet:path: /loginport: 8080initialDelaySeconds: 60timeoutSeconds: 5failureThreshold: 12readinessProbe:httpGet:path: /loginport: 8080initialDelaySeconds: 60timeoutSeconds: 5failureThreshold: 12volumeMounts:- name: jenkins-volumesubPath: jenkins-homemountPath: /var/jenkins_homevolumes:- name: jenkins-volumepersistentVolumeClaim:claimName: jenkins-k8s-pvc

kubectl apply -f jenkins-deployment.yaml
kubectl get pods -n jenkins-k8s

NAME                       READY   STATUS   RESTARTS     AGE
jenkins-75c478947f-2tx8h   0/1     Error    1 (3s ago)   4s

看到jenkins-75c478947f-2tx8h 是CrashLoopBackOff状态，查看日志：

kubectl logs jenkins-75c478947f-2tx8h -n jenkins-k8s

报错显示没有权限操作/var/jenkins_home/copy_reference_file.log文件

touch: cannot touch '/var/jenkins_home/copy_reference_file.log': Permission denied
Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions?

解决办法如下：在k8smaster执行

kubectl delete -f jenkins-deployment.yaml
chown -R 1000.1000 /data/v2

kubectl apply -f jenkins-deployment.yaml

查看pod是否创建成功:

kubectl get pods -n jenkins-k8s

把jenkins前端加上service，提供外部网络访问

cat jenkins-service.yaml

apiVersion: v1
kind: Service
metadata:name: jenkins-servicenamespace: jenkins-k8slabels:app: jenkins
spec:selector:app: jenkinstype: NodePortports:- name: webport: 8080targetPort: webnodePort: 30002- name: agentport: 50000targetPort: agent

kubectl apply -f jenkins-service.yaml
kubectl get svc -n jenkins-k8s

通过上面可以看到service的8080端口在物理机映射的端口是30002

配置Jenkins

在浏览器访问jenkins的web界面：

获取管理员密码

在nfs服务端，也就是我们的master1节点获取密码：

cat  /data/v2/jenkins-home/secrets/initialAdminPassword

把上面获取到的密码拷贝到上面管理员密码下的方框里

点击继续，出现如下界面

安装插件,安装推荐的插件

插件安装好之后显示如下,创建第一个管理员用户

用户名和密码都设置成admin，线上环境需要设置成复杂的密码
修改好之后点击保存并完成，出现如下界面
点击保存并完成，出现如下界面

点击保存并完成，出现如下界面

测试jenkins的CI/CD