kubekey 离线安装harbor、k8s、kubesphere

目录

参考文献

了解kubekey  英文和中文

前提条件

部署准备

下载kubukey

离线包配置和制作

配置离线包

制作离线包

离线安装集群

复制KubeKey 和制品 artifact到离线机器

创建初始换、安装配置文件

 安装镜像仓库harbor

初始化harbor 项目

修改配置文件

安装k8s集群和kubesphere

手动安装依赖包

检查环境是否满足要求

检查防火墙端口是否满足要求

安装完成并登录

安装过程中遇到的问题

制作制品，不能下载github上的操作系统iso

初始化安装harbor报错must specify a CommonName

初始化harbor后，发现harbor的一些模块容器没有正常启动

麒麟系统安装，安装包没有 Fkylin-v10-amd64.iso

离线安装却在线下载calicoctl

参考文献

官网离线安装文档

了解kubekey  英文和中文

./kk --help
Deploy a Kubernetes or KubeSphere cluster efficiently, flexibly and easily. There are three scenarios to use KubeKey.
1. Install Kubernetes only
2. Install Kubernetes and KubeSphere together in one command
3. Install Kubernetes first, then deploy KubeSphere on it using https://github.com/kubesphere/ks-installerUsage:kk [command]Available Commands:add         Add nodes to kubernetes clusteralpha       Commands for features in alphaartifact    Manage a KubeKey offline installation packagecerts       cluster certscompletion  Generate shell completion scriptscreate      Create a cluster or a cluster configuration filedelete      Delete node or clusterhelp        Help about any commandinit        Initializes the installation environmentplugin      Provides utilities for interacting with pluginsupgrade     Upgrade your cluster smoothly to a newer version with this commandversion     print the client version informationFlags:-h, --help   help for kkDeploy a kubernetes or kubesphere cluster efficiently, flexibly and easily. There are three scenarios to use kubekey.
1. 仅安装kubernetes 
2. 一条命令同时安装kubernetes和kubesphere
3. 现在安装kubernetes,然后在使用ks-installer在k8s上部署kubesphere，ks-installer参考：https://github.com/kubesphere/ks-installer
语法:kk [command]
可用命令s:add         k8s集群添加节点alpha       Commands for features in alphaartifact    管理kubekey离线下载的安装包certs       集群证书completion  生成 shell 完成脚本create      创建一个集群或创建集群配置文件delete      删除节点或删除集群help        帮助init        初始化安装环境plugin      Provides utilities for interacting with pluginsupgrade     平滑升级集群version     打印kk版本信息

前提条件

要开始进行多节点安装，您需要参考如下示例准备至少三台主机。

主机 IP主机名称角色
192.168.0.2node1联网主机用于制作离线包
192.168.0.3node2离线环境主节点
192.168.0.4node3离线环境镜像仓库节点

关闭防火墙、selinux、swap、dnsmasq(所有节点)
关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

关闭selinux

sed -i 's/enforcing/disabled/' /etc/selinux/config  #永久
setenforce 0  #临时

关闭swap（k8s禁止虚拟内存以提高性能）

sed -ri 's/.*swap.*/#&/' /etc/fstab #永久
swapoff -a #临时
//关闭dnsmasq（否则可能导致docker容器无法解析域名）

service dnsmasq stop 
systemctl disable dnsmaq

有的机器不允许关闭防火墙可以看下文需要开放的端口

部署准备

下载kubukey

执行以下命令下载 KubeKey 并解压

方式一（可以访问github）：

从 GitHub Release Page 下载 KubeKey 或者直接运行以下命令。

curl -sfL https://get-kk.kubesphere.io | VERSION=v3.0.13 sh -

方式二：

首先运行以下命令，以确保您从正确的区域下载 KubeKey。

export KKZONE=cn

运行以下命令来下载 KubeKey：

curl -sfL https://get-kk.kubesphere.io | VERSION=v3.0.13 sh -

离线包配置和制作

配置离线包

在联网主机上执行以下命令，并复制示例中的 manifest 内容。

vim manifest.yaml

---apiVersion: kubekey.kubesphere.io/v1alpha2kind: Manifestmetadata:name: samplespec:arches:- amd64operatingSystems:- arch: amd64type: linuxid: centosversion: "7"repository:iso:localPath:url: https://github.com/kubesphere/kubekey/releases/download/v3.0.10/centos7-rpms-amd64.iso- arch: amd64type: linuxid: ubuntuversion: "20.04"repository:iso:localPath:url: https://github.com/kubesphere/kubekey/releases/download/v3.0.10/ubuntu-20.04-debs-amd64.isokubernetesDistributions:- type: kubernetesversion: v1.23.15components:helm:version: v3.9.0cni:version: v1.2.0etcd:version: v3.4.13calicoctl:version: v3.23.2## For now, if your cluster container runtime is containerd, KubeKey will add a docker 20.10.8 container runtime in the below list.## The reason is KubeKey creates a cluster with containerd by installing a docker first and making kubelet connect the socket file of containerd which docker contained.containerRuntimes:- type: dockerversion: 20.10.8- type: containerdversion: 1.6.4crictl:version: v1.24.0docker-registry:version: "2"harbor:version: v2.5.3docker-compose:version: v2.2.2images:- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-apiserver:v1.23.15- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controller-manager:v1.23.15- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-proxy:v1.23.15- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-scheduler:v1.23.15- registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.6- registry.cn-beijing.aliyuncs.com/kubesphereio/coredns:1.8.6- registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.23.2- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.23.2- registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.23.2- registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.23.2- registry.cn-beijing.aliyuncs.com/kubesphereio/typha:v3.23.2- registry.cn-beijing.aliyuncs.com/kubesphereio/flannel:v0.12.0- registry.cn-beijing.aliyuncs.com/kubesphereio/provisioner-localpv:3.3.0- registry.cn-beijing.aliyuncs.com/kubesphereio/linux-utils:3.3.0- registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.3- registry.cn-beijing.aliyuncs.com/kubesphereio/nfs-subdir-external-provisioner:v4.0.2- registry.cn-beijing.aliyuncs.com/kubesphereio/k8s-dns-node-cache:1.15.12- registry.cn-beijing.aliyuncs.com/kubesphereio/ks-installer:v3.4.1- registry.cn-beijing.aliyuncs.com/kubesphereio/ks-apiserver:v3.4.1- registry.cn-beijing.aliyuncs.com/kubesphereio/ks-console:v3.4.1- registry.cn-beijing.aliyuncs.com/kubesphereio/ks-controller-manager:v3.4.1- registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.22.0- registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.21.0- registry.cn-beijing.aliyuncs.com/kubesphereio/kubectl:v1.20.0- registry.cn-beijing.aliyuncs.com/kubesphereio/kubefed:v0.8.1- registry.cn-beijing.aliyuncs.com/kubesphereio/tower:v0.2.1- registry.cn-beijing.aliyuncs.com/kubesphereio/minio:RELEASE.2019-08-07T01-59-21Z- registry.cn-beijing.aliyuncs.com/kubesphereio/mc:RELEASE.2019-08-07T23-14-43Z- registry.cn-beijing.aliyuncs.com/kubesphereio/snapshot-controller:v4.0.0- registry.cn-beijing.aliyuncs.com/kubesphereio/nginx-ingress-controller:v1.1.0- registry.cn-beijing.aliyuncs.com/kubesphereio/defaultbackend-amd64:1.4- registry.cn-beijing.aliyuncs.com/kubesphereio/metrics-server:v0.4.2- registry.cn-beijing.aliyuncs.com/kubesphereio/redis:5.0.14-alpine- registry.cn-beijing.aliyuncs.com/kubesphereio/haproxy:2.0.25-alpine- registry.cn-beijing.aliyuncs.com/kubesphereio/alpine:3.14- registry.cn-beijing.aliyuncs.com/kubesphereio/openldap:1.3.0- registry.cn-beijing.aliyuncs.com/kubesphereio/netshoot:v1.0- registry.cn-beijing.aliyuncs.com/kubesphereio/cloudcore:v1.13.0- registry.cn-beijing.aliyuncs.com/kubesphereio/iptables-manager:v1.13.0- registry.cn-beijing.aliyuncs.com/kubesphereio/edgeservice:v0.3.0- registry.cn-beijing.aliyuncs.com/kubesphereio/gatekeeper:v3.5.2- registry.cn-beijing.aliyuncs.com/kubesphereio/openpitrix-jobs:v3.3.2- registry.cn-beijing.aliyuncs.com/kubesphereio/devops-apiserver:ks-v3.4.1- registry.cn-beijing.aliyuncs.com/kubesphereio/devops-controller:ks-v3.4.1- registry.cn-beijing.aliyuncs.com/kubesphereio/devops-tools:ks-v3.4.1- registry.cn-beijing.aliyuncs.com/kubesphereio/ks-jenkins:v3.4.0-2.319.3-1- registry.cn-beijing.aliyuncs.com/kubesphereio/inbound-agent:4.10-2- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.2- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.1-jdk11- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.16- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.17- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.18- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-base:v3.2.2-podman- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-nodejs:v3.2.0-podman- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.0-podman- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-maven:v3.2.1-jdk11-podman- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-python:v3.2.0-podman- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.0-podman- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.16-podman- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.17-podman- registry.cn-beijing.aliyuncs.com/kubesphereio/builder-go:v3.2.2-1.18-podman- registry.cn-beijing.aliyuncs.com/kubesphereio/s2ioperator:v3.2.1- registry.cn-beijing.aliyuncs.com/kubesphereio/s2irun:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/s2i-binary:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java11-runtime:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/tomcat85-java8-runtime:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/java-8-runtime:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/java-11-runtime:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-8-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-6-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/nodejs-4-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/python-36-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/python-35-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/python-34-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/python-27-centos7:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/argocd:v2.3.3- registry.cn-beijing.aliyuncs.com/kubesphereio/argocd-applicationset:v0.4.1- registry.cn-beijing.aliyuncs.com/kubesphereio/dex:v2.30.2- registry.cn-beijing.aliyuncs.com/kubesphereio/redis:6.2.6-alpine- registry.cn-beijing.aliyuncs.com/kubesphereio/configmap-reload:v0.7.1- registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus:v2.39.1- registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-config-reloader:v0.55.1- registry.cn-beijing.aliyuncs.com/kubesphereio/prometheus-operator:v0.55.1- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.11.0- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-state-metrics:v2.6.0- registry.cn-beijing.aliyuncs.com/kubesphereio/node-exporter:v1.3.1- registry.cn-beijing.aliyuncs.com/kubesphereio/alertmanager:v0.23.0- registry.cn-beijing.aliyuncs.com/kubesphereio/thanos:v0.31.0- registry.cn-beijing.aliyuncs.com/kubesphereio/grafana:8.3.3- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-rbac-proxy:v0.11.0- registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager-operator:v2.3.0- registry.cn-beijing.aliyuncs.com/kubesphereio/notification-manager:v2.3.0- registry.cn-beijing.aliyuncs.com/kubesphereio/notification-tenant-sidecar:v3.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-curator:v5.7.6- registry.cn-beijing.aliyuncs.com/kubesphereio/elasticsearch-oss:6.8.22- registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch:2.6.0- registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch-dashboards:2.6.0- registry.cn-beijing.aliyuncs.com/kubesphereio/opensearch-curator:v0.0.5- registry.cn-beijing.aliyuncs.com/kubesphereio/fluentbit-operator:v0.14.0- registry.cn-beijing.aliyuncs.com/kubesphereio/docker:19.03- registry.cn-beijing.aliyuncs.com/kubesphereio/fluent-bit:v1.9.4- registry.cn-beijing.aliyuncs.com/kubesphereio/log-sidecar-injector:v1.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/filebeat:6.7.0- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-operator:v0.6.0- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-exporter:v0.6.0- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-events-ruler:v0.6.0- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-operator:v0.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-auditing-webhook:v0.2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/pilot:1.14.6- registry.cn-beijing.aliyuncs.com/kubesphereio/proxyv2:1.14.6- registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-operator:1.29- registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-agent:1.29- registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-collector:1.29- registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-query:1.29- registry.cn-beijing.aliyuncs.com/kubesphereio/jaeger-es-index-cleaner:1.29- registry.cn-beijing.aliyuncs.com/kubesphereio/kiali-operator:v1.50.1- registry.cn-beijing.aliyuncs.com/kubesphereio/kiali:v1.50- registry.cn-beijing.aliyuncs.com/kubesphereio/busybox:1.31.1- registry.cn-beijing.aliyuncs.com/kubesphereio/nginx:1.14-alpine- registry.cn-beijing.aliyuncs.com/kubesphereio/wget:1.0- registry.cn-beijing.aliyuncs.com/kubesphereio/hello:plain-text- registry.cn-beijing.aliyuncs.com/kubesphereio/wordpress:4.8-apache- registry.cn-beijing.aliyuncs.com/kubesphereio/hpa-example:latest- registry.cn-beijing.aliyuncs.com/kubesphereio/fluentd:v1.4.2-2.0- registry.cn-beijing.aliyuncs.com/kubesphereio/perl:latest- registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-productpage-v1:1.16.2- registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v1:1.16.2- registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-reviews-v2:1.16.2- registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-details-v1:1.16.2- registry.cn-beijing.aliyuncs.com/kubesphereio/examples-bookinfo-ratings-v1:1.16.3- registry.cn-beijing.aliyuncs.com/kubesphereio/scope:1.13.0

备注

  • 若需要导出的 artifact 文件中包含操作系统依赖文件（如：conntarck、chrony 等），可在 operationSystem 元素中的 .repostiory.iso.url 中配置相应的 ISO 依赖文件下载地址或者提前下载 ISO 包到本地在 localPath 里填写本地存放路径并删除 url 配置项。

  • 开启 harbor 和 docker-compose 配置项，为后面通过 KubeKey 自建 harbor 仓库推送镜像使用。

  • 默认创建的 manifest 里面的镜像列表从 docker.io 获取。

  • 可根据实际情况修改 manifest-sample.yaml 文件的内容，用于之后导出期望的 artifact 文件。

  • 您可以访问 Release v3.0.7 🌈 · kubesphere/kubekey · GitHub 下载 ISO 文件。

个人备注： 

 在这一步我遇到了操作系统配置在githab上，然后下载不下来的情况

然后我去github上手动下载下来放到了本地，然后下盖下面的配置部分

 operatingSystems:- arch: amd64type: linuxid: centosversion: "7"repository:iso:localPath: 添加你的本地地址url: - arch: amd64type: linuxid: ubuntuversion: "20.04"repository:iso:localPath: 添加你的本地地址url:

下载地址

 https://github.com/kubesphere/kubekey/releases/tag/v3.0.10

制作离线包

导出制品 artifact。 

方式一（可以访问github）：

执行以下命令：

./kk artifact export -m manifest-sample.yaml -o kubesphere.tar.gz

方式二：

依次运行以下命令：

export KKZONE=cn./kk artifact export -m manifest-sample.yaml -o kubesphere.tar.gz

备注

制品（artifact）是一个根据指定的 manifest 文件内容导出的包含镜像 tar 包和相关二进制文件的 tgz 包。在 KubeKey 初始化镜像仓库、创建集群、添加节点和升级集群的命令中均可指定一个 artifact，KubeKey 将自动解包该 artifact 并在执行命令时直接使用解包出来的文件。

  • 导出时请确保网络连接正常。

  • KubeKey 会解析镜像列表中的镜像名，若镜像名中的镜像仓库需要鉴权信息，可在 manifest 文件中的 .registry.auths 字段中进行配置。

离线安装集群

复制KubeKey 和制品 artifact到离线机器

将下载的 KubeKey 和制品 artifact 通过 U 盘等介质拷贝至离线环境安装节点。

创建初始换、安装配置文件

执行以下命令创建离线集群配置文件： 

./kk create config --with-kubesphere v3.4.1 --with-kubernetes v1.23.15 -f config-sample.yaml

执行以下命令修改配置文件：

vim config-sample.yaml

备注

  • 按照实际离线环境配置修改节点信息。
  • 必须指定 registry 仓库部署节点（用于 KubeKey 部署自建 Harbor 仓库）。
  • registry 里必须指定 type 类型为 harbor，否则默认安装 docker registry。

 安装镜像仓库harbor

执行以下命令安装镜像仓库

  1. ./kk init registry -f config-sample.yaml -a kubesphere.tar.gz

    备注

    命令中的参数解释如下：

    • config-sample.yaml 指离线环境集群的配置文件。

    • kubesphere.tar.gz 指源集群打包出来的 tar 包镜像。

个人备注

在执行初始换安装 harbor时报错

11:16:46 UTC success: [rs-node-178-02]
11:16:46 UTC success: [rs-node-177-01]
11:16:46 UTC success: [rs-master-174-01]
11:16:46 UTC success: [rs-node-179-03]
11:16:46 UTC success: [rs-master-175-02]
11:16:46 UTC success: [rs-master-176-03]
11:16:46 UTC success: [devops-180]
11:16:46 UTC [ConfigureOSModule] configure the ntp server for each node
11:16:46 UTC skipped: [rs-node-179-03]
11:16:46 UTC skipped: [rs-master-174-01]
11:16:46 UTC skipped: [rs-master-175-02]
11:16:46 UTC skipped: [rs-master-176-03]
11:16:46 UTC skipped: [devops-180]
11:16:46 UTC skipped: [rs-node-177-01]
11:16:46 UTC skipped: [rs-node-178-02]
11:16:46 UTC [InitRegistryModule] Fetch registry certs
11:16:46 UTC success: [devops-180]
11:16:46 UTC [InitRegistryModule] Generate registry Certs
[certs] Using existing ca certificate authority
11:16:46 UTC message: [LocalHost]
unable to sign certificate: must specify a CommonName
11:16:46 UTC failed: [LocalHost]
error: Pipeline[InitRegistryPipeline] execute failed: Module[InitRegistryModule] exec failed: 
failed: [LocalHost] [GenerateRegistryCerts] exec failed after 1 retries: unable to sign certificate: must specify a CommonName

解决方案：

https://ask.kubesphere.io/forum/d/22879-kubesphere34-unable-to-sign-certificate-must-specify-a-commonname

修改配置文件 

...  
registry:type: harborauths:"dockerhub.kubekey.local":username: adminpassword: Harbor12345privateRegistry: "dockerhub.kubekey.local"namespaceOverride: ""registryMirrors: []insecureRegistries: []addons: []

然后重新执行即可

 执行安装后去harbor对应服务器检查harbor启动情况，

如果有部分模块启动失败

进入  /opt/harbor  目录

chmod 777 -R ./common

并重启harbor

docker-compose down -vdocker-compose u -d

harbor启动后可以浏览器访问

初始化harbor 项目

备注

由于 Harbor 项目存在访问控制（RBAC）的限制，即只有指定角色的用户才能执行某些操作。如果您未创建项目，则镜像不能被推送到 Harbor。Harbor 中有两种类型的项目：

  • 公共项目（Public）：任何用户都可以从这个项目中拉取镜像。
  • 私有项目（Private）：只有作为项目成员的用户可以拉取镜像。

Harbor 管理员账号：admin，密码：Harbor12345。Harbor 安装文件在 /opt/harbor , 如需运维 Harbor，可至该目录下。

方式一： 

执行脚本创建 Harbor 项目。

a. 执行以下命令下载指定脚本初始化 Harbor 仓库：

curl -O https://raw.githubusercontent.com/kubesphere/ks-installer/master/scripts/create_project_harbor.sh

b. 执行以下命令修改脚本配置文件：

vim create_project_harbor.sh

修改成

#!/usr/bin/env bash# Copyright 2018 The KubeSphere Authors.## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at##     http://www.apache.org/licenses/LICENSE-2.0## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.url="https://dockerhub.kubekey.local"  #修改url的值为https://dockerhub.kubekey.localuser="admin"passwd="Harbor12345"harbor_projects=(librarykubesphereiokubesphereargoprojcalicocorednsopenebscsipluginminiomirrorgooglecontainersosixiapromthanosiojimmidysongrafanaelasticistiojaegertracingjenkinsweaveworksopenpitrixjoosthofmannginxdemosfluentkubeedgeopenpolicyagent)for project in "${harbor_projects[@]}"; doecho "creating $project"curl -u "${user}:${passwd}" -X POST -H "Content-Type: application/json" "${url}/api/v2.0/projects" -d "{ \"project_name\": \"${project}\", \"public\": true}" -k #curl命令末尾加上 -kdone

备注

  • 修改 url 的值为 https://dockerhub.kubekey.local。

  • 需要指定仓库项目名称和镜像列表的项目名称保持一致。

  • 脚本末尾 curl 命令末尾加上 -k。

c. 执行以下命令创建 Harbor 项目：

chmod +x create_project_harbor.sh
./create_project_harbor.sh

 方式二：

登录 Harbor 仓库创建项目。将项目设置为公开以便所有用户都能够拉取镜像。关于如何创建项目，请参阅创建项目。

修改配置文件

再次执行以下命令修改集群配置文件：

vim config-sample.yaml

  ...registry:type: harborauths:"dockerhub.kubekey.local":username: adminpassword: Harbor12345privateRegistry: "dockerhub.kubekey.local"namespaceOverride: "kubesphereio"registryMirrors: []insecureRegistries: []addons: []

备注

  • 新增 auths 配置增加 dockerhub.kubekey.local 和账号密码。
  • privateRegistry 增加 dockerhub.kubekey.local。
  • namespaceOverride 增加 kubesphereio。

安装k8s集群和kubesphere

 执行以下命令安装 KubeSphere 集群:

./kk create cluster -f config-sample.yaml -a kubesphere.tar.gz --with-packages

参数解释如下：

  • config-sample.yaml：离线环境集群的配置文件。
  • kubesphere.tar.gz：源集群打包出来的 tar 包镜像。
  • --with-packages：若需要安装操作系统依赖，需指定该选项。

执行以下命令查看集群状态：

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f

执行命令会看到以下提示：

[root@k8s-master kubekey]# ./kk create cluster -f config-sample.yaml -a kubesphere.tar.gz --with-packages_   __      _          _   __           
| | / /     | |        | | / /           
| |/ / _   _| |__   ___| |/ /  ___ _   _ 
|    \| | | | '_ \ / _ \    \ / _ \ | | |
| |\  \ |_| | |_) |  __/ |\  \  __/ |_| |
\_| \_/\__,_|_.__/ \___\_| \_/\___|\__, |__/ ||___/11:07:36 CST [GreetingsModule] Greetings
11:07:37 CST message: [k8s-master]
Greetings, KubeKey!
11:07:37 CST message: [k8s-node]
Greetings, KubeKey!
11:07:37 CST success: [k8s-master]
11:07:37 CST success: [k8s-node]
11:07:37 CST [NodePreCheckModule] A pre-check on nodes
11:07:44 CST success: [k8s-master]
11:07:44 CST success: [k8s-node]
11:07:44 CST [ConfirmModule] Display confirmation form
+------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| name       | sudo | curl | openssl | ebtables | socat | ipset | ipvsadm | conntrack | chrony | docker | containerd | nfs client | ceph client | glusterfs client | time         |
+------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| k8s-node   | y    | y    | y       | y        |       | y     |         |           | y      | 24.0.6 | v1.7.3     | y          |             |                  | CST 11:07:44 |
| k8s-master | y    | y    | y       | y        |       | y     |         |           | y      |        | y          | y          |             |                  | CST 11:07:43 |
+------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+This is a simple check of your environment.
Before installation, ensure that your machines meet all requirements specified at
https://github.com/kubesphere/kubekey#requirements-and-recommendationsContinue this installation? [yes/no]: no
手动安装依赖包

然后需要手动离线安装 

socat ipvsadm conntrack ceph client glusterfs client

离线安装依赖包方式参考下文

yum 离线安装 yumdownloader

并确保满足 https://github.com/kubesphere/kubekey#requirements-and-recommendations  中的安装条件

检查环境是否满足要求

  • Minimum resource requirements (For Minimal Installation of KubeSphere only)：
    • 2 vCPUs
    • 4 GB RAM
    • 20 GB Storage

/var/lib/docker is mainly used to store the container data, and will gradually increase in size during use and operation. In the case of a production environment, it is recommended that /var/lib/docker mounts a drive separately.

  • OS requirements:
    • SSH can access to all nodes.
    • Time synchronization for all nodes.
    • sudo/curl/openssl should be used in all nodes.
    • docker can be installed by yourself or by KubeKey.
    • Red Hat includes SELinux in its Linux release. It is recommended to close SELinux or switch the mode of SELinux to Permissive
  • It's recommended that Your OS is clean (without any other software installed), otherwise there may be conflicts.
  • A container image mirror (accelerator) is recommended to be prepared if you have trouble downloading images from dockerhub.io. Configure registry-mirrors for the Docker daemon.
  • KubeKey will install OpenEBS to provision LocalPV for development and testing environment by default, this is convenient for new users. For production, please use NFS / Ceph / GlusterFS or commercial products as persistent storage, and install the relevant client in all nodes.
  • If you encounter Permission denied when copying, it is recommended to check SELinux and turn off it first
  • Dependency requirements:

KubeKey can install Kubernetes and KubeSphere together. Some dependencies need to be installed before installing kubernetes after version 1.18. You can refer to the list below to check and install the relevant dependencies on your node in advance.

Kubernetes Version ≥ 1.18
socatRequired
conntrackRequired
ebtablesOptional but recommended
ipsetOptional but recommended
ipvsadmOptional but recommended
  • Networking and DNS requirements:
    • Make sure the DNS address in /etc/resolv.conf is available. Otherwise, it may cause some issues of DNS in cluster.
    • If your network configuration uses Firewall or Security Group，you must ensure infrastructure components can communicate with each other through specific ports. It's recommended that you turn off the firewall or follow the link configuriation: NetworkAccess.
检查防火墙端口是否满足要求

端口开放满足 https://github.com/kubesphere/kubekey/blob/master/docs/network-access.md

If your network configuration uses an firewall，you must ensure infrastructure components can communicate with each other through specific ports that act as communication endpoints for certain processes or services.

servicesprotocolactionstart portend portcomment
sshTCPallow22
etcdTCPallow23792380
apiserverTCPallow6443
calicoTCPallow90999100
bgpTCPallow179
nodeportTCPallow3000032767
masterTCPallow1025010258
dnsTCPallow53
dnsUDPallow53
local-registryTCPallow5000offline environment
local-aptTCPallow5080offline environment
rpcbindTCPallow111use NFS
ipipIPENCAP / IPIPallowcalico needs to allow the ipip protocol

安装完成后，您会看到以下内容：

**************************************************########################################################              Welcome to KubeSphere!           ########################################################Console: http://192.168.0.3:30880Account: adminPassword: P@88w0rdNOTES：1. After you log into the console, please check themonitoring status of service components inthe "Cluster Management". If any service is notready, please wait patiently until all componentsare up and running.1. Please change the default password after login.#####################################################https://kubesphere.io             2022-02-28 23:30:06#####################################################
安装完成并登录

通过 http://{IP}:30880 使用默认帐户和密码 admin/P@88w0rd 访问 KubeSphere 的 Web 控制台。

安装过程中遇到的问题

制作制品，不能下载github上的操作系统iso

  在这一步我遇到了操作系统配置在githab上，然后下载不下来的情况

然后我去github上手动下载下来放到了本地，然后下盖下面的配置部分

 operatingSystems:- arch: amd64type: linuxid: centosversion: "7"repository:iso:localPath: 添加你的本地地址url: - arch: amd64type: linuxid: ubuntuversion: "20.04"repository:iso:localPath: 添加你的本地地址url:

下载地址

 https://github.com/kubesphere/kubekey/releases/tag/v3.0.10

初始化安装harbor报错must specify a CommonName

11:16:46 UTC success: [rs-node-178-02]
11:16:46 UTC success: [rs-node-177-01]
11:16:46 UTC success: [rs-master-174-01]
11:16:46 UTC success: [rs-node-179-03]
11:16:46 UTC success: [rs-master-175-02]
11:16:46 UTC success: [rs-master-176-03]
11:16:46 UTC success: [devops-180]
11:16:46 UTC [ConfigureOSModule] configure the ntp server for each node
11:16:46 UTC skipped: [rs-node-179-03]
11:16:46 UTC skipped: [rs-master-174-01]
11:16:46 UTC skipped: [rs-master-175-02]
11:16:46 UTC skipped: [rs-master-176-03]
11:16:46 UTC skipped: [devops-180]
11:16:46 UTC skipped: [rs-node-177-01]
11:16:46 UTC skipped: [rs-node-178-02]
11:16:46 UTC [InitRegistryModule] Fetch registry certs
11:16:46 UTC success: [devops-180]
11:16:46 UTC [InitRegistryModule] Generate registry Certs
[certs] Using existing ca certificate authority
11:16:46 UTC message: [LocalHost]
unable to sign certificate: must specify a CommonName
11:16:46 UTC failed: [LocalHost]
error: Pipeline[InitRegistryPipeline] execute failed: Module[InitRegistryModule] exec failed: 
failed: [LocalHost] [GenerateRegistryCerts] exec failed after 1 retries: unable to sign certificate: must specify a CommonName

解决方案：

https://ask.kubesphere.io/forum/d/22879-kubesphere34-unable-to-sign-certificate-must-specify-a-commonname

修改配置文件 

...  
registry:type: harborauths:"dockerhub.kubekey.local":username: adminpassword: Harbor12345privateRegistry: "dockerhub.kubekey.local"namespaceOverride: ""registryMirrors: []insecureRegistries: []addons: []

然后重新执行即可

初始化harbor后，发现harbor的一些模块容器没有正常启动

 执行安装后去harbor对应服务器检查harbor启动情况，

如果有部分模块容器启动失败

进入  /opt/harbor  目录

chmod 777 -R ./common

并重启harbor

docker-compose down -vdocker-compose u -d

harbor启动后可以浏览器访问

麒麟系统安装，安装包没有 Fkylin-v10-amd64.iso

关于##kylin##上安装KubeSphere的问题，如何解决？ - KubeSphere 开发者社区

离线安装却在线下载calicoctl

离线安装 kubesphere v3.4.1 报错Failed to download calicoctl binary - KubeSphere 开发者社区

还有几个镜像也是拉取的3.26.1

  - registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.23.2- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.23.2- registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.23.2- registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.23.2- registry.cn-beijing.aliyuncs.com/kubesphereio/typha:v3.23.2

 修改成

  - registry.cn-beijing.aliyuncs.com/kubesphereio/cni:v3.26.1- registry.cn-beijing.aliyuncs.com/kubesphereio/kube-controllers:v3.26.1- registry.cn-beijing.aliyuncs.com/kubesphereio/node:v3.26.1- registry.cn-beijing.aliyuncs.com/kubesphereio/pod2daemon-flexvol:v3.26.1- registry.cn-beijing.aliyuncs.com/kubesphereio/typha:v3.26.1

重新制作制品

或者自己在有网的服务器上

docker pull下载这几个镜像

 然后 docker tag 重命名镜像

docker save  保存镜像到本地，并传到没有网的服务器

docker load  加载本地镜像到没有网的服务器

docker push  上传到harbor仓库

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.rhkb.cn/news/305530.html

如若内容造成侵权/违法违规/事实不符，请联系长河编程网进行投诉反馈email:809451989@qq.com，一经查实，立即删除！

相关文章

ios包上架系列 一、打包机Flutter项目环境配置

ios包上架系列 一、打包机Flutter项目环境配置

打包的时候一定要断开网络，上线包名只能在打包机配置 1、Xcode 需要从其它电脑空投 版本号：14.3.1 升级到Xcode14.3后发现,从这个版本开始,苹果从Xcode中移除了ARC相关的库,会导致fluter项目下的原生工程使用Xcode编译原生代码没 有问题, 但是flutter项…
阅读更多...
Python(1)：认识Python并且了解一些简单函数

Python(1)：认识Python并且了解一些简单函数

文章目录 一、Python的优势及其使用场景二、Python环境的安装三、Python中的变量及其命名四、Python中的注释五、一些简单常见的函数和认识ASCII表六、Python导入模块的方式 一、Python的优势及其使用场景 优点： 开发效率高：Python具有非常强大的第三方…
阅读更多...
MySQL 全文检索

MySQL 全文检索

不是所有的数据表都支持全文检索 MySQL支持多种底层数据库引擎，但是并非所有的引擎支持全文检索 ，目前最常用引擎是是MyISAM和InnoDB；前者支持全文检索，后者不支持。 booolean模式操作符 操作符含义必须有-必须不包含>包含对应…
阅读更多...
《战神4》和《战神5》有什么联系吗 苹果电脑如何运行《战神4》苹果电脑玩战神 Mac玩游戏 战神5攻略 crossover激活码

《战神4》和《战神5》有什么联系吗 苹果电脑如何运行《战神4》苹果电脑玩战神 Mac玩游戏 战神5攻略 crossover激活码

《战神4》（God of War 2018）和《战神5》（God of War: Ragnark）是一对引人注目的游戏作品，它们不仅在游戏界引起了广泛的关注，也给玩家带来了深入探索北欧神话世界的机会。这两部游戏之间的联系不仅体现在剧…
阅读更多...
【面试八股总结】排序算法（一）

【面试八股总结】排序算法（一）

参考资料 ：阿秀 一、冒泡排序 冒泡排序就是把小的元素往前交换或者把大的元素往后交换，比较相邻的两个元素，交换也发生在这两个元素之间。具体步骤： 比较相邻的元素。如果第一个比第二个大，就交换他们两个。对每一对…
阅读更多...
spring快速搭建聊天AI

spring快速搭建聊天AI

官网url: https://spring.io/projects/spring-ai 本文演示的是open AI 1创建java项目 2.拿到AI的key（没有的话可以去淘宝花几块钱买一个） //YOUR_API_KEY写你自己的open AI的key spring.ai.openai.api-keyYOUR_API_KEY spring.ai.openai.chat.options.…
阅读更多...
学习云计算HCIE选择誉天有什么优势？

学习云计算HCIE选择誉天有什么优势？

誉天云计算课程优势实战性强 课程注重实践操作，通过实际案例和实验操作，让学员深入了解云计算的应用场景和实际操作技能。课程内容全面 涵盖所有云计算涉及的IT基础知识、服务器、存储、网络等方面的基础知识，开源操作系统Linux，开…
阅读更多...
反序列化漏洞笔记

反序列化漏洞笔记

1 PHP 序列化基础概念 1.1 什么是序列化 序列化可以实现将对象压缩并格式化，方便数据的传输和存储。 为什么要序列化？ PHP 文件在执行结束时会把对象销毁，如果下次要引用这个对象的话就很麻烦，所以就有了对象序列化&#xff0…
阅读更多...
git 删除本地分支 删除远程仓库中的分支

git 删除本地分支 删除远程仓库中的分支

语法&#xff1a; 删除本地分支 git branch -D <分支名>删除远程分支 git push <remote名称> <分支名> --delete 示例&#xff1a; 删除本地分支 git branch -D feature/test_listview删除远程分支 git push origin feature/test_listview --delete 两个…
阅读更多...
Day36｜贪心算法part05：435. 无重叠区间、763.划分字母区间、56. 合并区间

Day36｜贪心算法part05：435. 无重叠区间、763.划分字母区间、56. 合并区间

435. 无重叠区间 有了上题射气球的因子&#xff0c;这题也就有思路了&#xff0c;反正无脑排序就行了&#xff1a; 首先将所有区间按照end的大小从小到大排序&#xff1b;选取最早end为起始x_end遍历所有区间&#xff0c;如果该区间的start比end大&#xff08;可重叠&#xf…
阅读更多...
利用Python实现可视化交互界面：Dash

利用Python实现可视化交互界面：Dash

Dash是一个低代码数据框架&#xff0c;用Python实现可视化交互界面&#xff0c;不用写Javascript&#xff0c;开源&#xff0c;支持回调、HTML组件等功能。 安装 pip install dash使用 # Import packages from dash import Dash, html, dash_table, dcc, callback, Output, …
阅读更多...
基于 WebRTC 实现的点对点文件传输和音视频聊天工具 | 开源日报 No.220

基于 WebRTC 实现的点对点文件传输和音视频聊天工具 | 开源日报 No.220

tl-open-source/tl-rtc-file Stars: 2.1k License: MIT tl-rtc-file 是一个基于 WebRTC 的文件传输工具&#xff0c;支持跨终端、不限平台的在线文件传输。它提供了丰富的功能和特性&#xff1a; 分片传输&#xff1a;支持大型文件的分片传输&#xff0c;确保高效稳定地完成上…
阅读更多...
使用htmlentities()和nl2br()将文本数据正确显示到前台

使用htmlentities()和nl2br()将文本数据正确显示到前台

问题&#xff1a; 在后台textarea里编辑了有一串字符串&#xff0c;虽然在textarea里编辑是有换行效果的&#xff0c;但是数据获取到就只是\n&#xff0c;前端是不认识这个的&#xff0c;正确输出到前台的换行只能是<br/>。 $str "ABCDEFGHIJKLMNOPQ"; echo…
阅读更多...
【opencv】示例-fback.cpp 使用OpenCV库来实现密集光流算法

【opencv】示例-fback.cpp 使用OpenCV库来实现密集光流算法

// 引入OpenCV库中有关视频跟踪的头文件 #include "opencv2/video/tracking.hpp" // 引入OpenCV库中有关图像处理的头文件 #include "opencv2/imgproc.hpp" // 引入OpenCV库中有关视频输入的头文件 #include "opencv2/videoio.hpp" // 引入OpenC…
阅读更多...
DVWA -XSS（Reflected）-通关教程-完结

DVWA -XSS（Reflected）-通关教程-完结

DVWA -XSS&#xff08;Reflected&#xff09;-通关教程-完结 XSS&#xff08;Reflected&#xff09; ​ XSS 攻击全称跨站脚本攻击。是指用户在 Web 页面中提交恶意脚本&#xff0c;从而使浏览包含恶意脚本的页面的用户在不知情的情况下执行该脚本&#xff0c;导致被攻击的行为…
阅读更多...
Elasticsearch部署安装

Elasticsearch部署安装

环境准备 Anolis OS 8 Firewall关闭状态&#xff0c;端口自行处理 Elasticsearch&#xff1a;7.16.1&#xff08;该版本需要jdk11&#xff09; JDK&#xff1a;11.0.19 JDK # 解压 tar -zxvf jdk-11.0.19_linux-x64_bin.tar.gz# 编辑/etc/profile vim /etc/profile# 加入如下…
阅读更多...
动态规划-入门三道题

动态规划-入门三道题

1137. 第 N 个泰波那契数 题目描述&#xff1a; 状态表示: dp[i]表示第i个泰波那契数。 状态转移方程&#xff1a; dp[i]dp[i-3]dp[i-2]dp[i-1]。 初始化: 动态规划问题的初始化就是为了去避免初始情况下的越界问题。这里就对dp[0]0,dp[1]1,dp[2]1这样进行初始化即可&#xf…
阅读更多...
基于Vue的宠物领养系统的设计与实现(论文+源码)_kaic

基于Vue的宠物领养系统的设计与实现(论文+源码)_kaic

目 录 摘 要 ABSTRACT 1 引言 1.1 课题背景 1.2 设计原则 1.3 论文组织结构 2 系统关键技术 2.1 JSP技术 2.2 JAVA技术 2.3 B/S结构 2.4 MYSQL数据库 3 系统分析 3.1 可行性分析 3.1.1 操作可行性 3.1.2 经济可行性 3.1.3 技术可行性 3.1.4 法律可行性 3.2 系统功能分析 3.3…
阅读更多...
搭建PyTorch神经网络进行气温预测（手写+调包两种方法）(保证学会！)+找到神经网络的最优情况

搭建PyTorch神经网络进行气温预测（手写+调包两种方法）(保证学会！)+找到神经网络的最优情况

代码上有注释&#xff01;&#xff01;&#xff01;&#xff01;&#xff01;&#xff01; 本篇主要包括三大部分&#xff1a; 第一部分&#xff1a;导入数据集导入第三方库数据集简单介绍与可视化数据集简单预处理 第二部分&#xff1a;手写神经网络代码实现气温预测&#…
阅读更多...
【高端电流检测IC储能产品应用方案】耐压28V侧轨的电流检测芯片FP130A 应用于电脑电源，开关电源以及多口快充充电器，户外移动电源，适配器，电池充电器等

【高端电流检测IC储能产品应用方案】耐压28V侧轨的电流检测芯片FP130A 应用于电脑电源，开关电源以及多口快充充电器，户外移动电源，适配器，电池充电器等

电流检测技术常用于高压短路保护、电机控制、DC/DC换流器、系统功耗管理、二次电池的电流管理、蓄电池管理等电流侦测等场景。对于大多数应用而言&#xff0c;都是间接测量电阻两端的跨压差来获取待测电流。 如下面的高端电流检测芯片FP130A&#xff0c;丝印是FC915。电路原理图…
阅读更多...
最新文章
推荐文章

Copyright @ 2022~2023