1. 轻量级容器管理工具 Containerd
2. Containerd的两种安装方式
3. Containerd容器镜像管理
4. Containerd数据持久化和网络管理
1、Containerd镜像管理
1.1 Containerd容器镜像管理命令
- docker使用
docker images
命令管理镜像 - 单机containerd使用
ctr images
命令管理镜像,containerd本身的CLI - k8s中containerd使用
crictl images
命令管理镜像,Kubernetes社区的专用CLI工具
获取命令帮助
# ctr --help
NAME:ctr -_______/ /______/ ___/ __/ ___/
/ /__/ /_/ /
\___/\__/_/......
获取命令帮助
# ctr images
NAME:ctr images - manage imagesUSAGE:ctr images command [command options] [arguments...]COMMANDS:check check existing images to ensure all content is available locallyexport export imagesimport import imageslist, ls list images known to containerdmount mount an image to a target pathunmount unmount the image from the targetpull pull an image from a remotepush push an image to a remotedelete, del, remove, rm remove one or more images by referencetag tag an imagelabel set and clear labels for an imageconvert convert an imageOPTIONS:--help, -h show help
1.2 查看镜像
# ctr images ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
1.3 下载镜像
containerd支持oci标准的镜像,所以可以直接使用docker官方或dockerfile构建的镜像
# 镜像名不能简写,--all-platforms:所有平台,不加的话下载当前平台架构
# ctr images pull --all-platforms docker.io/library/nginx:alpine
# ctr images pull docker.io/library/nginx:latest# uname -a
说明:
这里ctr命令pull镜像时,不能直接把镜像名字写成`nginx:alpine`
查看已下载容器镜像
# ctr images ls
REF | TYPE | DIGEST |
---|---|---|
docker.io/library/nginx:alpine | application/vnd.docker.distribution.manifest.list.v2+json | sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 |
SIZE | PLATFORMS | LABELS |
---|---|---|
9.7 MiB | linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x | - |
指定平台下载容器镜像
# ctr images pull --platform linux/amd64 docker.io/library/nginx:alpine
没有镜像是无法运行容器的。
1.4 镜像挂载
方便查看镜像中包含的内容。
把已下载的容器镜像挂载至当前文件系统
# ctr images mount docker.io/library/nginx:alpine /mnt
sha256:af2fcce448e2e4451a5f4796a9bf9cb5c9b5f88e0d6d10029cada42fb9d268ac
/mnt
[root@localhost ~]# ls /mnt
bin dev docker-entrypoint.d docker-entrypoint.sh etc home lib media mnt opt proc root run sbin srv sys tmp usr var
卸载
# umount /mnt
1.5 镜像导出
把容器镜像导出
# ctr i export --all-platforms nginx.img docker.io/library/nginx:alpine
说明
--all-platforms,导出所有平台镜像,本版本为1.6版本,1.4版本不需要添加此选项。
查看已导出容器镜像
# ls
nginx.img# ls -lh
总用量 196M-rw-r--r-- 1 root root 73M 2月 18 14:48 nginx.img
1.6 镜像删除
删除指定容器镜像
# ctr image rm docker.io/library/nginx:alpine
docker.io/library/nginx:alpine再次查看容器镜像
[root@192 ~]# ctr images ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
1.7 镜像导入
导入容器镜像
# ctr images import nginx.img
unpacking docker.io/library/nginx:alpine (sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3)...done
1.8 修改镜像tag
# ctr images tag docker.io/library/nginx:alpine nginx:alpine
nginx:alpine# 建议修改tag的名称为:镜像仓库地址/目录/镜像名
说明:
把docker.io/library/nginx:alpine 修改为 nginx:alpine
查看修改后的容器镜像
# ctr images ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/library/nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 9.7 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -
nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 9.7 MiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -
修改后对容器镜像做检查比对
# ctr images check
REF TYPE DIGEST STATUS SIZE UNPACKED
docker.io/library/nginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 complete (7/7) 9.7 MiB/9.7 MiB truenginx:alpine application/vnd.docker.distribution.manifest.list.v2+json sha256:da9c94bec1da829ebd52431a84502ec471c8e548ffb2cedbf36260fd9bd1d4d3 complete (7/7) 9.7 MiB/9.7 MiB true
2、Containerd容器管理
2.1 获取命令帮助
2.1.1 获取ctr命令帮助
[root@localhost ~]# ctr --help
NAME:ctr -_______/ /______/ ___/ __/ ___/
/ /__/ /_/ /
\___/\__/_/containerd CLI......
2.1.2 获取创建静态容器命令帮助
# ctr container --help
NAME:ctr containers - manage containersUSAGE:ctr containers command [command options] [arguments...]COMMANDS:create create containerdelete, del, remove, rm delete one or more existing containersinfo get info about a containerlist, ls list containerslabel set and clear labels for a containercheckpoint checkpoint a containerrestore restore a container from checkpointOPTIONS:--help, -h show help
说明:使用`ctr container create `命令创建容器后,容器并没有处于运行状态,其只是一个静态的容器。这个 container 对象只是包含了运行一个容器所需的资源及配置的数据结构,例如: namespaces、rootfs 和容器的配置都已经初始化成功了,只是用户进程(本案例为nginx)还没有启动。需要使用`ctr tasks`命令才能获取一个动态容器。
2.1.3 获取动态容器命令帮助
# ctr run --help
NAME:ctr run - run a container......
说明:
使用`ctr run`命令可以创建一个静态容器并使其运行。一步到位运行容器。
2.2 查看容器
container
表示静态容器,可用c
缩写代表container
# ctr container ls
CONTAINER IMAGE RUNTIME
或
# ctr c ls
CONTAINER IMAGE RUNTIME
以上命令无法查看出来容器的静态的还是动态的
2.3 查看任务
task表示容器里跑的进程, 可用t
缩写代表task
# ctr task ls
TASK PID STATUS
或
# ctr t ls
TASK PID STATUS
2.4 创建静态容器
# ctr c create docker.io/library/nginx:alpine nginx1
# ctr container ls
CONTAINER IMAGE RUNTIME
nginx1 docker.io/library/nginx:alpine io.containerd.runc.v2
查看容器详细信息
# ctr container info nginx1
2.5 静态容器启动为动态容器
复制containerd连接runC垫片工具至系统
# ls usr/local/bin/
containerd containerd-shim containerd-shim-runc-v1 containerd-shim-runc-v2 containerd-stress crictl critest ctd-decoder ctr
[root@localhost ~]# cp usr/local/bin/containerd-shim-runc-v2 /usr/bin/
启动task,即表时在容器中运行了进程,即为动态容器。
# ctr task start -d nginx1
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
说明:
-d表示daemon或者后台的意思,否则会卡住终端
查看容器所在宿主机进程,是以宿主机进程的方式存在的。
# ctr task ls
TASK PID STATUS
nginx1 3395 RUNNING
查看容器的进程(都是物理机的进程)
# ctr task ps nginx1
PID INFO
3395 -
3434 -
物理机查看到相应的进程
# ps -ef | grep 3395
root 3395 3375 0 19:16 ? 00:00:00 nginx: master process nginx -g daemon off;
101 3434 3395 0 19:16 ? 00:00:00 nginx: worker process
2.6 进入容器操作
# ctr task exec --exec-id 1 nginx1 /bin/sh# ifconfig 查看网卡信息...# curl 127.0.0.1 访问本地提供的web服务
...
说明:
为exec进程设定一个id,可以随意输入,只要保证唯一即可,也可使用$RANDOM变量。
2.7 直接运行一个动态容器
# ctr run -d --net-host docker.io/library/nginx:alpine nginx2
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
说明:
* -d 代表dameon,后台运行
* --net-host 代表容器的IP就是宿主机的IP(相当于docker里的host类型网络)
查看已运行容器
# ctr container ls
CONTAINER IMAGE RUNTIME
nginx2 docker.io/library/nginx:alpine io.containerd.runc.v2
查看已运行容器中运行的进程,既tasks
# ctr tasks ls
TASK PID STATUS
nginx2 4061 RUNNING
进入容器
# ctr task exec --exec-id 1 -t nginx2 /bin/sh
/ # ifconfig
ens33 Link encap:Ethernet HWaddr 00:0C:29:B1:B6:1Dinet addr:192.168.10.164 Bcast:192.168.10.255 Mask:255.255.255.0inet6 addr: fe80::2b33:40ed:9311:8812/64 Scope:Linkinet6 addr: fe80::adf4:a8bc:a1c:a9f7/64 Scope:LinkUP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1RX packets:55360 errors:0 dropped:0 overruns:0 frame:0TX packets:30526 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:1000RX bytes:53511295 (51.0 MiB) TX bytes:2735050 (2.6 MiB)lo Link encap:Local Loopbackinet addr:127.0.0.1 Mask:255.0.0.0inet6 addr: ::1/128 Scope:HostUP LOOPBACK RUNNING MTU:65536 Metric:1RX packets:68 errors:0 dropped:0 overruns:0 frame:0TX packets:68 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:1000RX bytes:5916 (5.7 KiB) TX bytes:5916 (5.7 KiB)virbr0 Link encap:Ethernet HWaddr 52:54:00:E9:51:82inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0UP BROADCAST MULTICAST MTU:1500 Metric:1RX packets:0 errors:0 dropped:0 overruns:0 frame:0TX packets:0 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:1000RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
为容器中运行的网站添加网站文件
/ # echo "nginx2" > /usr/share/nginx/html/index.html
/ # exit
在宿主机上访问网站
[root@localhost ~]# curl 192.168.10.164
nginx2
2.8 暂停容器
查看容器状态
# ctr tasks ls
TASK PID STATUS
nginx2 4061 RUNNING
暂停容器
# ctr tasks pause nginx2
再次查看容器状态,看到其状态为PAUSED,表示停止。
# ctr tasks ls
TASK PID STATUS
nginx2 4061 PAUSED
[root@localhost ~]# curl http://192.168.10.164
在宿主机访问,发现不可以访问到网站
2.9 恢复容器
使用resume命令恢复容器
# ctr tasks resume nginx2
查看恢复后状态
# ctr tasks ls
TASK PID STATUS
nginx2 4061 RUNNING
在宿主机上访问容器中提供的服务
# curl http://192.168.10.164
nginx2
2.10 停止容器
# ctr tasks --help
......
使用kill命令停止容器中运行的进程,既为停止容器
# ctr tasks kill nginx2
查看容器停止后状态,STATUS为STOPPED
# ctr tasks ls
TASK PID STATUS
nginx1 3395 RUNNING
nginx2 4061 STOPPED
2.11 删除容器
# ctr tasks delete nginx2
必须先删除task,再删除容器
查看静态容器,确认其还存在于系统中
# ctr container ls
CONTAINER IMAGE RUNTIME
nginx2 docker.io/library/nginx:alpine io.containerd.runc.v2
删除容器
# ctr container delete nginx2
3、Containerd使用私有容器镜像仓库 Harbor
3.1 Harbor准备
3.2 配置Containerd使用Harbor仓库
3.2.1 Harbor主机名解析
在所有安装containerd宿主机上添加此配置信息。
# vim /etc/hosts
# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.10.165 harbor.kubemsb.com
说明
* 192.168.10.165是harbor的IP* harbor.kubemsb.com建议用FQDN形式,如果用类似harbor这种短名,后面下载镜像会出问题
3.2.2 修改Containerd配置文件
此配置文件已提前替换过,仅修改本地容器镜像仓库地址即可。
# vim /etc/containerd/config.toml
# cat /etc/containerd/config.toml
root = "/var/lib/containerd"
state = "/run/containerd"
oom_score = -999[grpc]address = "/run/containerd/containerd.sock"uid = 0gid = 0max_recv_message_size = 16777216max_send_message_size = 16777216[debug]address = ""uid = 0gid = 0level = ""[metrics]address = ""grpc_histogram = false[cgroup]path = ""[plugins][plugins.cgroups]no_prometheus = false[plugins.cri]stream_server_address = "127.0.0.1"stream_server_port = "0"enable_selinux = falsesandbox_image = "easzlab/pause-amd64:3.2"stats_collect_period = 10systemd_cgroup = falseenable_tls_streaming = falsemax_container_log_line_size = 16384[plugins.cri.containerd]snapshotter = "overlayfs"no_pivot = false[plugins.cri.containerd.default_runtime]runtime_type = "io.containerd.runtime.v1.linux"runtime_engine = ""runtime_root = ""[plugins.cri.containerd.untrusted_workload_runtime]runtime_type = ""runtime_engine = ""runtime_root = ""[plugins.cri.cni]bin_dir = "/opt/kube/bin"conf_dir = "/etc/cni/net.d"conf_template = "/etc/cni/net.d/10-default.conf"[plugins.cri.registry][plugins.cri.registry.mirrors][plugins.cri.registry.mirrors."docker.io"]endpoint = ["https://docker.mirrors.ustc.edu.cn","http://hub-mirror.c.163.com"][plugins.cri.registry.mirrors."gcr.io"]endpoint = ["https://gcr.mirrors.ustc.edu.cn"][plugins.cri.registry.mirrors."k8s.gcr.io"]endpoint = ["https://gcr.mirrors.ustc.edu.cn/google-containers/"][plugins.cri.registry.mirrors."quay.io"]endpoint = ["https://quay.mirrors.ustc.edu.cn"][plugins.cri.registry.mirrors."harbor.kubemsb.com"] 在此处添加,在镜像加速器下面添加这一段endpoint = ["http://harbor.kubemsb.com"][plugins.cri.x509_key_pair_streaming]tls_cert_file = ""tls_key_file = ""[plugins.diff-service]default = ["walking"][plugins.linux]shim = "containerd-shim"runtime = "runc"runtime_root = ""no_shim = falseshim_debug = false[plugins.opt]path = "/opt/containerd"[plugins.restart]interval = "10s"[plugins.scheduler]pause_threshold = 0.02deletion_threshold = 0mutation_threshold = 100schedule_delay = "0s"startup_delay = "100ms"
重启containerd,以便于重新加载配置文件。
# systemctl daemon-reload
# systemctl restart containerd
3.2.3 ctr下载镜像
下载容器镜像
# ctr images pull --platform linux/amd64 docker.io/library/nginx:latest
说明:
* --platform linux/amd64 指定系统平台,也可以使用--all-platforms指定所有平台镜像。
查看已下载容器镜像
# ctr images ls
REF TYPE DIGEST SIZE PLATFORMS LABELSdocker.io/library/nginx:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x -
3.2.4 ctr上传镜像
上传到Harbor library公有项目
重新生成新的tag
# ctr images tag docker.io/library/nginx:latest harbor.kubemsb.com/library/nginx:latest
harbor.kubemsb.com/library/nginx:latest
查看已生成容器镜像
# ctr images ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
docker.io/library/nginx:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x -
harbor.kubemsb.com/library/nginx:latest application/vnd.docker.distribution.manifest.list.v2+json sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767 54.1 MiB linux/386,linux/amd64,linux/arm/v5,linux/arm/v7,linux/arm64/v8,linux/mips64le,linux/ppc64le,linux/s390x -
推送容器镜像至Harbor
# ctr images push --platform linux/amd64 --plain-http -u admin:密码 harbor.kubemsb.com/library/nginx:latest
说明:* 先tag再push
* 因为我们harbor是http协议,不是https协议,所以需要加上`--plain-http`
* `--user admin:Harbor12345`指定harbor的用户名与密码
输出:
manifest-sha256:0fd68ec4b64b8dbb2bef1f1a5de9d47b658afd3635dc9c45bf0cbeac46e72101: done |++++++++++++++++++++++++++++++++++++++|
config-sha256:dd025cdfe837e1c6395365870a491cf16bae668218edb07d85c626928a60e478: done |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.5 s total: 9.3 Ki (18.1 KiB/s)
下载已上传容器镜像
# ctr images pull --plain-http harbor.kubemsb.com/library/nginx:latest