在K8S集群中部署容器运行时：Containerd

一、Containerd简介

containerd 实现了 kubernetes 的 Container Runtime Interface (CRI) 接口，提供容器运行时核心功能，如镜像管理、容器管理等，相比 dockerd 更加简单、健壮和可移植。

而且由于kubernetes官方社区在发布的v1.20版本中声明将弃用 Dockershim，即 Docker 容器运行时接口（CRI），这意味着不再支持 Docker，并将在后续版本中删除。

二、Containerd部署

• 所需组件下载地址：下载最新的二进制包
  • https://github.com/kubernetes-sigs/cri-tools/releases/
  • https://github.com/opencontainers/runc/releases/
  • https://github.com/containernetworking/plugins/releases/
  • https://github.com/containerd/containerd/releases/

• 如果下载太慢，那么下载到本地电脑然后上传至服务器
• 注意：经过测试，建议containerd部署v1.6+以上版本，低版本测试发现重启服务器后containerd.sock文件需要很久才会生成。

[[email protected] ~]# cd /opt/k8s/work/
[[email protected] work]# mkdir containerd
[[email protected] work]# tar -xvf containerd-1.6.0-linux-amd64.tar.gz -C containerd
[[email protected] work]# tar -xvf crictl-v1.23.0-linux-amd64.tar.gz

[[email protected] work]# mkdir cni-plugins
[[email protected] work]# tar -xvf cni-plugins-linux-amd64-v1.0.1.tgz -C cni-plugins
[[email protected] work]# mv runc.amd64 runc

• 将程序包分发给Worker节点
• NODE_IPS变量可以在profile中定义worker节点的IP地址数组

[[email protected] work]# for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"

    ssh [email protected]${node_ip} "mkdir -p /opt/k8s/bin"
    scp containerd/bin/*  crictl  cni-plugins/*  runc  [email protected]${node_ip}:/opt/k8s/bin
    ssh [email protected]${node_ip} "chmod a+x /opt/k8s/bin/* && mkdir -p /etc/cni/net.d"
  done

• 通过下面的指令生成默认的配置文件

mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml

• 注意修改一些配置
  • (1) 比如我这里的root和state所对应的目录路径跟默认的是不一样的，需要我们手动修改
  • (2) 修改pause镜像的地址，不然使用默认的地址会下载不下来
  • (3) 使用 systemd cgroup 驱动程序，将配置中的SystemdCgroup改为true

[[email protected] work]# vim /etc/containerd/config.toml
disabled_plugins = []
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/data/k8s/containerd/root"
state = "/data/k8s/containerd/state"
version = 2
...

[plugins]
  [plugins."io.containerd.grpc.v1.cri"]
    sandbox_image = "k8s.dockerproxy.com/pause"
    [plugins."io.containerd.grpc.v1.cri".cni]
      bin_dir = "/opt/k8s/bin"
      conf_dir = "/etc/cni/net.d"
...
          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
            BinaryName = ""
            CriuImagePath = ""
            CriuPath = ""
            CriuWorkPath = ""
            IoGid = 0
            IoUid = 0
            NoNewKeyring = false
            NoPivotRoot = false
            Root = ""
            ShimCgroup = ""
            SystemdCgroup = true
...

• 分发配置文件至集群各个worker节点
• NODE_IPS变量我是在/原文链接：https://www.dqzboy.cometc/profile中定义的各个Worker节点IP数组
• CONTAINERD_DIR同样也是在/etc/profile中定义的，这里我定义的路径为/data/k8s/containerd

[[email protected] work]# for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh [email protected]${node_ip} "mkdir -p /etc/containerd/ ${CONTAINERD_DIR}/{root,state}"
    scp containerd-config.toml [email protected]${node_ip}:/etc/containerd/config.toml
  done

[[email protected] work]# cat <<EOF | sudo tee containerd.service
[Unit]
Description=Lightweight Kubernetes
Documentation=https://containerd.io
After=network-online.target

[Service]
Environment="PATH=/opt/k8s/bin:/bin:/sbin:/usr/bin:/usr/sbin"
Environment="KUBELET_EXTRA_ARGS=--cgroup-driver=systemd"
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStartPre=-/bin/mkdir -p /run/containerd
ExecStart=/opt/k8s/bin/containerd \\
         -c /etc/containerd/config.toml \\
         -a /run/containerd/containerd.sock \\
         --state /data/k8s/containerd/state \\
         --root /data/k8s/containerd/root 

KillMode=process
Delegate=yes
OOMScoreAdjust=-999
LimitNOFILE=1024000
LimitNPROC=1024000
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target
EOF

[[email protected] work]# for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp containerd.service [email protected]${node_ip}:/etc/systemd/system
    ssh [email protected]${node_ip} "systemctl enable containerd && systemctl restart containerd"
  done

三、部署crictl命令行工具

crictl 是兼容 CRI 容器运行时的命令行工具，提供类似于 docker 命令的功能。具体参考官方文档。

[[email protected] work]# cat << EOF | sudo tee crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
EOF

• 分发到所有 worker 节点：

[[email protected] work]# for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp crictl.yaml [email protected]${node_ip}:/etc/crictl.yaml
  done

四、修改kubele使用systemd

• 在kubelet启动配置中追加如下参数，然后重启kubelet

--cgroup-driver=systemd