距离上一次部署K8S集群，已经是8年前的事情了，这次记录下在Ubuntu 24.04上部署Kubernetes 1.36集群。
环境要求
Ubuntu 24.04
Kubernetes v1.36
Rancher v2.15
IP
HostName
角色
10.10.10.60
k8s-master
控制节点
10.10.10.61
k8s-worker-01
Worker节点
10.10.10.62
k8s-worker-02
Worker节点
部署前配置 Hosts 所有服务器均配置
1 2 3 4 5 6 7 sudo vim /etc/hosts10.10.10.60 k8s-master 10.10.10.61 k8s-worker-01 10.10.10.62 k8s-worker-02
关闭Swap 所有服务器都需要配置。
1 2 3 4 5 6 sudo swapoff -asudo sed -i '/\sswap\s/s/^/#/' /etc/fstabfree -h
修改前：
1 2 3 4 root@k8s-master:~# free -h total used free shared buff/cache available Mem: 538Mi 380Mi 229Mi 3.1Mi 111Mi 157Mi Swap: 2.0Gi 32Mi 2.0Gi
修改后
1 2 3 4 root@k8s-master:~# free -h total used free shared buff/cache available Mem: 538Mi 413Mi 196Mi 3.9Mi 112Mi 124Mi Swap: 0B 0B 0B
屏蔽Systemd Swap单元
1 2 3 4 sudo systemctl mask swap.targetswapon --show
加载内核模块 所有服务器都需要配置。
1 2 3 4 5 6 7 sudo cat > /etc/modules-load.d/k8s.conf <<'EOF' overlay br_netfilter EOF sudo modprobe overlaysudo modprobe br_netfilter
验证
1 lsmod | egrep 'overlay|br_netfilter'
修改内核参数 所有服务器都需要配置。
1 2 3 4 5 6 7 8 9 sudo cat > /etc/sysctl.d/99-kubernetes.conf <<'EOF' net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 net.ipv4.ip_forward = 1 net.ipv4.conf.all.rp_filter = 0 net.ipv4.conf.default.rp_filter = 0 EOF sudo sysctl --system
验证
1 2 sudo sysctl net.ipv4.ip_forwardsudo sysctl net.bridge.bridge-nf-call-iptables
配置SELinux和防火墙 Ubuntu默认不启用SELinux，无需配置。
对于防火墙ufw，可以使用如下命令关闭。
1 2 3 4 5 6 7 sudo systemctl stop ufwsudo systemctl disable ufwsudo ufw statusgetenforce
containerd运行时配置 安装containerd
Kubernetes 最新版推荐使用containerd，所以先安装它，Ubuntu官方源自带了这个容器，直接使用如下命令安装即可。
1 2 sudo apt updatesudo apt install -y containerd
生成默认配置
1 2 sudo mkdir -p /etc/containerdcontainerd config default | sudo tee /etc/containerd/config.toml
修改 cgroup 驱动为 systemd
1 sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
kubeadm 管理的 kubelet 默认使用 systemd cgroup 驱动，containerd 也必须保持一致。如果不改，kubeadm init 大概率会失败。
配置国内加速地址
编辑/etc/containerd/config.toml，找到对应版本下的[plugins."...registry"]段落，设置config_path指向/etc/containerd/certs.d
由于本次安装的containerd的版本是2.2.1，对于2.x，请使用如下配置：
1 2 3 4 5 6 7 8 [plugins."io.containerd.cri.v1.images".registry] config_path = "/etc/containerd/certs.d" [plugins.'io.containerd.cri.v1.images'.registry] config_path = '/etc/containerd/certs.d:/etc/docker/certs.d'
修改sandbox的镜像地址
编辑/etc/containerd/config.toml，找到sandbox的配置
1 2 [plugins.'io.containerd.cri.v1.images'.pinned_images] sandbox = 'registry.k8s.io/pause:3.10.1'
将其修改为如下：
1 2 [plugins.'io.containerd.cri.v1.images' .pinned_images] sandbox = 'registry.aliyuncs.com/google_containers/pause:3.10.2'
创建加速器hosts.toml文件
为需要加速的仓库创建对应的目录和配置文件。以 docker.io 为例
1 2 sudo mkdir -p /etc/containerd/certs.d/docker.iosudo vim /etc/containerd/certs.d/docker.io/hosts.toml
在hosts.toml中填入如下内容：
1 2 3 4 server = "https://docker.io" [host."<加速器地址>"] capabilities = ["pull" , "resolve" ]
例如本次的加速地址：
1 2 3 4 5 6 7 8 9 10 11 12 13 server = "https://docker.io" [host."https://docker.1panel.live"] capabilities = ["pull" , "resolve" ] [host."https://docker.sparkcr.cn"] capabilities = ["pull" , "resolve" ] [host."https://hub.rat.dev"] capabilities = ["pull" , "resolve" ] [host."https://dockerproxy.net"] capabilities = ["pull" , "resolve" ]
containerd 会按顺序 尝试这些 [host]，第一个成功的即被使用。
重启并设置开机自启
1 2 sudo systemctl restart containerdsudo systemctl enable containerd
测试加速地址
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 sudo ctr images pull --hosts-dir "/etc/containerd/certs.d" docker.io/library/nginx:latestdocker.io/library/nginx:latest saved └──index (abe47724e466) already exists ├──manifest (e57665aa5d61) already exists │ └──config (4c1e47d5156e) already exists ├──manifest (918e8d119b7c) already exists │ └──config (24e95a0b421e) already exists ├──manifest (80e8942d76c2) already exists │ └──config (07954515bed9) complete |++++++++++++++++++++++++++++++++++++++| ├──manifest (a3f4937fe245) already exists │ └──config (111065a1fc90) already exists ├──manifest (7c0a84ff6658) already exists │ └──config (9b02f35d286f) complete |++++++++++++++++++++++++++++++++++++++| ├──manifest (9c0f39aa1c46) already exists │ ├──config (e9567a94eddc) already exists │ ├──layer (46243d3234ed) extracted |++++++++++++++++++++++++++++++++++++++| │ ├──layer (2056b40bae09) extracted |++++++++++++++++++++++++++++++++++++++| │ ├──layer (6b37362b3da7) extracted |++++++++++++++++++++++++++++++++++++++| │ ├──layer (f802f27d954b) extracted |++++++++++++++++++++++++++++++++++++++| │ ├──layer (3326c3817340) extracted |++++++++++++++++++++++++++++++++++++++| │ ├──layer (afa8dec48454) extracted |++++++++++++++++++++++++++++++++++++++| │ └──layer (f1169c633cbc) extracted |++++++++++++++++++++++++++++++++++++++| ├──manifest (86753677ff59) already exists │ └──config (1428287eb324) already exists ├──manifest (11e4aa39f602) already exists │ └──config (9298f24a2e15) already exists ├──manifest (1ad1d69bf75e) already exists │ └──config (0fecf3ebfb0e) already exists ├──manifest (9f1d10e21b69) already exists │ └──config (7f608302f44f) already exists ├──manifest (fdd27b1339a4) already exists │ └──config (d5dd4cf6ff02) already exists ├──manifest (9a536b164be3) already exists │ └──config (ee1cc8f2f941) already exists ├──manifest (752ced86f64e) already exists │ └──config (a7f7e2385f4e) already exists ├──manifest (7091b89b716b) already exists │ └──config (474fe4d961f5) already exists ├──manifest (b70a461b4fe9) already exists │ └──config (15438ba8ed88) complete |++++++++++++++++++++++++++++++++++++++| └──manifest (aebba161d93e) already exists └──config (9e6692401de7) already exists application/vnd.oci.image.index.v1+json sha256:abe47724e466aeab9a345d8e46a221c2fa8953c7848bb4a3bd9976a7199f8cf2 Pulling from OCI Registry (docker.io/library/nginx:latest) elapsed: 19.8s total: 60.6 M (3.1 MiB/s)
部署Kubernetes 替换清华大学源 1 2 3 4 5 6 7 8 9 sudo cp /etc/apt/sources.list /etc/apt/sources.list.baksudo sed -i "s@http://.*archive.ubuntu.com@https://mirrors.tuna.tsinghua.edu.cn@g" /etc/apt/sources.listsudo sed -i "s@http://.*security.ubuntu.com@https://mirrors.tuna.tsinghua.edu.cn@g" /etc/apt/sources.listsudo apt update
添加清华大学K8S源，并且指定v1.36
1 2 3 4 5 6 7 8 9 10 sudo mkdir -p /etc/apt/keyringscurl -fsSL https://pkgs.k8s.io/core:/stable:/v1.36/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg sudo chmod 644 /etc/apt/keyrings/kubernetes-apt-keyring.gpgecho 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.36/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.listsudo chmod 644 /etc/apt/sources.list.d/kubernetes.list
安装k8s 安装指定版本的组件，并且锁定
1 2 3 4 5 6 7 sudo apt updatesudo apt install -y kubelet kubeadm kubectlsudo apt-mark hold kubelet kubeadm kubectl
确认kubeadm和kubectl版本一致
1 2 3 4 5 kubeadm version kubelet --version kubeadm version: &version.Info{Major:"1" , Minor:"36" , EmulationMajor:"" , EmulationMinor:"" , MinCompatibilityMajor:"" , MinCompatibilityMinor:"" , GitVersion:"v1.36.5" , GitCommit:"ad950d1cc78b0183c476bd4d3f1934c104229727" , GitTreeState:"clean" , BuildDate:"2026-09-23T17:09:21Z" , GoVersion:"go1.26.8" , Compiler:"gc" , Platform:"linux/amd64" } Kubernetes v1.36.5
如果版本不一致，kubeadm init 会报错。
初始化集群 1 2 3 4 5 sudo kubeadm init \ --apiserver-advertise-address=10.10.10.60 \ --image-repository=registry.aliyuncs.com/google_containers \ --kubernetes-version=v1.36.5 \ --pod-network-cidr=192.168.0.0/16
--apiserver-advertise-address k8s master的IP地址--image-repository 镜像仓库，推荐使用阿里云的--kubernetes-version k8s版本与之前kubeadm version输出一致--pod-network-cidr 必须与后续 CNI 插件要求的网段一致，Calico 默认用 192.168.0.0/16，也可以设置为其他IP段，不要与现有的网络重复即可，推荐使用默认地址。
集群创建成功时，会提示如下信息：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME /.kube sudo cp -i /etc/kubernetes/admin.conf $HOME /.kube/config sudo chown $(id -u):$(id -g) $HOME /.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 10.10.10.60:6443 --token hqk9jq.bkgaawc0eiclcfcj \ --discovery-token-ca-cert-hash sha256:709f98f4cf9c4a2dfb712ae7772219b5bed93ca0f6d822cd76d63051e150b26b
如果创建失败，请删除配置重新创建
1 2 3 4 sudo kubeadm reset -fsudo rm -rf /etc/kubernetes/manifests/etcd.yamlsudo rm -rf /var/lib/etcdsudo rm -rf /etc/kubernetes/pki/etcd
然后安装cri-tools,并配置
1 2 3 4 5 6 7 sudo apt install -y cri-toolssudo tee /etc/crictl.yaml <<EOF runtime-endpoint: unix:///run/containerd/containerd.sock image-endpoint: unix:///run/containerd/containerd.sock timeout: 10 debug: false EOF
查看所有控制平面容器状态
1 2 3 sudo crictl ps -a | grep -E 'etcd|kube-apiserver' sudo crictl logs <etcd-container-id>
或者通过如下方式查看日志：
1 2 3 4 sudo ctr plugins ls | grep crisudo crictl versionsudo cat /var/lib/kubelet/kubeadm-flags.envsudo journalctl -u kubelet --no-pager | head -50
或者带上详细信息去创建集群
1 2 3 4 5 6 sudo kubeadm init \ --apiserver-advertise-address=10.10.10.60 \ --image-repository=registry.aliyuncs.com/google_containers \ --kubernetes-version=v1.36.5 \ --pod-network-cidr=192.168.0.0/16 \ --v=5
配置kubectl 根据安装成功的提示，执行如下命令即可
1 2 3 mkdir -p $HOME /.kubesudo cp -i /etc/kubernetes/admin.conf $HOME /.kube/configsudo chown $(id -u):$(id -g) $HOME /.kube/config
验证：
1 2 3 4 kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master NotReady control-plane 7m24s v1.36.5
安装CNI插件（Calico） 在master节点10.10.10.60执行：
1 2 3 4 mkdir ~/k8scd ~/k8swget https://ghproxy.net/https://raw.githubusercontent.com/projectcalico/calico/v3.32.2/manifests/calico.yaml
我已经将对应的镜像推送到harbor中，下面进行地址替换
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 grep "image:" calico.yaml image: quay.io/calico/cni:v3.32.2 image: quay.io/calico/cni:v3.32.2 image: quay.io/calico/node:v3.32.2 image: quay.io/calico/node:v3.32.2 image: quay.io/calico/kube-controllers:v3.32.2 sed -i 's#quay.io/calico#harbor.dev.net.cn/k8s/quay.io/calico#g' calico.yaml grep "image:" calico.yaml image: harbor.dev.net.cn/k8s/quay.io/calico/cni:v3.32.2 image: harbor.dev.net.cn/k8s/quay.io/calico/cni:v3.32.2 image: harbor.dev.net.cn/k8s/quay.io/calico/node:v3.32.2 image: harbor.dev.net.cn/k8s/quay.io/calico/node:v3.32.2 image: harbor.dev.net.cn/k8s/quay.io/calico/kube-controllers:v3.32.2
尝试pull镜像
也可以提前全部拉去，加快速度
1 sudo crictl pull harbor.dev.net.cn/k8s/quay.io/calico/node:v3.32.2
应用服务
1 2 3 4 5 6 7 8 9 10 11 12 13 kubectl apply -f calico.yaml kubectl get pods -n kube-system -w NAME READY STATUS RESTARTS AGE calico-kube-controllers-95cb87bb-m4mqt 1/1 Running 0 5m9s calico-node-4jnt5 1/1 Running 0 5m9s coredns-6b5f954497-2r679 1/1 Running 0 36m coredns-6b5f954497-wgx57 1/1 Running 0 36m etcd-k8s-master 1/1 Running 1 36m kube-apiserver-k8s-master 1/1 Running 0 36m kube-controller-manager-k8s-master 1/1 Running 1 36m kube-proxy-xtgv9 1/1 Running 0 36m kube-scheduler-k8s-master 1/1 Running 1 36m
添加worker节点 将之前master节点初始化集群时，给出的信息分别在另外两台服务器上执行一遍
1 2 kubeadm join 10.10.10.60:6443 --token hqk9jq.bkgaawc0eiclcfcj \ --discovery-token-ca-cert-hash sha256:709f98f4cf9c4a2dfb712ae7772219b5bed93ca0f6d822cd76d63051e150b26b
等待拉去镜像后即可加入集群，期间可通过如下命令查看是否成功
1 2 3 4 5 6 7 8 kubectl get nodes root@k8s-master:~/k8s# kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready control-plane 44m v1.36.5 k8s-worker-01 Ready <none> 5m8s v1.36.5 k8s-worker-02 Ready <none> 5m17s v1.36.5
刚加入集群应该是NotReady，下载镜像还需要时间。等下载完镜像后就可以成功启动
1 2 3 4 5 6 root@k8s-master:~/k8s# kubectl get pods -n kube-system -o wide | grep calico calico-kube-controllers-95cb87bb-m4mqt 1/1 Running 0 9m4s 192.168.235.193 k8s-master <none> <none> calico-node-4jnt5 1/1 Running 0 9m4s 10.10.10.60 k8s-master <none> <none> calico-node-tt2lp 0/1 Init:0/3 0 68s 10.10.10.61 k8s-worker-01 <none> <none> calico-node-zhdvl 0/1 Init:0/3 0 77s 10.10.10.62 k8s-worker-02 <none> <none>
过一会再执行一次查看节点状态
1 2 3 4 5 root@k8s-master:~/k8s# kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-master Ready control-plane 42m v1.36.5 10.10.10.60 <none> Ubuntu 24.04.5 LTS 6.8.0-146-generic (amd64) containerd://2.2.1 k8s-worker-01 Ready <none> 3m5s v1.36.5 10.10.10.61 <none> Ubuntu 24.04.5 LTS 6.8.0-146-generic (amd64) containerd://2.2.1 k8s-worker-02 Ready <none> 3m14s v1.36.5 10.10.10.62 <none> Ubuntu 24.04.5 LTS 6.8.0-146-generic (amd64) containerd://2.2.1
此时就表示worker节点已经成功加入集群。一般不需要Master节点参与pod的调度（生产环境一定不要），对于测试环境可通过如下方式让master节点也参与调度（切记不可用于生产环境）
1 kubectl taint nodes k8s-master node-role.kubernetes.io/control-plane:NoSchedule-
配置StorageClass持久化存储 由于本次部署需要使用HPC，所以会采用Lustre，对于测试环境，可以使用NFS。
本次部署是实验环境，采用Rancher出品的local-path-provisioner
创建数据目录
1 mkdir -p /data/k8s-local-pv
下载配置
1 2 3 4 5 6 curl -fL -o local-path-storage.yaml \ https://ghproxy.net/https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.37/deploy/local-path-storage.yaml curl -fL -o local-path-storage.yaml \ https://cdn.jsdelivr.net/gh/rancher/local-path-provisioner@v0.0.37/deploy/local-path-storage.yaml
替换存储地址
1 2 3 4 5 6 7 8 grep -n '/opt/local-path-provisioner' local-path-storage.yaml sed -i 's#/opt/local-path-provisioner#/data/k8s-local-pv#g' local-path-storage.yaml grep -n -E 'k8s-local-pv|/opt/local-path-provisioner' local-path-storage.yaml
替换国内加速镜像
1 2 3 4 5 sed -i 's#image: rancher/local-path-provisioner:v0.0.37#image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/rancher/local-path-provisioner:v0.0.37#' local-path-storage.yaml sed -i 's#image: busybox#image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/library/busybox:latest#' local-path-storage.yaml
部署
1 2 3 4 5 6 7 8 9 10 kubectl apply -f local-path-storage.yaml kubectl -n local-path-storage rollout status deployment/local-path-provisioner --timeout =180s kubectl -n local-path-storage get pods kubectl get storageclass
部署成功后，可以做一个测试：
创建一个测试PVC，观察它在被 Pod 使用前是否保持 Pending：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 cat <<EOF | kubectl apply -f - apiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-local-pvc spec: accessModes: - ReadWriteOnce storageClassName: local-path resources: requests: storage: 128Mi EOF kubectl get pvc test-local-pvc
此时应该显示 Pending（因为还没有 Pod 使用它）。
然后创建一个 Pod 挂载这个 PVC：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 cat <<EOF | kubectl apply -f - apiVersion: v1 kind: Pod metadata: name: test-local-pod spec: containers: - name: test image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/library/busybox:latest command: ["sh", "-c", "echo hello > /data/test.txt && sleep 3600"] volumeMounts: - name: data mountPath: /data volumes: - name: data persistentVolumeClaim: claimName: test-local-pvc EOF kubectl get pvc test-local-pvc -w
Pod 创建后，PVC 应该从 Pending 变为 Bound。确认绑定后，可以查看数据是否写入：
1 kubectl exec test-local-pod -- cat /data/test.txt
如果输出 hello，说明存储读写正常。
清理测试资源
1 2 kubectl delete pod test-local-pod kubectl delete pvc test-local-pvc
由于 RECLAIMPOLICY 是 Delete，删除 PVC 后对应的存储目录会被自动清理。
注意：Local Path Provisioner只需要在master节点部署即可，调度到其他节点时会自动创建
安装Metrics Server 配置Gateway API（暂时不用） Insgress已经不在维护，需要换成官方推荐的Gateway API
安装Gateway API标准CRDs 1 curl -fL -o gateway-standard-install.yaml https://ghproxy.net/https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.6.2/standard-install.yaml
部署
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 kubectl apply -f gateway-standard-install.yaml kubectl get crd | grep gateway.networking.k8s.io backendtlspolicies.gateway.networking.k8s.io 2026-10-05T01:18:43Z gatewayclasses.gateway.networking.k8s.io 2026-10-05T01:18:43Z gateways.gateway.networking.k8s.io 2026-10-05T01:18:43Z grpcroutes.gateway.networking.k8s.io 2026-10-05T01:18:44Z httproutes.gateway.networking.k8s.io 2026-10-05T01:18:44Z listenersets.gateway.networking.k8s.io 2026-10-05T01:18:44Z referencegrants.gateway.networking.k8s.io 2026-10-05T01:18:44Z tcproutes.gateway.networking.k8s.io 2026-10-05T01:18:44Z tlsroutes.gateway.networking.k8s.io 2026-10-05T01:18:44Z udproutes.gateway.networking.k8s.io 2026-10-05T01:18:44Z
安装Envoy Gateway控制器 对于Gateway API的实现，还有另外几个：Istio、Cilium Gateway API、NGINX Gateway Fabric。目前
使用 Helm 从 OCI 仓库安装。注意，如果你的集群已经安装了 Gateway API v1.5 或更高版本的 CRDs，安装 Envoy Gateway 时必须加上 --skip-crds 参数 。因为 Envoy Gateway 的 Helm Chart 可能自带旧版 CRDs，直接安装会尝试降级，被 Gateway API 的 ValidatingAdmissionPolicy 阻止。
1 kubectl apply -f https://ghproxy.net/https://github.com/envoyproxy/gateway/releases/download/v1.8.5/install.yaml
安装Cert-Manager 由于 Rancher 强制要求全局 HTTPS 访问 ,安装Rancher前需要安装证书管理器(Cert-Manager)
上一步已经添加了对应的仓库,只需要安装即可
1 2 3 4 helm install cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set crds.enabled=true
有如下提示即表示安装成功
1 cert-manager v1.21.2 has been deployed successfully!
安装ingress-nginx 1 2 curl -fL --retry 3 -o ingress-nginx.yaml \ https://ghproxy.net/https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.15.1/deploy/static/provider/baremetal/deploy.yaml
查看镜像地址
1 2 3 4 root@k8s-master:~/k8s# grep "image:" ingress-nginx.yaml image: registry.k8s.io/ingress-nginx/controller:v1.15.1@sha256:594ceea76b01c592858f803f9ff4d2cb40542cae2060410b2c95f75907d659e1 image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.6.9@sha256:01038e7de14b78d702d2849c3aad72fd25903c4765af63cf16aa3398f5d5f2dd image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.6.9@sha256:01038e7de14b78d702d2849c3aad72fd25903c4765af63cf16aa3398f5d5f2dd
批量修改镜像地址
1 2 3 4 sed -i 's#registry.k8s.io/ingress-nginx/controller:v1.15.1@sha256:594ceea76b01c592858f803f9ff4d2cb40542cae2060410b2c95f75907d659e1#swr.cn-north-4.myhuaweicloud.com/ddn-k8s/registry.k8s.io/ingress-nginx/controller:v1.15.1#g' ingress-nginx.yaml sed -i 's#registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.6.9@sha256:01038e7de14b78d702d2849c3aad72fd25903c4765af63cf16aa3398f5d5f2dd#swr.cn-north-4.myhuaweicloud.com/ddn-k8s/registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.6.9#g' ingress-nginx.yaml
确认镜像地址已被修改
1 2 3 4 grep "image:" ingress-nginx.yaml image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/registry.k8s.io/ingress-nginx/controller:v1.15.1 image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.6.9 image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.6.9
部署和验证
1 2 3 4 5 6 7 8 9 10 11 12 kubectl apply -f ingress-nginx.yaml kubectl get pods -n ingress-nginx -w NAME READY STATUS RESTARTS AGE ingress-nginx-controller-7d97db8899-gg7pj 1/1 Running 0 54s kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx-controller NodePort 10.107.178.45 <none> 80:30434/TCP,443:31442/TCP 106s ingress-nginx-controller-admission ClusterIP 10.96.17.28 <none> 443/TCP 106s
1 curl -fL --retry 3 -o metallb-frr-k8s.yaml \ https://ghproxy.net/https://raw.githubusercontent.com/metallb/metallb/v0.16.1/config/manifests/metallb-frr-k8s.yaml
替换国内源：
1 2 3 4 5 6 7 8 9 10 11 12 13 cat metallb-frr-k8s.yaml |grep "image:" image: quay.io/metallb/controller:v0.16.1 image: quay.io/metallb/frr-k8s:v0.0.25 image: quay.io/metallb/frr-k8s:v0.0.25 image: quay.io/frrouting/frr:10.4.3 image: quay.io/frrouting/frr:10.4.3 image: quay.io/frrouting/frr:10.4.3 image: quay.io/frrouting/frr:10.4.3 image: quay.io/frrouting/frr:10.4.3 image: quay.io/metallb/frr-k8s:v0.0.25 image: quay.io/metallb/frr-k8s:v0.0.25 image: quay.io/metallb/frr-k8s:v0.0.25 image: quay.io/metallb/speaker:v0.16.1
替换
1 2 3 4 5 6 7 sed -i \ -e 's#quay.io/metallb/#harbor.dev.net.cn/k8s/quay.io/metallb/#g' \ metallb-frr-k8s.yaml sed -i \ -e 's#quay.io/frrouting/#harbor.dev.net.cn/k8s/quay.io/frrouting/#g' \ metallb-frr-k8s.yaml
对于本地三节点的服务器，推荐使用原生模式（二层模式）
1 2 3 4 5 6 7 8 9 10 11 12 wget -O metallb-native.yaml \ https://ghproxy.net/https://raw.githubusercontent.com/metallb/metallb/v0.16.1/config/manifests/metallb-native.yaml sed -i \ 's#quay.io/metallb/#harbor.dev.net.cn/k8s/quay.io/metallb/#g' \ metallb-native.yaml grep "image:" metallb-native.yaml image: harbor.dev.net.cn/k8s/quay.io/metallb/controller:v0.16.1 image: harbor.dev.net.cn/k8s/quay.io/metallb/speaker:v0.16.1
部署
1 2 3 4 5 6 7 8 9 kubectl apply -f metallb-native.yaml root@k8s-master:~/k8s# kubectl get pods -n metallb-system -w NAME READY STATUS RESTARTS AGE controller-78b9cdcfb5-fvwn7 1/1 Running 0 2m35s speaker-8cpc7 1/1 Running 0 112s speaker-m8vv8 1/1 Running 0 103s speaker-tfwgw 1/1 Running 0 97s
配置
确保10.10.10.70-10.10.10.79未被使用。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 cat > metallb-config.yaml <<'EOF' apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: lan-pool namespace: metallb-system spec: addresses: - 10.10.10.70-10.10.10.79 --- apiVersion: metallb.io/v1beta1 kind: L2Advertisement metadata: name: lan namespace: metallb-system spec: ipAddressPools: - lan-pool EOF kubectl apply -f metallb-config.yaml
将ingress-nginx修改为LoadBalancer
1 2 3 kubectl patch svc ingress-nginx-controller \ -n ingress-nginx \ -p '{"spec":{"type":"LoadBalancer"}}'
查看分配地址
1 2 3 4 5 kubectl get svc ingress-nginx-controller -n ingress-nginx -w root@k8s-master:~/k8s# kubectl get svc ingress-nginx-controller -n ingress-nginx -w NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx-controller LoadBalancer 10.107.178.45 10.10.10.70 80:30434/TCP,443:31442/TCP 39m
后面就可以访问https://rancher.10.10.10.70.sslip.io这个地址了。
Rancher切换为metallb
由于我是先安装的Rancher，所以需要做这一步，如果是新安装的，直接指定到新的地址即可。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 helm repo add jetstack https://charts.jetstack.io helm repo update helm upgrade --install cert-manager jetstack/cert-manager \ --namespace cert-manager \ --create-namespace \ --set crds.enabled=true kubectl get pods -n cert-manager kubectl get crd | grep cert-manager.io certificaterequests.cert-manager.io 2026-10-05T05:14:12Z certificates.cert-manager.io 2026-10-05T05:14:12Z challenges.acme.cert-manager.io 2026-10-05T05:14:12Z clusterissuers.cert-manager.io 2026-10-05T05:14:12Z issuers.cert-manager.io 2026-10-05T05:14:13Z orders.acme.cert-manager.io 2026-10-05T05:14:12Z helm upgrade rancher rancher-stable/rancher \ --namespace cattle-system \ --reuse-values \ --set hostname=rancher.10.10.10.70.sslip.io \ --set ingress.ingressClassName=nginx
安装Helm 下载
1 wget https://mirrors.huaweicloud.com/helm/v3.22.0/helm-v3.22.0-linux-amd64.tar.gz
安装
1 2 3 4 5 6 7 tar -zxvf helm-v3.22.0-linux-amd64.tar.gz sudo mv linux-amd64/helm /usr/local/bin/helmrm -fr linux-amd64/
验证
1 2 helm version version.BuildInfo{Version:"v3.22.0" , GitCommit:"144ca65f8501953fa8b41cd1d37c7223051c85b7" , GitTreeState:"clean" , GoVersion:"go1.26.8" }
添加仓库
1 2 helm repo add rancher-stable https://releases.rancher.com/server-charts/stable helm repo add jetstack https://charts.jetstack.io
验证
1 2 helm repo list helm repo update
安装Rancher管理集群 创建Rancher专用命名空间
1 kubectl create namespace cattle-system
安装Rancher
1 2 3 4 5 6 7 8 9 helm install rancher rancher-stable/rancher \ --namespace cattle-system \ --set hostname=rancher.10.10.10.60.sslip.io \ --set bootstrapPassword=P@ssw0rd123 \ --set global.cattle.psp.enabled=false \ --set replicas=1 \ --set ingress.tls.source=rancher \ --set ingress.ingressClassName=nginx \ --set systemDefaultRegistry=docker.m.daocloud.io
--namespace 刚才创建的命名空间hostname 访问Rancher的入口，没有域名的话可以使用开源的DNS映射服务sslip.io或nip.io，如果是线上环境，则填写域名即可。bootstrapPassword 登录Rancher的密码。replicas 副本，测试环境设置为1(只启动一个pod)，生产环境需要设置为3global.cattle.psp.enabled 必加项，显式关闭 PodSecurityPolicy (PSP)，否则可能会提示API找不到的错误ingress.tls.source 指定使用 Rancher 内置的 cert-manager 模式自动生成 TLS 证书systemDefaultRegistry 镜像加速地址，否则会出现镜像拉取失败
以本教程为例：访问rancher的地址是：https://rancher.10.10.10.60.sslip.io，密码是P@ssw0rd123。
Update
如果安装了Metallb负载均衡，这里的hostname应当是负载均衡的地址：例如rancher.10.10.10.70.sslip.io
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 NAME: rancher LAST DEPLOYED: Mon Oct 5 02:18:12 2026 NAMESPACE: cattle-system STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: Rancher Server has been installed. Rancher may take several minutes to fully initialize. Please standby while Certificates are being issued, Containers are started and the Ingress rule comes up. Check out our docs at https://rancher.com/docs/ If you provided your own bootstrap password during installation, browse to https://rancher.10.10.10.60.sslip.io to get started. If this is the first time you installed Rancher, get started by running this command and clicking the URL it generates: ``` echo https://rancher.10.10.10.60.sslip.io/dashboard/?setup=$(kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{.data.bootstrapPassword|base64decode}}' )``` To get just the bootstrap password on its own, run: ``` kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{.data.bootstrapPassword|base64decode}}{{ "\n" }}' ``` Happy Containering!
如果失败，可以先卸载
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 helm uninstall rancher -n cattle-system kubectl delete deployment rancher -n cattle-system --force --grace-period=0 kubectl delete pod -n cattle-system --all --force --grace-period=0 kubectl delete job -n cattle-system --all --force --grace-period=0 crictl pull docker.m.daocloud.io/rancher/rancher:v2.15.2 ctr -n k8s.io i pull docker.m.daocloud.io/rancher/rancher:v2.15.2
由于安装了ingress-nginx，所以需要将rancher的Ingress指定为 IngressClass
1 2 3 4 5 6 7 8 9 10 11 12 13 14 kubectl get ingress -n cattle-system NAME CLASS HOSTS ADDRESS PORTS AGE rancher <none> rancher.10.10.10.60.sslip.io 80, 443 95m kubectl patch ingress rancher \ -n cattle-system \ --type =merge \ -p '{"spec":{"ingressClassName":"nginx"}}' kubectl get ingress -n cattle-system NAME CLASS HOSTS ADDRESS PORTS AGE rancher nginx rancher.10.10.10.60.sslip.io 80, 443 98m
即可登录rancher。
参考资料 安装 ：： MetalLB，Kubernetes 裸机负载均衡器
Installing Rancher :: SUSE® Rancher Manager
Release Local Path Provisioner v0.0.37 · rancher/local-path-provisioner
Release controller-v1.15.1 · kubernetes/ingress-nginx
Releases · envoyproxy/gateway
Release v1.6.2 · kubernetes-sigs/gateway-api
Kubernetes
wechat
alipay
