Overview
This note captures the install commands used to bootstrap a K3s cluster with Google OIDC authentication and S3-backed etcd snapshots. Replace the placeholder values before running.
Control Plane Node 1 (bootstrap)
Use this to initialize the first server and enable Google OIDC claims.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
| curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.29.3+k3s1 sh -s - server \
--cluster-init \
--node-external-ip=<PUBLIC_IP> \
--flannel-backend=wireguard-native \
--flannel-external-ip \
--disable=traefik \
--tls-san <CONTROL_PLANE_DNS> \
--write-kubeconfig-mode "0644" \
--default-local-storage-path /k3s-data \
--node-taint node-role.kubernetes.io/control-plane=true:NoSchedule \
--node-taint CriticalAddonsOnly=true:NoExecute \
--node-taint node-role.kubernetes.io/master=true:NoSchedule \
--kube-apiserver-arg=oidc-issuer-url=https://accounts.google.com \
--kube-apiserver-arg=oidc-client-id=<OIDC_CLIENT_ID> \
--kube-apiserver-arg=oidc-username-claim=email \
--etcd-s3 \
--etcd-s3-endpoint <S3_ENDPOINT> \
--etcd-s3-access-key <S3_ACCESS_KEY> \
--etcd-s3-secret-key <S3_SECRET_KEY> \
--etcd-s3-bucket <S3_BUCKET> \
--etcd-s3-region <S3_REGION> \
--etcd-s3-folder etcd \
--flannel-backend none \
--cluster-cidr=10.42.0.0/16 \
--service-cidr=10.43.0.0/16 \
--disable-network-policy
|
Control Plane Node 2 (join)
Join an additional server to the existing control plane using the cluster token.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
| curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.29.3+k3s1 \
K3S_TOKEN=<K3S_TOKEN> \
sh -s - server \
--server https://<CONTROL_PLANE_IP>:6443 \
--node-external-ip=<PUBLIC_IP> \
--disable=traefik \
--flannel-external-ip \
--tls-san <CONTROL_PLANE_DNS> \
--write-kubeconfig-mode "0644" \
--default-local-storage-path /k3s-data \
--node-taint node-role.kubernetes.io/control-plane=true:NoSchedule \
--node-taint CriticalAddonsOnly=true:NoExecute \
--node-taint node-role.kubernetes.io/master=true:NoSchedule \
--kube-apiserver-arg=oidc-issuer-url=https://accounts.google.com \
--kube-apiserver-arg=oidc-client-id=<OIDC_CLIENT_ID> \
--kube-apiserver-arg=oidc-username-claim=email \
--etcd-s3 \
--etcd-s3-endpoint <S3_ENDPOINT> \
--etcd-s3-access-key <S3_ACCESS_KEY> \
--etcd-s3-secret-key <S3_SECRET_KEY> \
--etcd-s3-bucket <S3_BUCKET> \
--etcd-s3-region <S3_REGION> \
--etcd-s3-folder etcd \
--flannel-backend none \
--cluster-cidr=10.42.0.0/16 \
--service-cidr=10.43.0.0/16 \
--disable-network-policy
|
Worker Node (Multus)
Join a worker node to the cluster.
1
2
3
4
5
6
| curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.29.3+k3s1 \
K3S_URL=https://<CONTROL_PLANE_DNS>:6443 \
K3S_TOKEN=<K3S_TOKEN> \
sh -s - agent \
--node-external-ip=<PUBLIC_IP> \
--node-ip=<PRIVATE_IP>
|
Multus CNI Plugin Install (example)
Use this if you need to install additional CNI plugins on a node.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
| apt-get dist-upgrade
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.29.3+k3s1 \
K3S_URL=https://<CONTROL_PLANE_DNS>:6443 \
K3S_TOKEN=<K3S_TOKEN> \
sh -s - agent \
--node-external-ip=<PUBLIC_IP> \
--node-ip=<PRIVATE_IP>
mkdir /tmp/cni-plugins
wget https://github.com/containernetworking/plugins/releases/download/v1.3.0/cni-plugins-linux-amd64-v1.3.0.tgz
tar xvf cni-plugins-linux-amd64-v1.3.0.tgz -C /tmp/cni-plugins
mv /tmp/cni-plugins/* /opt/cni/bin/
ls -la /opt/cni/bin/
dd if=/dev/zero of=/dev/k3s-data/data bs=1M count=100
reboot
|
