K3s Lightweight K8s

K3s cheatsheet — install, configure, manage lightweight Kubernetes. k3s server, k3s agent, k3s kubectl, kubeconfig location. Perfect for edge, IoT, and dev environments.

8 min read

What it is

K3s is a lightweight, certified Kubernetes distribution designed for edge, IoT, CI, and development environments, offering a simplified installation and operation experience.

Installation

Linux (Systemd)

curl -sfL https://get.k3s.io | sh -

Installs K3s as a systemd service.

Linux (Systemd with specific version)

curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.28.5+k3s1" sh -

Installs a specific version of K3s.

Linux (ARM64)

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable servicelb --disable traefik" sh -

Installs K3s on ARM64, disabling default components if needed.

Linux (as a server, without systemd)

curl -sfL https://get.k3s.io | sh -s - --disable servicelb

Installs K3s as a server without systemd, disabling ServiceLB.

Linux (as an agent/worker node)

curl -sfL https://get.k3s.io | K3S_URL=https://your-server-ip:6443 K3S_TOKEN=your-node-token sh -

Joins an existing K3s cluster as an agent. Replace your-server-ip and your-node-token.

Mac (Docker Desktop)

docker run -d --privileged \
  -p 6443:6443 \
  -p 8472:8472/udp \
  -p 10250:10250 \
  --name k3s \
  rancher/k3s:latest

Runs K3s in a Docker container on macOS.

Windows (WSL2)

Follow the Linux instructions within your WSL2 distribution.

Core Concepts

Server vs. Agent

K3s operates in a client-server model. A server node runs the Kubernetes API server, etcd (or SQLite by default), and scheduler. Agent nodes (formerly called workers) run the Kubelet and Kube-proxy to host your workloads.

Embedded etcd vs. SQLite

By default, K3s uses SQLite for its datastore, simplifying setup. For production or HA, you can configure K3s to use an external etcd cluster.

Embedded Service Load Balancer (ServiceLB)

K3s includes a built-in Service Load Balancer that automatically assigns external IPs to Services of type LoadBalancer.

Embedded Ingress Controller (Traefik)

K3s bundles Traefik as its default Ingress controller, simplifying ingress management.

Commands / Usage

k3s (client)

This is the primary command-line tool for interacting with the K3s API server. You’ll typically use kubectl after configuring it to point to your K3s server.

k3s kubectl ...

Directly invokes kubectl commands through the K3s binary. This is useful if you haven’t set up a separate kubectl configuration.

k3s kubectl get nodes

Gets a list of nodes in the cluster.

k3s kubectl apply -f my-deployment.yaml

Applies a Kubernetes manifest.

k3s server

Starts K3s in server mode. Usually managed by systemd.

--disable servicelb

Disables the embedded Service Load Balancer.

k3s server --disable servicelb

--disable traefik

Disables the embedded Traefik Ingress controller.

k3s server --disable traefik

--data-dir /var/lib/rancher/k3s

Specifies the directory for K3s data.

k3s server --data-dir /mnt/ssd/k3s-data

--cluster-init

Initializes a new K3s cluster (used for HA setups).

k3s server --cluster-init

--server-url https://127.0.0.1:6443

Specifies the server URL (used when joining as an agent).

k3s agent --server-url https://192.168.1.100:6443 --token my-secret-token

--token my-secret-token

Specifies the cluster token for agent registration.

k3s agent --server-url https://192.168.1.100:6443 --token abcdef1234567890

k3s agent

Starts K3s in agent mode (worker node). Usually managed by systemd.

--server https://your-server-ip:6443

The URL of the K3s server to connect to.

k3s agent --server https://10.0.0.5:6443 --token K10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

--token K10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

The secret token used to join the cluster.

k3s agent --server https://k3s-master.local:6443 --token K1somerandomlongstringtoken

k3s etcd-snapshot

Commands for managing etcd snapshots.

save --name <snapshot-name>

Saves a new etcd snapshot.

k3s etcd-snapshot save --name my-cluster-backup-20231027

list

Lists available etcd snapshots.

k3s etcd-snapshot list

delete <snapshot-name>

Deletes a specific etcd snapshot.

k3s etcd-snapshot delete my-cluster-backup-20231027

restore --name <snapshot-name>

Restores the cluster from a specific etcd snapshot. Caution: This will overwrite current cluster state.

k3s etcd-snapshot restore --name my-cluster-backup-20231027

prune

Removes old snapshots, keeping a specified number.

k3s etcd-snapshot prune --snapshot-retention 5

Keeps the 5 most recent snapshots.

k3s config

Generates K3s configuration files.

ctr

Generates a containerd configuration file.

k3s config ctr > containerd-config.toml

kubelet

Generates a kubelet configuration file.

k3s config kubelet > kubelet-config.yaml

kube-proxy

Generates a kube-proxy configuration file.

k3s config kube-proxy > kube-proxy-config.yaml

k3s secrets

Encrypts and decrypts secrets.

encrypt <plaintext>

Encrypts a plaintext string.

k3s secrets encrypt "my-super-secret-password"

decrypt <encrypted-string>

Decrypts an encrypted string.

k3s secrets decrypt U2VjcmV0SW5wdXQ=

k3s ctl

Contains helper commands for managing K3s.

channel

Shows the current K3s release channel.

k3s ctl channel

get-token

Retrieves the cluster token.

k3s ctl get-token

k3s collect-logs

Bundles logs for debugging.

k3s collect-logs

Creates a tarball of logs.

k3s reset

Resets a K3s installation. Use with caution, this will remove K3s and associated data.

k3s-uninstall.sh

For systemd installations, this script removes K3s.

k3s reset

Resets the K3s service and cleans up data directories.

Common Patterns

Accessing kubectl

After installing K3s server, its configuration is usually placed at /etc/rancher/k3s/k3s.yaml. To use kubectl:

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
kubectl get nodes

Or, if you installed K3s directly on your workstation:

k3s kubectl get nodes

Adding a Worker Node

  1. Get the server IP and token:
    # On the server node
SERVER_IP=$(curl -s ifconfig.me) # Or your server's actual IP
NODE_TOKEN=$(sudo k3s-agent --server https://${SERVER_IP}:6443 --token-file /var/lib/rancher/k3s/server/node-token --data-dir /tmp | grep "token:" | awk '{print $2}')
# Note: The above command is a bit hacky, better to grab the token from /var/lib/rancher/k3s/server/node-token on the server
    A more robust way to get the token on the server:
    sudo cat /var/lib/rancher/k3s/server/node-token
  2. Install K3s on the worker node using the server’s IP and token:
    curl -sfL https://get.k3s.io | K3S_URL=https://<server_ip>:6443 K3S_TOKEN=<your_node_token> sh -

Deploying an Application

Create a deployment YAML file (my-app.yaml):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80

Apply it:

k3s kubectl apply -f my-app.yaml

Exposing an Application with LoadBalancer

Create a service YAML file (my-app-service.yaml):

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: LoadBalancer

Apply it:

k3s kubectl apply -f my-app-service.yaml

The K3s ServiceLB will assign an external IP. Check with:

k3s kubectl get svc nginx-service

Exposing an Application with Ingress (Traefik)

Create an ingress YAML file (my-app-ingress.yaml):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
spec:
  rules:
  - host: myapp.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-service
            port:
              number: 80

Apply it:

k3s kubectl apply -f my-app-ingress.yaml

You’ll need to configure your local DNS or /etc/hosts to resolve myapp.local to the K3s server’s IP.

High Availability (HA) Setup

Requires an external database (like etcd, PostgreSQL, MySQL).

  1. Start the first server:
    # Example using etcd
export ENDPOINTS="https://etcd1:2379,https://etcd2:2379,https://etcd3:2379"
export ETCD_CAFILE=/path/to/ca.crt
export ETCD_CERTFILE=/path/to/etcd.crt
export ETCD_KEYFILE=/path/to/etcd.key
k3s server \
  --cluster-init \
  --data-dir /var/lib/rancher/k3s/server \
  --tls-san <your_server_ip> \
  --disable servicelb \
  --disable traefik \
  --disable localstorage # If using external storage
  2. Start subsequent servers:
    k3s server \
  --data-dir /var/lib/rancher/k3s/server \
  --tls-san <your_server_ip> \
  --disable servicelb \
  --disable traefik \
  --disable localstorage # If using external storage
  3. Start agent nodes: Use the K3S_URL and K3S_TOKEN pointing to one of the server IPs.

Gotchas

Default Ports

K3s listens on 6443 for the Kubernetes API. Ensure this port is accessible if you’re connecting from other machines.

ServiceLB IP Range

ServiceLB allocates IPs from a default range (10.43.0.0/16). If this conflicts with your network, you might need to configure it.

Traefik Configuration

Traefik is deployed as a DaemonSet and Deployment. Its configuration is managed via ConfigMaps. You can customize it by editing these resources.

Node Token Security

The node token is crucial for joining nodes. Protect it, as anyone with the token can join your cluster. It’s stored at /var/lib/rancher/k3s/server/node-token on the server.

Data Persistence

By default, K3s uses SQLite and stores data in /var/lib/rancher/k3s. For production, consider using an external database and persistent storage for your applications.

k3s-uninstall.sh

This script is automatically installed and is the recommended way to remove K3s from a systemd-based Linux installation. Running it cleans up services, directories, and iptables rules.

kubectl Context

If you have multiple Kubernetes clusters configured, ensure your kubectl context is set correctly to point to your K3s cluster. k3s kubectl bypasses this by default.