Install HA k3s with Traefik, CertManager, and a Virtual IP

Configure Master Node

Update and install necessary packages:

apt-get update && apt-get upgrade && apt-get install curl keepalived vim

Edit the keepalived configuration:

vim /etc/keepalived/keepalived.conf

And add the following configuration:

vrrp_instance VI_1 {
    state MASTER
    interface eth0  # Replace with your active network interface
    virtual_router_id 91
    priority 100
    virtual_ipaddress {
        10.1.0.91 # Any free IP in the network
    }
}

Restart keepalived and install k3s:

systemctl restart keepalived
curl -sfL https://get.k3s.io/ | sh -s - server --token=EXAMPLE_TOKEN --tls-san 10.1.0.91 --disable=traefik --cluster-init

Adding --tls-san ensures no certificate errors over the virtual IP.

If no errors occur, the first node should be started. For HA, we need at least two more nodes.

Set Up Slave Nodes

Both nodes are set up exactly the same way:

Update and install necessary packages:

apt-get update && apt-get upgrade && apt-get install curl keepalived vim

Edit the keepalived configuration:

vim /etc/keepalived/keepalived.conf

And add the same configuration as above.

Restart keepalived and join the cluster:

systemctl restart keepalived
curl -sfL https://get.k3s.io/ | sh -s - server --token=EXAMPLE_TOKEN --tls-san 10.1.0.91 --disable=traefik --server https://ipv4-first-server:6443/

Test the Installation

Install kubectl on the local machine.
Copy /etc/rancher/k3s/k3s.yaml from the first node to local ~/.kube/config.
Run kubectl get nodes. The output should look like this:

NAME              STATUS   ROLES                       AGE   VERSION
scpx-k3s-prod-1   Ready    control-plane,etcd,master   12d   v1.27.6+k3s1
scpx-k3s-prod-2   Ready    control-plane,etcd,master   11d   v1.27.6+k3s1
scpx-k3s-prod-3   Ready    control-plane,etcd,master   11d   v1.27.6+k3s1

Install CertManager

Follow the steps below:

Install Helm package manager locally.
Set up certmanager:

mkdir certmanager && cd certmanager
vim values.yaml

And add the following configuration:

installCRDs: false
replicaCount: 3
extraArgs:
  - --dns01-recursive-nameservers=1.1.1.1:53,9.9.9.9:53
  - --dns01-recursive-nameservers-only
podDnsPolicy: None
podDnsConfig:
  nameservers:
    - 1.1.1.1
    - 9.9.9.9

Create secret and certificates:

vim secret-cf-token.yaml

And add the following configuration:

---
apiVersion: v1
kind: Secret
metadata:
  name: cloudflare-api-key-secret
  namespace: cert-manager
type: Opaque
stringData:
  api-key: GLOABAL-API-KEY

vim letsencrypt-production.yaml

And add the following configuration:

---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-production
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: CLOUDFLARE-EMAIL
    privateKeySecretRef:
      name: letsencrypt-production
    solvers:
      - dns01:
          cloudflare:
            email: CLOUDFLARE-EMAIL
            apiKeySecretRef:
              name: cloudflare-api-key-secret
              key: api-key

Install cert-manager and apply configurations:

helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --version v1.13.1 --values=values.yaml --set installCRDs=true
kubectl apply -f secret-cf-token.yaml
kubectl apply -f letsencrypt-production.yaml

Install Traefik

Follow the steps below:

Setup traefik:

mkdir traefik && cd traefik
helm repo add traefik https://traefik.github.io/charts
vim values.yaml

And add the following configuration:

globalArguments:
  - "--global.sendanonymoususage=false"
  - "--global.checknewversion=false"
 
additionalArguments:
  - "--serversTransport.insecureSkipVerify=true"
  - "--accesslog=true"
  - "--log.level=INFO"
 
deployment:
  enabled: true
  replicas: 3
  annotations: {}
  podAnnotations: {}
  additionalContainers: []
  initContainers: []
 
ports:
  web:
    redirectTo: websecure
  websecure:
    tls:
      enabled: true
 
ingressRoute:
  dashboard:
    enabled: false
 
providers:
  kubernetesCRD:
    enabled: true
    ingressClass: traefik-external
    allowExternalNameServices: true
  kubernetesIngress:
    enabled: true
    allowExternalNameServices: true
    publishedService:
      enabled: false
 
rbac:
  enabled: true

Install traefik and configure dashboard:

helm install traefik traefik/traefik -f values.yaml -n traefik
vim dashboard.yaml

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: traefik-dashboard
  namespace: traefik
  annotations:
    kubernetes.io/ingress.class: traefik-external
spec:
  entryPoints:
    - websecure
  routes:
    - match: Host(`DOMAIN.TLD`)
      kind: Rule
      services:
        - name: api@internal
          kind: TraefikService
  tls:
    secretName: traefik-tls
 
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: traefik-tls
  namespace: traefik
spec:
  secretName: traefik-tls
  issuerRef:
    name: letsencrypt-production
    kind: ClusterIssuer
  dnsNames:
    - DOMAIN.TLD

kubectl apply -f dashboard.yaml

Finally, forward the ports in the firewall and set up DNS entries. It should work now.