How to deploy Nginx instead of Traefik as your ingress controller on K3s

Task

This knowledge base article will provide the directions for deploying NGINX instead of Traefik as your Kubernetes ingress controller on K3s. Please note that Traefik is the support ingress controller for K3s and NGINX is not officially supported by SUSE Rancher support.

Requirements

• K3s 1.17+ (may apply to other versions)

Background

By default, K3s uses Traefik as the ingress controller for your cluster. The decision to use Traefik over NGINX was based on multi-architecture support across x86 and ARM based platforms. Normally Traefik meets the needs of most Kubernetes clusters. However, there are unique use cases where NGINX may be required or preferred. If you don't think you need NGINX, it's recommended to stick with Traefik.

Solution

The first step to using NGINX or any alternative ingress controller is to tell K3s that you do not want to deploy Traefik. When installing K3s add the following --no-deploy traefik flag to the INSTALL_K3S_EXEC environment variable.  Or in some cases, you may need a --disable traefik option instead:

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--no-deploy traefik" sh -s -
or
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable traefik" sh -s -

If you have already downloaded the k3s.sh install script, you can run the following:

INSTALL_K3S_EXEC="--no-deploy traefik" k3s.sh
or
INSTALL_K3S_EXEC="--disable traefik" k3s.sh

This will install the K3s server and form a single node cluster. You can confirm the cluster is operational ("Ready") by running:

$ kubectl get nodes
NAME                STATUS   ROLES    AGE    VERSION
ip-10-0-0-100       Ready    master   1m     v1.18.4+k3s1

Note, if you already had the kubectl binary installed on your host and it is not configured correctly, you may need to run k3s kubectl instead of kubectl.

Next, confirm your out-of-box pods are running and Traefik is not running:

$ kubectl get pods -A
NAMESPACE                   NAME                                                      READY   STATUS      RESTARTS   AGE
kube-system                 local-path-provisioner-58fb86bdfd-vt57d                   1/1     Running     0          1m
kube-system                 metrics-server-6d684c7b5-qmlcn                            1/1     Running     0          1m
kube-system                 coredns-d798c9dd-72qrq                                    1/1     Running     0          1m

K3s has a nice feature that allows you to deploy Helm Charts by placing a HelmChart YAML in /var/lib/rancher/k3s/server/manifests. Create this file by running:

cat >/var/lib/rancher/k3s/server/manifests/ingress-nginx.yaml <<EOF
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: ingress-nginx
  namespace: kube-system
spec:
  chart: ingress-nginx
  repo: https://kubernetes.github.io/ingress-nginx
  targetNamespace: ingress-nginx
  version: v3.29.0
  set:
  valuesContent: |-
    fullnameOverride: ingress-nginx
    controller:
      kind: DaemonSet
      hostNetwork: true
      hostPort:
        enabled: true
      service:
        enabled: false
      publishService:
        enabled: false
      metrics:
        enabled: true
        serviceMonitor:
          enabled: true
      config:
        use-forwarded-headers: "true"
EOF

K3s periodically polls the manifests folder and applies the YAML in these files. After about a minute, you should see new pods running, including the NGINX Ingress Controller and default backend:

$ kubectl get pods -A
NAMESPACE                   NAME                                                      READY   STATUS      RESTARTS   AGE
kube-system                 local-path-provisioner-58fb86bdfd-vt57d                   1/1     Running     0          2m
kube-system                 metrics-server-6d684c7b5-qmlcn                            1/1     Running     0          2m
kube-system                 coredns-d798c9dd-72qrq                                    1/1     Running     0          2m
kube-system                 helm-install-ingress-nginx-s99ct                          0/1     Completed   0          1m
ingress-nginx               ingress-nginx-default-backend-7fb8995f4d-h6rkb            1/1     Running     0          1m
ingress-nginx               ingress-nginx-controller-c8mkg                            1/1     Running     0          1m

You'll also see a helm-install-ingress-nginx pod in your environment. K3s uses this pod to deploy the Helm Chart and it's normal for it to be in a READY=0/1 and STATUS=Completed state once the Helm Chart has been successfully deployed. In the event your Helm Chart failed to deploy, you can view the logs of this pod to troubleshoot further.

Reference

• K3s documentation
• NGINX Ingress Controller