GitOps repository

1) Harvester Hyperconverged Infrastructure

[...]

Configure Harvester HCI nodes through cloud-init (requires node reboot):

kubectl apply -f system/Harvester/cloudinit-disable-nic-offloading.yaml

2) Persistent storage

2.1) CSI plugin for SMB (CIFS):

kubectl apply -f storage/csi-driver-smb/application-csi-driver-smb.yaml

2.2) Harvester CSI plugin

See Harvester CSI Driver

3) GitOps

3.1) Install Helm Chart

See ArgoCD:

helm repo add argo https://argoproj.github.io/argo-helm
helm repo update
helm install argo-cd -n argo-cd --create-namespace argo/argo-cd --values system/ArgoCD/chart-values.yml

Retrieve initial password:

kubectl get secret -n argocd argocd-initial-admin-secret -oyaml | yq e '.data.password | @base64d'

Login with username admin and the initial password, browse to User Info and Update Password.

3.1) Adopt through GitOps

kubectl apply -f system/ArgoCD/application-argo-cd.yaml

4) Secret management

Prereq: latest kubeseal release

kubectl apply -f system/SealedSecrets/application-sealed-secrets-controller.yaml

Retrieve public/private keys (store these on a secure location!):

kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml > BitnamiSealedSecrets.masterkey.yml

5) Services

5.1) Gitea (git repository)

Required for all other workloads

kubectl apply -f services/Gitea/application-gitea.yaml

5.2) Argus (release management)

kubectl apply -f services/Argus/application-argus.yaml

5.3) Authelia (single sign-on))

kubectl apply -f services/Authelia/application-authelia.yaml

5.4) Vaultwarden (password manager)

kubectl apply -f services/Vaultwarden/application-vaultwarden.yaml

5.5) DDclient (dynamic dns)

kubectl apply -f services/DDclient/application-ddclient.yaml

5.6) Gotify (notifications)

kubectl apply -f services/Gotify/application-gotify.yaml

5.7) Webtop (remote desktop)

kubectl apply -f services/Webtop/application-webtop.yaml

5.8) Lighttpd (webserver)

kubectl apply -f services/Lighttpd/application-lighttpd.yaml

5.9) PVR toolsuite (automated media management)

API-keys whitelisted in ingressroutes:

spec:
  routes:
  - match: Host(`<fqdn>`) && (Headers(`X-Api-Key`, `<secret>`) || Query(`apikey`, `<secret>`))
    [...]

5.9.1) Jellyfin (media library)

kubectl apply -f services/PVR/Jellyfin/application-jellyfin.yaml

5.9.2) Jellyseerr (media requests management)

kubectl apply -f services/PVR/Jellyseerr/application-jellyseerr.yaml

5.9.3) Prowlarr (indexer management)

kubectl apply -f services/PVR/Prowlarr/application-prowlarr.yaml

5.9.4) Radarr (movie management)

kubectl apply -f services/PVR/Radarr/application-radarr.yaml

5.9.5) SABnzbd (download client)

kubectl apply -f services/PVR/SABnzbd/application-sabnzbd.yaml

5.9.6) Sonarr (tv management)

kubectl apply -f services/PVR/Sonarr/application-sonarr.yaml

6) Miscellaneous

Various notes/useful links

• Replacement for not-yet-deprecated kubectl get all -A:

  kubectl get $(kubectl api-resources --verbs=list -o name | paste -sd, -) --ignore-not-found --all-namespaces
  

• DaemonSet to configure nodes' sysctl fs.inotify.max-user-watches:

  kubectl apply -f system/InotifyMaxWatchers/daemonSet-InotifyMaxWatchers.yml
  

• Debug DNS lookups within the cluster:

  kubectl run -it --rm dnsutils --restart=Never --image=gcr.io/kubernetes-e2e-test-images/dnsutils -- nslookup [-debug] [fqdn]
  

  or

  kubectl run -it --rm busybox --restart=Never --image=busybox:1.28 -- nslookup api.github.com [-debug] [fqdn]
  

• Memory-leak liveness probe:

  livenessProbe:
    exec:
      command:
        - sh
        - -c
        - test $(cat /proc/1/smaps | grep -i pss |  awk '{Total+=$2} END {print int(Total/1024)}') -le <limit>