Deploy Sealed Secrets on Kubernetes

Overview

Sealed Secrets (Bitnami) lets you encrypt a Kubernetes Secret client-side into a SealedSecret custom resource that's safe to commit to a Git repo. Only the in-cluster controller (holding the private key) can decrypt it back into a real Secret. This is a much lighter-weight alternative to External Secrets Operator/Vault — no external secret store to run — and is a good fit if the goal is simply "stop committing plaintext secrets into the Ansible/Fabric/Kubernetes repos" without standing up Vault.

Use Sealed Secrets instead of ESO when:

• You want GitOps-friendly, committable encrypted secrets, not a live external secret store
• You don't need centralized rotation/auditing across many consumers — each secret is sealed per-cluster
• You want to avoid operating Vault or a cloud KMS just for Kubernetes secrets

Use ESO instead when: secrets need to be shared across multiple clusters/apps from one source of truth, or need frequent rotation without a re-seal-and-commit cycle.

1. Install the Sealed Secrets controller

helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm repo update

kubectl create namespace kube-system --dry-run=client -o yaml | kubectl apply -f -

helm install sealed-secrets sealed-secrets/sealed-secrets \
  --namespace kube-system \
  --set fullnameOverride=sealed-secrets-controller

Verify:

kubectl get pods -n kube-system -l name=sealed-secrets-controller

The controller generates a keypair on first start (a 4096-bit RSA key by default) and stores the private key as a Secret in-cluster. This private key is the single point of failure — back it up immediately (step 4).

2. Install the kubeseal CLI

On your jump host / workstation (not in-cluster):

KUBESEAL_VERSION='0.27.1'
curl -OL "https://github.com/bitnami-labs/sealed-secrets/releases/download/v${KUBESEAL_VERSION}/kubeseal-${KUBESEAL_VERSION}-linux-amd64.tar.gz"
tar -xvzf kubeseal-${KUBESEAL_VERSION}-linux-amd64.tar.gz kubeseal
sudo install -m 755 kubeseal /usr/local/bin/kubeseal

Fetch the controller's public cert (safe to distribute, used for sealing — no cluster access required to seal a secret once you have this):

kubeseal --controller-name=sealed-secrets-controller \
  --controller-namespace=kube-system \
  --fetch-cert > ./sealed-secrets-pub-cert.pem

Store this cert in your configuration repo so anyone sealing secrets doesn't need live cluster access, just this public cert.

3. Seal a secret

Example: sealing MariaDB credentials for an application.

kubectl create secret generic mariadb-credentials \
  --namespace default \
  --from-literal=username=solusvm_reader \
  --from-literal=password='<plaintext-password>' \
  --dry-run=client -o yaml > mariadb-credentials.yaml

kubeseal --cert sealed-secrets-pub-cert.pem \
  --format yaml \
  < mariadb-credentials.yaml > mariadb-sealedsecret.yaml

rm mariadb-credentials.yaml   # never commit the plaintext intermediate

mariadb-sealedsecret.yaml now contains only encrypted data and is safe to commit to Git.

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: mariadb-credentials
  namespace: default
spec:
  encryptedData:
    username: AgB3f9k2...
    password: AgCx82nQ...

Apply it — the controller decrypts and materializes the real Secret automatically:

kubectl apply -f mariadb-sealedsecret.yaml
kubectl get secret mariadb-credentials -o yaml

4. Back up the controller's private key (critical)

If the controller's private key is lost — cluster rebuild, namespace wipe, etc. — every previously sealed secret becomes permanently undecryptable, including anything committed to Git.

kubectl get secret -n kube-system \
  -l sealedsecrets.bitnami.com/sealed-secrets-key \
  -o yaml > sealed-secrets-master-key-backup.yaml

Store this backup encrypted, off-cluster, e.g. in Vault (if ESO/Vault is also deployed) or an offline encrypted archive — not in the same Git repo as the SealedSecrets themselves. Treat it with the same care as an SSH CA private key.

To restore into a rebuilt cluster:

kubectl apply -f sealed-secrets-master-key-backup.yaml
kubectl delete pod -n kube-system -l name=sealed-secrets-controller
# controller picks up the restored key on restart

5. Key rotation

The controller rotates its active signing key on a schedule (default: every 30 days) but retains old keys for decrypting previously sealed secrets — you don't need to re-seal everything on rotation. To force immediate rotation (e.g., suspected key compromise):

kubectl delete secret -n kube-system \
  -l sealedsecrets.bitnami.com/sealed-secrets-key
kubectl delete pod -n kube-system -l name=sealed-secrets-controller

This generates a fresh key. Old SealedSecrets remain decryptable only if the old key secrets weren't also deleted — be deliberate about this distinction between "rotate" and "revoke."

6. Scope options when sealing

By default, a SealedSecret is bound to a specific namespace + name (can't be renamed or moved to another namespace after sealing) — this is the safest default. Two other modes exist if needed:

# Namespace-wide (any name within the target namespace)
kubeseal --scope namespace-wide ...

# Cluster-wide (any namespace, any name — least restrictive, avoid unless truly needed)
kubeseal --scope cluster-wide ...

For per-app credentials, stick with the default strict scope so a sealed secret can't be replayed into an unintended namespace.

7. Operational checklist

• Public cert (sealed-secrets-pub-cert.pem) distributed to the Ansible repo / jump host, versioned alongside it
• Private key backup stored off-cluster, encrypted, access-restricted — test restore at least once before relying on it
• .gitignore or pre-commit hook in place to prevent accidentally committing the plaintext intermediate kubectl create secret ... --dry-run=client output instead of the sealed version
• Default strict scope used unless there's a specific reason for namespace-wide/cluster-wide
• Controller pod has a PodDisruptionBudget / runs with replicas: 1 understood as a constraint — if it's down, sealing/unsealing new secrets pauses, though already-materialized Secrets keep working

8. Common failure modes