TLS Automation
    Let's Encrypt

    Deploy cert-manager on Kubernetes

    Automate TLS certificate issuance and renewal in Kubernetes with cert-manager and Let's Encrypt — HTTP-01 and Cloudflare DNS-01 issuers.

    Overview

    cert-manager automates issuance and renewal of TLS certificates inside Kubernetes, backed by ACME (Let's Encrypt), a self-hosted CA, or Vault. This guide covers a production install on a self-managed cluster running on RamNode infrastructure (KVM/OpenStack nodes, kubeadm or k3s), with Let's Encrypt as the primary issuer.

    Assumptions

    • Kubernetes 1.27+ cluster reachable via kubectl, nodes provisioned on RamNode KVM/OpenStack instances
    • Helm 3 installed
    • Public DNS for the zones you'll issue certs for is on Cloudflare — used here for DNS-01 challenges
    • An nginx-ingress or Traefik ingress controller already deployed, OR you're issuing certs for non-ingress workloads (mail, internal services, etc.)

    1. Install cert-manager

    shell
    helm repo add jetstack https://charts.jetstack.io
    helm repo update
    
    kubectl create namespace cert-manager
    
    helm install cert-manager jetstack/cert-manager \
      --namespace cert-manager \
      --version v1.16.2 \
      --set crds.enabled=true \
      --set prometheus.enabled=true

    Verify:

    shell
    kubectl get pods -n cert-manager
    # cert-manager, cert-manager-cainjector, cert-manager-webhook should be Running

    2. Choose a challenge type

    ChallengeUse whenNotes
    HTTP-01Cert is for a host with a public ingress already routedSimplest, but needs port 80 reachable per-host
    DNS-01Wildcard certs, internal-only hosts, or multi-region hosts (ATL, EWR, PNW, LAX, NLX) that don't all sit behind one ingressRequires a DNS API token — use Cloudflare

    For RamNode's multi-datacenter footprint, DNS-01 via Cloudflare is recommended as the default — it avoids needing HTTP-01 reachability from every region and supports wildcards for things like *.apps.example.com or per-DC subdomains.

    Cloudflare API token scope

    Create a scoped token (not the global key) with Zone.DNS: Edit on the relevant zone(s).

    shell
    kubectl create secret generic cloudflare-api-token \
      --namespace cert-manager \
      --from-literal=api-token=<CF_SCOPED_TOKEN>

    3. Create a ClusterIssuer

    shell
    # cluster-issuer-letsencrypt.yaml
    apiVersion: cert-manager.io/v1
    kind: ClusterIssuer
    metadata:
      name: letsencrypt-prod
    spec:
      acme:
        server: https://acme-v02.api.letsencrypt.org/directory
        email: admin@example.com
        privateKeySecretRef:
          name: letsencrypt-prod-account-key
        solvers:
          - dns01:
              cloudflare:
                apiTokenSecretRef:
                  name: cloudflare-api-token
                  key: api-token
            selector:
              dnsZones:
                - "ramnode.com"

    Apply, and add a letsencrypt-staging issuer pointed at https://acme-staging-v02.api.letsencrypt.org/directory for testing — always validate against staging first to avoid Let's Encrypt rate limits.

    shell
    kubectl apply -f cluster-issuer-letsencrypt.yaml
    kubectl describe clusterissuer letsencrypt-prod

    4. Request a certificate

    Option A — via Ingress annotation (HTTP-01 or DNS-01)

    shell
    metadata:
      annotations:
        cert-manager.io/cluster-issuer: "letsencrypt-prod"
    spec:
      tls:
        - hosts:
            - lookingglass.ramnode.com
          secretName: lookingglass-tls

    Option B — explicit Certificate resource (recommended for non-ingress or wildcard certs)

    shell
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
      name: wildcard-ramnode
      namespace: default
    spec:
      secretName: wildcard-ramnode-tls
      issuerRef:
        name: letsencrypt-prod
        kind: ClusterIssuer
      dnsNames:
        - "*.ramnode.com"
        - "ramnode.com"
    shell
    kubectl apply -f certificate.yaml
    kubectl get certificate -A
    kubectl describe certificate wildcard-ramnode

    A Ready: True condition confirms issuance. Cert renews automatically at ~2/3 of its lifetime (Let's Encrypt certs are 90 days, so renewal fires around day 60).


    5. Multi-datacenter considerations

    If your nodes span RamNode regions (ATL, EWR, PNW, LAX, NLX):

    • If each region runs its own cluster (rather than one cluster spanning regions), each cluster needs its own cert-manager install and its own Cloudflare token secret — tokens aren't automatically shared across clusters.
    • Consider a single wildcard cert issued once and synced via a tool like kubernetes-replicator or manual secret sync if multiple clusters need to present the same cert (e.g., *.example.com mirrored across regions), rather than re-issuing identical certs five times against Let's Encrypt's rate limits.
    • For internal-only services (Nagios, NRPE endpoints, jump-host-facing dashboards) that don't need public trust, consider a self-signed ClusterIssuer (kind: CA) instead of burning Let's Encrypt issuance quota.

    6. Operational checklist

    • Staging issuer validated before switching Ingress/Certificate to letsencrypt-prod
    • Cloudflare token scoped to DNS edit only, stored as a Secret (not committed to the Ansible repo)
    • cert-manager-controller pod resource requests set (default is fine for small clusters, bump for large fleets)
    • Prometheus scraping enabled (--set prometheus.enabled=true above) and alerting on certmanager_certificate_expiration_timestamp_seconds to catch renewal failures before expiry
    • Confirm webhook TLS bootstrap succeeded: kubectl get validatingwebhookconfigurations | grep cert-manager

    7. Common failure modes

    SymptomLikely cause
    Certificate stuck in IssuingDNS-01 record not propagating — check kubectl describe order and challenge resources
    401 from CloudflareToken missing Zone.DNS:Edit scope or wrong zone
    Rate limit errors from Let's EncryptTesting against prod instead of staging, or re-issuing wildcard too often across regions
    Webhook timeout on cert-manager installCNI/network policy blocking the webhook pod — check if cluster has restrictive NetworkPolicies