Let's Encrypt certificate expired in Kubernetes

Expired Let's Encrypt in Kubernetes means the live ingress still mounts an old tls.crt—renew, sync Secret, restart data plane.

When this happens

kubectl get secret shows notAfter in the past while Certificate claims Ready. Service mesh sidecars terminate mTLS before traffic hits the ingress that holds the public Let us Encrypt cert. Pod churn during node upgrades can leave ingress using projected secrets from a previous revision while cert-manager already wrote a newer tls.crt.

Only one region's load balancer was updated after cert-manager renewal. EKS Fargate-only ingress may lack solver options supported on EC2 node groups. Service mesh rotates its own mesh-gateway certificates on a different cadence, masking the still-expired public ingress cert in the same namespace.

First response

Let's Encrypt certs expired in Kubernetes—renew cert-manager Certificates, sync TLS Secrets, restart ingress.

Searches for kubernetes letsencrypt certificate expired usually mean automated renewal failed, succeeded locally but never reached the TLS listener, or hit Let's Encrypt rate limits. The fix is almost always permissions, challenge reachability, or a missing reload—not buying a longer-lived commercial cert as the first move.

For Let's Encrypt certificate expired in Kubernetes, read the ACME or panel error verbatim, run a dry-run renewal if your stack supports it, then confirm what certificate strangers receive on port 443 after you deploy. Certificate files on disk lie; only live handshakes matter for users.

Ingress still serves an expired Let's Encrypt cert when the Secret stopped updating but pods were not restarted. In Kubernetes the failure is often sync lag between cert-manager, the TLS Secret, and ingress controller caches—not the CA refusing renewal. Triage should capture timestamps, account URLs, and the last Boulder error type before infrastructure changes widen blast radius. Pause destructive retries until you can cite the last Boulder error type and the challenge mechanism in use; random package reinstalls rarely fix presentation gaps.

See also cert-manager ACME solver failed and cert-manager not renewing Certificate resource.

Why renewal failed

Controllers cache TLS configuration; Secret updates need reconcile and sometimes pod restart. Record namespace, secretName, and ingressClass in every TLS incident template so rotating clusters do not lose tribal knowledge about which Git repo owns the Certificate. Kubernetes TLS is a pipeline: Issuer, Order, Secret, Ingress, cloud LB—monitor entire chain.

Manual helm secrets bypass cert-manager and expire quietly. Node-local cache controllers occasionally serve old TLS session tickets—retest with fresh openssl after pod restarts, not only Certificate Ready flips. Platform upgrades should include cert-manager conformance tests in staging.

Rate-limited issuers leave Secrets untouched with aging data. Multi-cluster fleets should standardize issuer names to avoid copy-paste typo ClusterIssuers. Escalate with timestamps, hostnames, and serial comparisons attached so the next engineer does not repeat already-failed renew attempts.

letsencrypt expired in cluster context means the tls.crt data is old or not mounted. Check Certificate Ready condition, secret resourceVersion, and whether ingress references the correct secretName. Clarify in tickets whether failure is validation, issuance, deploy, or listener presentment—teams too often mix these phases when reading logs. Platform SLOs should include Certificate age per namespace because ingress outages read as application outages to product teams.

Not always cert-manager fault—manual secrets or helm charts can pin static PEMs. kubectl get secrets showing future notAfter does not prove data plane freshness if ingress controller pods have not reloaded mounts after cert-manager writes.

Common causes

These patterns drive most kubernetes letsencrypt certificate expired tickets. They explain why automation logs look green while browsers or APIs still break.

Issuer rate limited; secret never rotated.

Ingress class changed; solver could not reach pod.

Argo rolled back secret from git with old cert. Mutating webhooks copying Secrets across namespaces propagate expired copies faster than issuers renew. ExternalDNS creating records for stale ingress names can resurrect challenges for deleted routes, confusing renewal queues.

Diagnose

Decode Secret cert, compare to public openssl, trace ingress secretName. Always capture the cloud load balancer listener description alongside kubectl output; many expired presentations trace to cloud TLS not cluster secrets.

kubectl get secret TLS_SECRET -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -dates
kubectl describe certificate -n APP_NS

kubectl describe certificate,order,challenge; verify events in namespace timeline aligns with expiry window. Decode ingress tls stanza from live object, not git, before blaming cert-manager reconciliation lag. Include cloud provider certificate association resource identifiers when opening vendor tickets about stale edge TLS.

Fix step-by-step

Follow this sequence for let's encrypt certificate expired in kubernetes. Capture command output after each step so the next responder can pick up without rerunning destructive changes.

1. Describe Certificate and fix issuer if not Ready.
2. Force renew if renewal stalled.
3. Confirm Secret resourceVersion changed.
4. Verify ingress references correct secretName.
5. Restart ingress controller deployment if needed.
6. Test public LoadBalancer openssl serial.
7. Audit helm releases pinning static tls files.
8. Enable cluster-wide cert expiry metrics.
9. Archive redacted ACME logs and openssl transcripts in the ticket before closing.
10. Schedule a follow-up dry-run before the next natural renewal window.

Verify the live certificate

OpenSSL

echo | openssl s_client -connect HOST:443 -servername HOST 2>/dev/null | openssl x509 -noout -dates -subject -issuer

Replace HOST with the public hostname on the certificate. Run the same command after any reload or deploy change; acceptance means notAfter moved forward and serial changed versus the incident-opening capture.

How to read the output

openssl LoadBalancer hostname; kubectl get secret -o yaml compare base64 cert dates. Compare openssl to the LoadBalancer DNS name customers use, not only kubectl port-forward to a Service, which may bypass the public certificate path entirely.

curl

curl -vI --connect-timeout 10 https://HOST/ 2>&1 | grep -Ei 'expire|SSL certificate|subject:|issuer:'

Replace HOST with the public hostname users request.

Roll out and verify

Fix issuer; force renewal via cmctl.

Rolling restart ingress controller deployment. Restart only ingress data plane after Secret update unless docs for your controller version require full cluster roll.

Cluster-wide cert expiry exporter. Publish a short post-incident note listing which hop served stale TLS and which command proved the fix so the next renewal inherits a checklist.

What to automate

kube-state-metrics on Certificate notAfter prevents silent drift. Prometheus rules combining Certificate renewalTime and Secret age catch silent stalls.

ssldays validates external names the same way customers reach the cluster. ssldays validates what mobile clients see via public load balancers.