Caddy automatic HTTPS renewal failed

Caddy automatic HTTPS stops when ACME challenge or storage paths break—read obtain errors in logs before hand-rolling PEMs.

When this happens

Sites flip to HTTP redirect loops or TLS errors after a Caddy upgrade. Kubernetes sidecars mounting read-only Caddy data volumes block renewal writes after the first successful obtain. Automatic HTTPS on admin hostnames enabled by copy-paste can steal obtain attempts meant for customer sites.

Wildcard labels were added without a DNS provider stanza. Developers enabling tls internal on .localhost names should not pollute production logs.

First response

Caddy automatic HTTPS stopped renewing—read ACME obtain errors, DNS provider tokens, and challenge port reachability.

Searches for caddy certificate renewal failed usually mean automated renewal failed, succeeded locally but never reached the TLS listener, or hit Let's Encrypt rate limits. The fix is almost always permissions, challenge reachability, or a missing reload—not buying a longer-lived commercial cert as the first move.

For Caddy automatic HTTPS renewal failed, read the ACME or panel error verbatim, run a dry-run renewal if your stack supports it, then confirm what certificate strangers receive on port 443 after you deploy. Certificate files on disk lie; only live handshakes matter for users.

Caddy hides TLS complexity until automatic HTTPS stops and logs show obtain certificate failed for a sudden apex rename. Caddy terminates ACME internally, so read journalctl -u caddy for order errors before you hand certificates manually. A single bad tls block or DNS provider stanza can stall every site in the same Caddyfile. Triage should capture timestamps, account URLs, and the last Boulder error type before infrastructure changes widen blast radius.

See also ACME DNS-01 challenge propagation delays and ACME HTTP-01 challenge failed 403.

Why renewal failed

Caddy manages ACME internally; configuration mistakes stall every site using the same issuer block. JSON config with multiple tls issuers needs consistent email and storage paths—mixing internal and public CAs in one block can stall all routes when one issuer fails. Treat Caddy data directory durability like a small database with backups and restore drills.

On-demand TLS needs deliberate rate and permission controls; misconfiguration looks like random obtain failures. When running behind another reverse proxy, disable stale automatic HTTPS on internal hostnames that are not publicly resolvable to avoid endless obtain loops. Config management should test caddy validate on every merge.

Manual tls file paths override automation for that site until removed. Keep DNS API tokens in systemd credentials or cgroup-visible files Caddy user can read.

Caddy requests and renews certificates through its admin-managed ACME clients, storing assets in its data directory. Failure means challenge solving or permission to write certs broke—not that browsers bypass TLS. OnDemand TLS and wildcards require explicit DNS provider modules with API credentials. Clarify in tickets whether failure is validation, issuance, deploy, or listener presentment—teams too often mix these phases when reading logs.

Caddy renewal failure is not cured by putting Cloudflare orange cloud in front without matching DNS-01 or TLS-ALPN challenge settings. It is not fixed by chmod 777 on /var/lib/caddy alone when configuration JSON is invalid. Treating obtain failures as application bugs sends engineers to microservice logs while the fix remains Caddy storage, DNS credentials, or challenge port publishing.

Common causes

These patterns drive most caddy certificate renewal failed tickets. They explain why automation logs look green while browsers or APIs still break.

Missing acme_dns provider configuration for wildcard labels. Port 80 blocked during HTTP-01 while TLS-ALPN-01 was disabled unintentionally. Forgotten capability drops after container hardening often remove CAP_NET_BIND_SERVICE needed for HTTP-01 listeners.

File descriptor limits under heavy load prevented Caddy from opening challenge listeners. System clock drift on a VM snapshot broke ACME nonce validation.

Operators pinned a manual cert path in the site block, disabling renewal for that host only. Duplicate site labels caused Caddy to skip issuance for one of two identical server names. Systemd ProtectHome prevents Caddy from reading API tokens in user home directories.

Diagnose

Validate config, then inspect logs during a single obtain attempt. Separate admin API health from automatic HTTPS on customer routes; admins may still curl localhost while users hit stale edge names.

caddy validate --config /etc/caddy/Caddyfile
journalctl -u caddy -n 100 --no-pager

Enable debug logging briefly and reproduce one failing hostname. Confirm which challenge type Caddy selected and the HTTP status Boulder would see. Cross-check uid permissions on the data directory and systemd unit ProtectSystem hardening options. Compare cgroup memory limits with ACME obtain spikes—OOM kills during obtain look like random renewal failure.

Fix step-by-step

Follow this sequence for caddy automatic https renewal failed. Capture command output after each step so the next responder can pick up without rerunning destructive changes.

1. Run caddy validate and fix JSON or Caddyfile errors.
2. Supply DNS provider credentials for wildcard or DNS challenge mode.
3. Open ports required for configured challenge type.
4. Remove static cert paths that block renewal.
5. Fix permissions on the data directory.
6. caddy reload after corrections.
7. External openssl for each customer hostname.
8. Pin upgrades through staging with test issuances.
9. Archive redacted ACME logs and openssl transcripts in the ticket before closing.
10. Schedule a follow-up dry-run before the next natural renewal window.

Verify the live certificate

OpenSSL

echo | openssl s_client -connect HOST:443 -servername HOST 2>/dev/null | openssl x509 -noout -dates -subject -issuer

Replace HOST with the public hostname on the certificate.

How to read the output

Hit the public name and compare issuer to Let's Encrypt or ZeroSSL as configured. Check Caddy admin API /config/ for certificate load paths if openssl still shows expired material after logs claim success.

curl

curl -vI --connect-timeout 10 https://HOST/ 2>&1 | grep -Ei 'expire|SSL certificate|subject:|issuer:'

Replace HOST with the public hostname users request.

Roll out and verify

Validate Caddyfile with caddy validate --config and fix DNS provider tokens. Open required ports temporarily or switch challenge types deliberately.

Run caddy reload after fixing config; watch logs for renewed assets written under data dir. For manual overrides, remove static cert paths so automatic HTTPS can resume. After reload, query admin API local listener for loaded certificate IDs matching each public site.

Export metrics or logs to alert on repeated obtain failures. Pin Caddy version upgrades through staging. Document which sites use OnDemand TLS because they need tighter monitoring.

What to automate

Alert on repeated obtain failures in Caddy logs before notAfter crosses into user-visible expiry. Stream Caddy obtain error metrics tagged by site name from structured logs.

ssldays monitors the live chain Caddy presents, not only admin API health. ssldays TLS checks validate automatic HTTPS outcomes per public hostname.