Harbor registry SSL certificate expired

x509: certificate has expired on docker pull points at Harbor before your app—even when harbor ssl certificate expired is only on the notary or chartmuseum hostname, main UI cert may still look valid in a careless spot check.

Harbor fronts the OCI/Docker v2 API, Notary (content trust), optional ChartMuseum, and portal UI via nginx or an upstream ingress controller. Each endpoint can terminate TLS on different secrets depending on harbor.yml and Helm expose settings.

Kubernetes teams renew the ingress cert for harbor.example.com while the in-cluster harbor-nginx certificate for internal replication still expired—replication jobs fail east-west with TLS errors unrelated to developer laptops.

First response

Record failing command (docker login, helm push, ctr pull), exact registry FQDN, and whether content trust is enabled. OpenSSL each hostname clients use:

echo | openssl s_client -connect harbor.example.com:443 -servername harbor.example.com 2>/dev/null | openssl x509 -noout -dates -subject
echo | openssl s_client -connect notary.harbor.example.com:443 -servername notary.harbor.example.com 2>/dev/null | openssl x509 -noout -dates -subject

For Helm installs: kubectl get secret -n harbor and note core.tls, notary-server, ingress TLS secret names.

What this error means

Container runtimes verify the registry HTTPS certificate before WWW-Authenticate and token exchange. Expired leaf aborts pulls across every cluster using that registry mirror.

Notary signs tags when content trust is on. Expired notary TLS breaks docker trust workflows even when pulls without trust succeed. ChartMuseum at chartmuseum.harbor.example.com is another common forgotten SAN.

Harbor internal components (jobservice, registryctl) may use auto-generated long-lived certs—still check if you customized internal_tls.

Platform-wide outages ripple fast: every node pulling on five-minute intervals logs handshake failures simultaneously, flooding observability with red herring OCI errors unrelated to image content.

Harbor proxy-cache mode still terminates TLS to downstream clients even when upstream registry TLS is healthy. Operators must separate local listener expiry from upstream fetch errors when diagnosing pull failures.

Robot accounts and OIDC login succeed only after TLS handshake completes. Teams that focus on token rotation during incidents waste hours when the listener cert expired first.

Common causes

cert-manager renewed harbor-tls in Kubernetes but Harbor core still mounts previous secret until pod restart. Installer placed renewed PEM on host path while docker-compose nginx bind mount cached old inode.

Split DNS: external users hit expired cert on public LB; internal CI uses insecure-registry drift hiding expiry until a new runner enforces verify. Let’s Encrypt DNS challenge succeeded for apex only, not notary subdomain.

Harbor upgrade rewrote harbor.yml to default self-signed; operators assumed Helm upgrade preserved commercial cert paths.

Replication policies between Harbor instances validate peer TLS on both directions; renewing destination only leaves nightly sync jobs failing with PKIX errors that never touch developer docker pull URLs.

Trivy and scanner adapters that pull from Harbor over HTTPS inherit the same listener identity; security dashboards show scan stalls coinciding with registry cert lapses.

Air-gapped installs with long-lived commercial certs sometimes forget chart museum and notary hostnames on the initial SAN purchase, forcing emergency reissue under production pull load.

Confirm from the outside

docker-style curl

curl -vI https://harbor.example.com/v2/ 2>&1 | grep -Ei 'expire|SSL certificate|401'

Expect 401 Unauthorized after valid TLS—not SSL errors.

crane / skopeo

crane catalog harbor.example.com 2>&1 | head

OpenSSL all published hosts

for h in harbor.example.com notary.harbor.example.com chartmuseum.harbor.example.com; do
  echo "== $h =="
  echo | openssl s_client -connect "$h:443" -servername "$h" 2>/dev/null | openssl x509 -noout -dates
done

How to read the output

Any hostname in daemon insecure-registries list that still must verify needs future notAfter. Chain must include issuer intermediates Harbor ships as ca.crt for docker trust configuration.

Test from a cluster node that recently joined the fleet—new nodes without legacy insecure-registry JSON enforce verify and reproduce failures older nodes hide.

Correlate notAfter on each host with kubectl describe certificate resources when cert-manager manages rotation; Ready=True without pod rollout leaves live mismatch.

Fix checklist

1. Map Harbor components to cert files or K8s secrets from harbor.yml / Helm values.
2. Reissue cert covering core, notary, and chart FQDNs (wildcards acceptable if policy allows).
3. Update secret or /data/cert/server.crt + key; run docker compose restart nginx or rollout Harbor pods.
4. If using cert-manager, ensure Certificate references correct DNS names; wait for Ready=True.
5. Restart harbor-core, registry, notary-server if they cache TLS context.
6. Re-run docker login, signed pull with trust, helm push to chart museum.
7. Remove temporary insecure-registry daemon.json entries on nodes.
8. Document secret name and renewal owner in platform wiki.

Roll Harbor workloads in dependency order when notary and core share nodes: notary signing outages block content trust pulls even after core nginx reload succeeds.

After secret update, watch harbor-core logs for TLS handshake errors from replication jobs and cancel stuck replication executions that retry with backoff against expired peer URLs.

Validate helm OCI and docker v2 separately—they may hit different ingress rules on multi-host installs.

For docker-compose labs promoted to production, confirm host bind paths on nginx container match host filesystem renewal targets—inode caching caused many “we updated the cert” false positives.

When to use automation

Clusters pull images every few minutes—harbor ssl certificate expired is a multi-cluster outage. Monitor live TLS on every Harbor-facing DNS name, including notary and chartmuseum.

/check each FQDN from the public Internet. For in-cluster registries, add synthetic crane or curl /v2/ probes that verify certificates. See Docker registry TLS expired for generic registry patterns.