Istio ingress gateway certificate expired

istio certificate expired on the ingress gateway means istio-proxy is presenting a downstream leaf past notAfter—often while sidecars and east-west mTLS still look healthy.

Cluster operators notice this when only north-south traffic fails: browsers, partner webhooks, and mobile apps break while in-mesh RPC succeeds. GitOps repos may show a renewed Secret, but the gateway Deployment never picked up the new version.

Treat istio certificate expired as a listener and SDS problem first. Renewed PEM files in Git, vaulted artifacts, or cert-manager archives only matter once the ingress Gateway resource references current material and gateway pods load it.

Istio revision upgrades complicate expiry drills: multiple control plane revisions can leave inactive ingress deployments bound to deprecated Secrets while active revision gateways look healthy in Grafana. Inventory ingress deployments per revision label before replacing PEMs. When GitOps reconcilers manage Gateway objects, verify the live cluster object matches the merged manifest—Argo CD diff often shows Secret reference drift operators ignored for weeks.

istio certificate expired incidents on multi-tenant clusters need tenant hostname lists attached to each Gateway credentialName. One tenant’s overdue OV cert must not block blameless postmortems on shared platform teams if the Gateway selector was wrong.

What to check

Map which Gateway and VirtualService hostnames terminate TLS on istio-ingressgateway (or your custom ingress deployment). Mesh CA certificates for workload identities use a different PKI tree than public HTTPS credentials.

Check whether TLS uses credentialName Secret references, automatic SDS from file mounts, or Gateway API resources in newer Istio versions. Each path has distinct reload semantics.

Verify DNS and cloud load balancers in front of the cluster. An expired cert at the cloud edge mimics an istio certificate expired symptom even when the in-cluster Secret is valid.

Capture whether cert-manager manages the Secret or operators rotate commercial OV files manually—renewal playbooks diverge immediately after that answer.

If the ingress sits behind a cloud WAF, collect whether the WAF presents its own certificate or passes through to the cluster. Misattributed ownership lengthens outages when Kubernetes engineers optimize Secrets that never reached the client path. Application teams using EXTERNAL_NAME services may still rely on the same Gateway hostname for partner callbacks—include those URLs in handshake tests.

Step-by-step

External openssl against the user-facing name

HOST=api.example.com
echo | openssl s_client -connect "${HOST}:443" -servername "${HOST}" 2>/dev/null \
  | openssl x509 -noout -dates -subject -issuer -serial

Record serial and notAfter before any cluster changes.

Find the Gateway binding

kubectl get gateway -A -o wide
kubectl get secret -A | grep -E 'credential|tls|ingress'

Open the Gateway manifest section that sets credentialName and note namespace boundaries—Secrets must live in the same namespace as the gateway workload unless your chart documents cross-namespace copies.

Compare Secret contents to live TLS

kubectl get secret -n istio-system my-ingress-cert -o jsonpath='{.data.tls\.crt}' | base64 -d \
  | openssl x509 -noout -dates -serial

Mismatch between fresh Secret dates and stale openssl output implies pods have not reloaded SDS or a fronting load balancer still holds old material.

Renew via cert-manager or manual replacement

For cert-manager:

kubectl describe certificate -n istio-system my-ingress-cert
kubectl get certificaterequest -n istio-system

Fix Challenge failures before rolling gateways. For manual certs, replace tls.crt, tls.key, and full chain in the Secret, bump labels if your operator requires a reconcile annotation.

Roll ingress gateway pods

kubectl rollout restart deployment/istio-ingressgateway -n istio-system
kubectl rollout status deployment/istio-ingressgateway -n istio-system

On OpenShift or revision-based upgrades, target the active ingress deployment name your platform documents.

Verify again from outside the cluster

Repeat openssl and a quick HTTPS fetch from a host outside the VPC. Serial must differ from the incident-opening capture when you issued new material.

Validate cert-manager orders when ACME drives ingress

When Certificates stay in Ready=False, inspect Challenge objects and HTTP-01 solver pods before blaming Istio. DNS-01 failures against wildcard hostnames often leave istio-system Secrets untouched while operators assume mesh fault. After Challenges succeed, wait for Secret resourceVersion to bump, then roll gateways. Link troubleshooting steps to ACME guides if authorization never completes.

How to read the output

Expired notAfter on the public handshake is the acceptance criterion for closing the incident. issuer should match your Let’s Encrypt, corporate, or cloud CA policy—unexpected issuers hint at wrong Secret attachment.

Gateway credentialName must reference a Secret containing tls.crt and tls.key in the expected format. Some teams store only the leaf without intermediates; clients may fail chain building even before expiry if intermediates were always wrong, but expiry incidents show dates in the past.

Rollout status success means Kubernetes swapped pods; it does not replace openssl proof. SDS inside istio-proxy can lag until the new pod starts and watches the Secret volume.

If multiple hostnames share a wildcard, one openssl check on a representative subdomain is enough when SAN coverage is documented.

Common mistakes

Renewing the wrong Secret namespace while the Gateway in istio-system still references the previous object name.

Updating Helm values for ingress certs without upgrading the release, leaving stale templates in the live cluster.

Testing only in-mesh curl with ISTIO_MUTUAL while external users hit SIMPLE TLS on port 443.

Forgetting cloud armor or WAF certificates when Kubernetes gateways already show fresh serials.

Disabling istio-ingressgateway autoscaling min replicas during incidents, then rolling only one pod while others serve expired leaves.

Ignoring Let's Encrypt certificate expired in Kubernetes guidance when ACME authorization failures blocked the Secret update entirely.

Applying istio-ingressgateway horizontal autoscaling without verifying every new pod mounts the latest Secret volume leaves outliers serving stale SDS snapshots during scale events. NetworkPolicy changes blocking ingress to port 15021 can stall readiness while old pods continue serving expired public certs until removed manually.

When to use automation

Ingress gateway renewal windows are shorter than annual commercial contracts. Without external checks, stale pods can survive unnoticed until the first customer in a distant region retries HTTPS.

ssldays probes the live TLS certificate presented to the internet, capturing issuer, SANs, chain behavior, and expiry—complementing in-cluster cert-manager Ready conditions. Where registries publish RDAP data, registrar expiry can sit beside TLS dates on the same hostname row. Alerts at 30, 14, 7, and 1 days give time for gateway rollouts and change windows. Email is included on all plans; Slack and Discord webhooks ship on Starter and Pro. Free monitoring covers three domains daily; Starter handles 25 hourly; Pro watches 100 hourly.

After you confirm a new serial on live check, register production hostnames at register so the next rotation is caught before browsers fail. Cross-link remediation with Envoy TLS certificate expired when you debug SDS at the proxy layer directly.