Vault PKI Secrets Engine in Production: Intermediate CAs, cert-manager, and Short-Lived Certificates as a Security Primitive

Modern service infrastructure lives or dies by its certificate hygiene. If you are still handing out one-year certificates with a vague promise to check the OCSP responder, this post is for you. We will walk through HashiCorp Vault’s PKI secrets engine — from the philosophical shift that short-lived certificates represent, through every vault write command needed to stand up a production two-tier CA hierarchy, to cert-manager integration, Vault Agent sidecar patterns, SPIFFE identity, and the operational monitoring that keeps it all honest.

Versions current as of this writing: Vault 2.0.1 (released May 2025, the first release under IBM’s major-version lifecycle policy — PKI secrets engine behavior is unchanged from the 1.x line), cert-manager 1.20.0 (March 2026).

1. Why Short-Lived Certificates Are a Security Primitive

The Revocation Problem Is Real and Largely Unsolved

The traditional X.509 revocation story has two mechanisms: Certificate Revocation Lists (CRLs) and the Online Certificate Status Protocol (OCSP). Both have deep structural problems in practice.

CRLs are files signed by the CA listing revoked serial numbers. Browsers and TLS libraries download them, cache them for the stated “next update” window (often 24–72 hours), and check locally. This means a revoked certificate remains usable for the entire cache window after revocation — potentially days. For a compromised service credential in your infrastructure, that window is unacceptable.

OCSP was designed to fix CRL latency by enabling real-time per-certificate status checks. In the web PKI it largely failed for three structural reasons:

1. Soft-fail: Most TLS implementations treat an OCSP timeout as “valid.” An attacker who blocks the OCSP endpoint gets the certificate treated as good indefinitely.
2. Privacy: Every OCSP check tells the CA exactly which certificate a user is connecting to. For public web PKI this leaked browsing behavior. For private PKI this is less critical but still an information leak.
3. Availability: The OCSP responder becomes a hard dependency in your TLS handshake path. Outages in your certificate infrastructure cascade directly to service connectivity.

OCSP Stapling improves matters — the server pre-fetches its own status and staples it to the TLS handshake — but it requires server-side configuration and the staple itself has a validity window that reintroduces the latency problem.

The Mental Model Shift: Expiry Over Revocation

The short-lived certificate model abandons the “revoke when compromised” mental model entirely. The new model is: certs expire before an attacker can exploit them.

If a certificate is valid for 24 hours, a stolen private key can be abused for at most 24 hours. If the certificate is valid for 6 hours, the window is 6 hours. This bounds the blast radius of any key compromise without requiring any revocation infrastructure to function correctly.

The CA/B Forum Baseline Requirements (as of March 2026) formally recognize this: certificates with a validity period of 7 days or fewer are classified as “short-lived” and are exempt from CRL and OCSP requirements. The standards body acknowledges that expiry is a better revocation mechanism for short-lived credentials.

For internal service-to-service communication — mTLS between microservices, Kubernetes workload identity, API-to-API authentication — this is the correct model. Issue 24-hour certificates. Rotate automatically. Never build revocation infrastructure for these certs. The compromise window is shorter than most incident response cycles anyway.

The comparison that matters: A 1-year certificate with OCSP that an attacker has exploited is protected only by infrastructure that can fail silently. A 24-hour certificate with no revocation infrastructure is protected by physics — the cert stops working in 24 hours regardless of what the attacker does.

mTLS for Service Identity

Short-lived certificates become transformative when paired with mutual TLS. In mTLS, both client and server present certificates, establishing cryptographic proof of identity in both directions. Combined with short TTLs:

• Each service instance has a unique, short-lived identity certificate.
• Compromise of one instance’s key is temporally bounded.
• Identity is cryptographic, not network-position-based (no more “trust everything from the internal network” assumptions).
• Certificate metadata (subject, SANs, issuing CA) carries verifiable identity information usable for authorization decisions.

Vault’s PKI secrets engine is purpose-built for this use case: high-throughput issuance of short-lived certificates through an authenticated API.

2. Vault PKI Secrets Engine Architecture

Enabling a PKI Mount

Vault organizes secrets engines at mount paths. A PKI mount is an isolated certificate authority — it holds a CA certificate and private key, configuration, roles, and issued certificate records. You can have multiple PKI mounts, each representing a different CA or a different level of the hierarchy.

1
2

vault secrets enable -path=pki pki
vault secrets tune -max-lease-ttl=87600h pki    # 10 years for root

The max-lease-ttl on the mount is a hard cap — no certificate issued from this mount can exceed it. This is a safety control.

Root CA vs. Intermediate CA: The Two-Tier Model

The canonical PKI hierarchy for Vault production use has three levels:

Offline Root CA  (external, never in Vault)
       |
       v
Vault Intermediate CA  (pki_int/ mount)
       |
       v
Leaf Certificates  (issued to services, max TTL hours to days)

Why you never put your root CA private key in Vault: The root CA is the ultimate trust anchor. Compromise of the root private key invalidates your entire PKI — every certificate issued, every trust relationship. The correct approach is to generate the root CA offline (air-gapped), store the private key in a hardware security module or encrypted offline storage, and only bring it online to sign the intermediate CA certificate (an infrequent operation). Vault never sees the root private key.

The intermediate CA lives in Vault. Its private key is generated inside Vault’s seal, protected by Vault’s unsealing mechanism (AWS KMS, Azure Key Vault, GCP CKMS, or Shamir seal). The intermediate CA’s private key performs the high-throughput day-to-day signing of leaf certificates. If the intermediate is compromised, you revoke it at the root (a rare, high-impact operation) and re-issue a new intermediate. The root’s isolation is preserved.

Multiple Issuers per Mount (Vault 1.11+)

Since Vault 1.11.0, a single PKI mount can contain multiple issuer certificates — different CA certs backed by different (or the same) key material. This is the mechanism for CA rotation without downtime: you add the new intermediate as an additional issuer, update roles to use it, and let old certificates expire naturally. Both issuers can sign certificates simultaneously during the transition window.

3. Full Two-Tier CA Setup: Step by Step

Step 1: Root CA (Offline or Vault-internal for labs)

Production: Use OpenSSL, CFSSL, or certstrap to generate an offline root. We show that path further below with the external root workflow.

Lab/staging: Generate a self-signed root directly in Vault.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

# Enable the root PKI mount
vault secrets enable -path=pki pki
vault secrets tune -max-lease-ttl=87600h pki    # 10 years

# Generate a self-signed root CA
vault write -field=certificate pki/root/generate/internal \
    common_name="MyOrg Root CA" \
    issuer_name="root-2026" \
    key_type="ec" \
    key_bits=384 \
    ttl=87600h \
    > root_ca_2026.crt

# Configure the CA and CRL distribution URLs
vault write pki/config/urls \
    issuing_certificates="${VAULT_ADDR}/v1/pki/ca" \
    crl_distribution_points="${VAULT_ADDR}/v1/pki/crl" \
    ocsp_servers="${VAULT_ADDR}/v1/pki/ocsp"

Step 2: Intermediate CA in Vault

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# Enable the intermediate PKI mount
vault secrets enable -path=pki_int pki
vault secrets tune -max-lease-ttl=43800h pki_int    # 5 years max

# Generate the intermediate CSR (key stays inside Vault)
vault write -format=json pki_int/intermediate/generate/internal \
    common_name="MyOrg Intermediate CA 2026" \
    issuer_name="myorg-intermediate-2026" \
    key_type="ec" \
    key_bits=384 \
    | jq -r '.data.csr' > pki_intermediate.csr

Step 3: Sign the Intermediate with the Root

If your root is in Vault (lab path):

1
2
3
4
5
6
7
8

vault write -format=json pki/root/sign-intermediate \
    issuer_ref="root-2026" \
    csr=@pki_intermediate.csr \
    common_name="MyOrg Intermediate CA 2026" \
    format=pem_bundle \
    path_length=0 \
    ttl="43800h" \
    | jq -r '.data.certificate' > intermediate_signed.pem

If your root is an external/offline root (production path):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# Using certstrap for the offline root:
certstrap init \
    --organization "MyOrg" \
    --common-name "MyOrg Root CA" \
    --expires "10 year" \
    --key-bits 4096

# Sign the Vault-generated CSR with the offline root:
certstrap sign \
    --expires "5 year" \
    --csr pki_intermediate.csr \
    --cert intermediate_signed.pem \
    --intermediate \
    --path-length "0" \
    --CA "MyOrg Root CA" \
    "MyOrg Intermediate CA 2026"

# Concatenate the chain (intermediate + root) for import:
cat intermediate_signed.pem root_ca.pem > intermediate_chain.pem

Step 4: Import the Signed Intermediate into Vault

1
2

vault write pki_int/intermediate/set-signed \
    certificate=@intermediate_chain.pem

Step 5: Configure Intermediate CA URLs

1
2
3
4

vault write pki_int/config/urls \
    issuing_certificates="${VAULT_ADDR}/v1/pki_int/ca" \
    crl_distribution_points="${VAULT_ADDR}/v1/pki_int/crl" \
    ocsp_servers="${VAULT_ADDR}/v1/pki_int/ocsp"

Step 6: Configure CRL Behavior

1
2
3
4
5
6
7
8

vault write pki_int/config/crl \
    expiry="72h" \
    auto_rebuild=true \
    auto_rebuild_grace_period="12h" \
    enable_delta=true \
    delta_rebuild_interval="15m" \
    ocsp_disable=false \
    ocsp_expiry="12h"

On CRL for short-lived certificates: If your maximum certificate TTL is 24 hours, the case for maintaining revocation infrastructure weakens considerably. A revoked 24-hour cert will expire before most CRL caches would have refreshed anyway. For pure service mesh / mTLS use cases where all certs are sub-24h, you can set disable=true in the CRL config. The trade-off: you lose the ability to do emergency revocation, but you gain operational simplicity and eliminate the CRL as a failure point. Make this decision explicitly, per environment.

Step 7: Enable Auto-Tidy

The tidy operation cleans up expired certificates from storage and rebuilds the CRL to remove entries for already-expired certs. Without auto-tidy, your storage fills with expired cert records and your CRL grows unboundedly.

1
2
3
4
5
6
7

vault write pki_int/config/auto-tidy \
    enabled=true \
    interval_duration="12h" \
    tidy_cert_store=true \
    tidy_revoked_certs=true \
    tidy_expired_issuers=false \
    safety_buffer="72h"

You can also trigger tidy manually:

1
2
3

vault write pki_int/tidy \
    tidy_cert_store=true \
    tidy_revoked_certs=true

4. PKI Roles: The Policy Layer for Certificate Issuance

Vault PKI roles define the policy for certificate issuance. Every vault write pki_int/issue/<role-name> call is validated against the matching role. Roles are the primary access control surface between “authenticated Vault client” and “certificate with these properties.”

Role Reference

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

vault write pki_int/roles/my-service-role \
    # --- Identity scope ---
    issuer_ref="$(vault read -field=default pki_int/config/issuers)" \
    allowed_domains="svc.cluster.local,internal.example.com" \
    allow_subdomains=true \
    allow_bare_domains=false \
    allow_glob_domains=false \
    allow_wildcard_certificates=false \
    \
    # --- Subject Alternative Names ---
    allowed_uri_sans="spiffe://cluster.local/*" \
    allow_ip_sans=true \
    allowed_ip_sans="" \        # empty = any IP allowed if allow_ip_sans=true
    \
    # --- Key material ---
    key_type="ec" \
    key_bits=256 \
    \
    # --- Validity ---
    ttl="24h" \
    max_ttl="72h" \
    \
    # --- Certificate profile ---
    server_flag=true \
    client_flag=true \
    code_signing_flag=false \
    require_cn=false \
    enforce_hostnames=true \
    \
    # --- Performance: critical for high-volume issuance ---
    generate_lease=false \
    no_store=false

Field-by-field breakdown of the important ones:

The generate_lease=false + no_store=true combination is the recommended configuration for high-frequency service certificate issuance. Your audit log becomes the record of issuance; Vault storage is not the source of truth for which certs exist.

5. Issuing Certificates

CLI Issuance

1
2
3
4
5
6

vault write pki_int/issue/my-service-role \
    common_name="api.svc.cluster.local" \
    alt_names="api.svc.cluster.local,api-v2.svc.cluster.local" \
    ip_sans="10.96.0.10" \
    uri_sans="spiffe://cluster.local/ns/default/sa/api-service" \
    ttl="24h"

Response fields:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

{
  "data": {
    "certificate":   "-----BEGIN CERTIFICATE-----\n...",
    "issuing_ca":    "-----BEGIN CERTIFICATE-----\n...",
    "ca_chain":      ["-----BEGIN CERTIFICATE-----\n...", "..."],
    "private_key":   "-----BEGIN EC PRIVATE KEY-----\n...",
    "private_key_type": "ec",
    "serial_number": "1a:2b:3c:...",
    "expiration":    1748390400
  }
}

• certificate: The leaf certificate for the requester.
• issuing_ca: The intermediate CA certificate.
• ca_chain: Full chain from leaf to root (use this for TLS configuration that needs the full chain).
• private_key: Returned once and never stored by Vault. The requester must persist it.
• serial_number: Hex-encoded serial. Use this for revocation.
• expiration: Unix timestamp. Useful for automation to know when to renew.

API Issuance

1
2
3
4

curl --header "X-Vault-Token: ${VAULT_TOKEN}" \
     --request POST \
     --data '{"common_name":"api.svc.cluster.local","ttl":"24h"}' \
     "${VAULT_ADDR}/v1/pki_int/issue/my-service-role"

The sign vs. sign-verbatim Endpoints

When callers generate their own key pair and submit a CSR instead of having Vault generate the key:

pki_int/sign/<role-name> — The CSR subject and SANs are validated against the role policy. Use this for external CSRs that must still respect domain and SAN constraints. The role’s key type and bits are advisory when signing external CSRs.

pki_int/sign-verbatim — Accepts the CSR with minimal role validation. It will use the SANs from the CSR essentially as-is. Useful for one-off tooling but dangerous: it bypasses domain restrictions. Restrict access to this endpoint via Vault policy to specific break-glass use cases only. Note: As of Vault 2.0, sign-verbatim no longer ignores the BasicConstraints extension in CSRs — if isCA=true in the CSR, Vault now returns an error.

6. cert-manager Vault Issuer

cert-manager is the standard Kubernetes-native certificate lifecycle manager. Its Vault issuer integrates directly with the PKI secrets engine.

Vault Side: Auth and Policy Setup

cert-manager needs a Vault identity with permission to call the PKI signing endpoint. The recommended approach is Kubernetes auth — cert-manager authenticates to Vault using its Kubernetes service account token, which Vault validates via the Kubernetes TokenReview API.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

# Enable Kubernetes auth
vault auth enable kubernetes

# Configure the Kubernetes auth backend
vault write auth/kubernetes/config \
    kubernetes_host="https://kubernetes.default.svc:443"

# Create a Vault policy for cert-manager
vault policy write cert-manager-pki - <<'EOF'
# Allow reading PKI CA certs and CRL
path "pki_int/ca" {
  capabilities = ["read"]
}
path "pki_int/ca_chain" {
  capabilities = ["read"]
}
path "pki_int/crl" {
  capabilities = ["read"]
}
# Allow signing certificates via specific roles
path "pki_int/sign/my-service-role" {
  capabilities = ["create", "update"]
}
path "pki_int/sign/web-service-role" {
  capabilities = ["create", "update"]
}
EOF

# Create a Kubernetes auth role binding cert-manager's SA to the policy
vault write auth/kubernetes/role/cert-manager \
    bound_service_account_names=cert-manager \
    bound_service_account_namespaces=cert-manager \
    policies=cert-manager-pki \
    ttl=20m \
    token_max_ttl=1h

ClusterIssuer YAML

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: vault-intermediate-issuer
spec:
  vault:
    server: https://vault.vault.svc.cluster.local:8200
    path: pki_int/sign/my-service-role
    # Base64-encoded Vault CA cert (needed if Vault's TLS cert is self-signed
    # or signed by your internal CA — which it should be in production)
    caBundle: <base64-encoded-vault-tls-ca>
    auth:
      kubernetes:
        role: cert-manager
        mountPath: /v1/auth/kubernetes
        serviceAccountRef:
          name: cert-manager    # the cert-manager controller's SA

For AppRole auth (useful for off-cluster tooling):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

spec:
  vault:
    server: https://vault.example.com:8200
    path: pki_int/sign/my-service-role
    caBundle: <base64-ca>
    auth:
      appRole:
        path: approle
        roleId: "291b9d21-8ff5-48ee-a14d-4b1b27eb9cf8"
        secretRef:
          name: cert-manager-vault-approle-secret
          key: secretId

Create the AppRole secret:

1
2
3

kubectl create secret generic cert-manager-vault-approle-secret \
    --namespace cert-manager \
    --from-literal=secretId="${VAULT_APPROLE_SECRET_ID}"

Certificate Resource

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: api-service-tls
  namespace: default
spec:
  secretName: api-service-tls-secret
  issuerRef:
    name: vault-intermediate-issuer
    kind: ClusterIssuer
    group: cert-manager.io
  commonName: api.svc.cluster.local
  dnsNames:
    - api.svc.cluster.local
    - api-v2.svc.cluster.local
  uriSANs:
    - spiffe://cluster.local/ns/default/sa/api-service
  duration: 24h        # requested cert lifetime
  renewBefore: 8h      # cert-manager renews this many hours before expiry
                       # (default: renew at 2/3 of lifetime)
  privateKey:
    algorithm: ECDSA
    size: 256
    rotationPolicy: Always    # rotate the private key on every renewal

The resulting api-service-tls-secret is a standard Kubernetes TLS secret with keys tls.crt, tls.key, and ca.crt. cert-manager will automatically renew it before renewBefore elapses.

cert-manager renewal timing: By default, cert-manager targets renewal at the 2/3 lifetime point. For a 24-hour certificate, renewal is triggered at hour 16. Setting renewBefore: 8h achieves the same: renewal triggered when 8 hours remain on a 24-hour cert.

7. Auto-Rotation Without cert-manager

Vault Agent Sidecar (Standalone)

For non-Kubernetes environments or cases where you want Vault Agent to manage certificates directly, use the template stanza with the pkiCert function.

Vault Agent configuration file (vault-agent.hcl):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

vault {
  address = "https://vault.example.com:8200"
}

auto_auth {
  method "aws" {
    mount_path = "auth/aws"
    config = {
      type = "iam"
      role = "my-service-role"
    }
  }
  sink "file" {
    config = {
      path = "/tmp/.vault-token"
    }
  }
}

template_config {
  static_secret_render_interval = "5m"
  exit_on_retry_failure         = true
}

# Render all three files from a single PKI call
template {
  contents = <<EOT
{{- with pkiCert "pki_int/issue/my-service-role" "common_name=api.svc.internal" "ttl=24h" -}}
{{- .Data.Key | writeToFile "/etc/ssl/private/service.key" "root" "root" "0600" -}}
{{- .Data.Cert | writeToFile "/etc/ssl/certs/service.crt" "root" "root" "0644" -}}
{{- .Data.CA | writeToFile "/etc/ssl/certs/ca.crt" "root" "root" "0644" -}}
{{- end -}}
EOT
  destination = "/dev/null"    # writeToFile handles actual output
  exec {
    command = ["systemctl", "reload", "my-service"]
    timeout = "30s"
  }
}

The pkiCert function (introduced in Vault Agent as of Vault 1.11, preferred over the older secret function for PKI) fetches a new certificate on startup if none exists or if the existing one has expired. It tracks the certificate expiration and requests a renewal at roughly the 90% mark of the cert’s lifetime.

Vault Agent Injector in Kubernetes

The Vault Agent Injector operates as a mutating admission webhook. Annotate your Pod spec:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-service
spec:
  template:
    metadata:
      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/role: "my-k8s-vault-role"
        # Define the "secret" path (triggers the inject mechanism)
        vault.hashicorp.com/agent-inject-secret-tls: "pki_int/issue/my-service-role"
        # Define the template for rendering
        vault.hashicorp.com/agent-inject-template-tls: |
          {{- with pkiCert "pki_int/issue/my-service-role" "common_name=my-service.default.svc.cluster.local" "ttl=24h" -}}
          {{- .Data.Key  | writeToFile "/vault/secrets/tls.key"  "vault" "vault" "0600" -}}
          {{- .Data.Cert | writeToFile "/vault/secrets/tls.crt"  "vault" "vault" "0644" -}}
          {{- .Data.CA   | writeToFile "/vault/secrets/ca.crt"   "vault" "vault" "0644" -}}
          {{- end -}}
    spec:
      serviceAccountName: my-service
      containers:
      - name: my-service
        image: my-service:latest
        volumeMounts:
        - name: vault-secrets
          mountPath: /vault/secrets

The injector adds an init container (runs before your app, ensures certs are present at startup) and a sidecar container (keeps running, renews certs before they expire). Application notification on renewal is handled via the command option in the template stanza — add it to the agent ConfigMap if you need SIGHUP or a reload command.

8. SPIFFE/SPIRE Integration

Vault PKI as a SPIFFE CA

SPIFFE (Secure Production Identity Framework For Everyone) defines a standard for workload identity via X.509 SVIDs — certificates whose URI SAN follows the pattern spiffe://<trust-domain>/<workload-path>.

To issue SPIFFE SVIDs from Vault PKI:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# Create a role that allows SPIFFE URI SANs
vault write pki_int/roles/spiffe-issuer \
    allowed_domains="svc.cluster.local" \
    allow_subdomains=true \
    allowed_uri_sans="spiffe://cluster.local/*" \
    allowed_uri_sans_template=true \
    require_cn=false \
    key_type="ec" \
    key_bits=256 \
    ttl="1h" \
    max_ttl="4h" \
    server_flag=false \
    client_flag=true \
    generate_lease=false \
    no_store=true

Issue a SPIFFE SVID:

1
2
3

vault write pki_int/issue/spiffe-issuer \
    uri_sans="spiffe://cluster.local/ns/default/sa/my-service" \
    ttl="1h"

Vault PKI vs. Running SPIRE

When to use Vault PKI for SPIFFE: You already run Vault, your identity requirements map cleanly to existing Vault auth methods, and you do not need SPIFFE federation between trust domains or the advanced workload attestation that SPIRE provides.

When to use SPIRE: You need multi-cluster federation, hardware-backed node attestation (TPM), or the richer workload registration model. SPIRE can use Vault PKI as its upstream CA — a common hybrid pattern where SPIRE handles attestation and workload registration, but Vault PKI is the certificate authority.

Vault 2.0 added native SPIFFE JWT-SVID support: authenticated workloads can now request JWT-SVIDs from Vault directly, enabling the full SPIFFE identity model without SPIRE for environments that are already Vault-native.

9. The Revocation Story End to End

CRL Configuration Deep Dive

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

# View current CRL config
vault read pki_int/config/crl

# Production configuration
vault write pki_int/config/crl \
    expiry="72h" \
    disable=false \
    auto_rebuild=true \
    auto_rebuild_grace_period="12h" \
    enable_delta=true \
    delta_rebuild_interval="15m" \
    ocsp_disable=false \
    ocsp_expiry="12h" \
    cross_cluster_revocation=false \
    unified_crl=false

# For short-lived-only workloads (max_ttl <= 24h), consider:
vault write pki_int/config/crl disable=true

Delta CRLs (available since Vault 1.12, now stable): Instead of rebuilding the complete CRL on every revocation, delta CRLs are small incremental additions since the last full CRL. For environments with active revocations, this reduces the CRL rebuild from an O(n) operation to O(delta). The delta_rebuild_interval controls how frequently the delta CRL is rebuilt; it defaults to 15 minutes.

Revoking a Certificate

1
2
3
4
5
6

# Revoke by serial number
vault write pki_int/revoke \
    serial_number="1a:2b:3c:4d:..."

# Force rotate the CRL immediately after revocation
vault write pki_int/crl/rotate

OCSP Responder

Vault includes a built-in OCSP responder at pki_int/ocsp. Configure it in the URLs:

1
2

vault write pki_int/config/urls \
    ocsp_servers="${VAULT_ADDR}/v1/pki_int/ocsp"

The OCSP responder handles DER-encoded single-serial requests per RFC 6960. Limitations: one serial per request; Ed25519-signed certs are not OCSP-checkable (Ed25519 is not supported by the RFC for OCSP signatures).

The “Don’t Build Revocation Infrastructure for Short-Lived Certs” Case

For a concrete example: your service certificates have max_ttl=24h. An attacker steals a private key at T+0. The certificate expires at T+24h. The CA/B Forum considers 7-day certs “short-lived” and exempts them from revocation requirements. At 24 hours, you are well inside that window.

Even if your CRL update interval is 72 hours and your OCSP staple is 12 hours old, the cert expires before you would have caught it via either revocation path. The attacker’s window is bounded by the certificate lifetime, not by your revocation infrastructure’s response time.

For these workloads: set generate_lease=false, no_store=true, consider disable=true in CRL config, and invest the saved operational energy in ensuring your rotation automation is reliable.

10. Cross-Cluster and Cross-Environment PKI

Sharing One Intermediate CA Across Clusters

The simplest multi-cluster pattern is to use the same Vault cluster (or Vault Enterprise performance replication) and the same pki_int mount for all clusters. Each cluster’s cert-manager or Vault Agent authenticates to Vault via its own Kubernetes auth backend (or separate AppRole credentials) and issues certificates from the shared intermediate.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# Kubernetes auth for cluster-1
vault write auth/kubernetes-cluster1/config \
    kubernetes_host="https://k8s-cluster1-api:6443" \
    kubernetes_ca_cert=@cluster1-ca.pem

# Kubernetes auth for cluster-2
vault write auth/kubernetes-cluster2/config \
    kubernetes_host="https://k8s-cluster2-api:6443" \
    kubernetes_ca_cert=@cluster2-ca.pem

# Both clusters can use the same PKI roles, or separate roles per cluster
vault write auth/kubernetes-cluster1/role/cert-manager \
    bound_service_account_names=cert-manager \
    bound_service_account_namespaces=cert-manager \
    policies=cert-manager-pki \
    ttl=20m

Separate Intermediates per Environment

For stronger isolation between dev/staging/prod, use separate intermediate CAs — each issued from the same offline root but living in separate Vault mounts (or separate Vault clusters).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Separate mounts
vault secrets enable -path=pki_int_dev pki
vault secrets enable -path=pki_int_staging pki
vault secrets enable -path=pki_int_prod pki

# Each gets its own intermediate signed by the offline root
# with appropriate max TTLs:
vault secrets tune -max-lease-ttl=43800h pki_int_prod
vault secrets tune -max-lease-ttl=8760h  pki_int_staging
vault secrets tune -max-lease-ttl=720h   pki_int_dev

This gives you:

• Different max TTLs per environment (prod can issue up to 90-day certs; dev is limited to 30 days).
• Independent revocation: revoking the staging intermediate does not affect prod.
• Separate audit trails per environment.
• Separate policies: dev teams can have more permissive issuance rights on pki_int_dev without touching prod.

Vault Enterprise: Namespace Isolation

Vault Enterprise namespaces provide tenant isolation within a single Vault cluster. Each namespace has its own auth methods, policies, and secrets engine mounts.

1
2
3
4
5
6
7

# Create a namespace for each business unit or environment
vault namespace create payments
vault namespace create platform-eng

# Each namespace gets its own intermediate CA
VAULT_NAMESPACE=payments vault secrets enable -path=pki_int pki
VAULT_NAMESPACE=platform-eng vault secrets enable -path=pki_int pki

Performance replication in Vault Enterprise: PKI mounts store issued certificates locally per cluster by default (not replicated to secondaries). Roles and configuration are replicated. Enable cross_cluster_revocation=true and unified_crl=true in your CRL config if you need revocations on one cluster to propagate to all clusters — but note this requires a primary cluster to be the coordination point.

11. Monitoring and Operations

Vault PKI Telemetry Metrics

Vault emits the following PKI-specific tidy metrics (via Prometheus or StatsD):

General Vault issuance metrics are captured via the audit log and the vault.core.handle_request latency metrics rather than per-secrets-engine counters.

The vault pki health-check Command

Run this regularly in CI or as a monitoring job:

1
2
3
4
5

# Run all health checks against your intermediate CA mount
vault pki health-check pki_int

# Run with a config file to set custom thresholds
vault pki health-check -health-config=pki-health-config.json pki_int

Example config file (pki-health-config.json):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

{
  "crl_validity_period": {
    "enabled": true,
    "crl_expiry_pct_critical": 95,
    "crl_minimum_expiry": "24h"
  },
  "ca_validity_period": {
    "enabled": true,
    "critical_days": 30,
    "warn_days": 90
  },
  "role_no_store_false": {
    "enabled": true
  },
  "enable_auto_tidy": {
    "enabled": true,
    "interval_duration_critical": "168h"
  },
  "too_many_certs": {
    "enabled": true,
    "count_critical": 250000,
    "count_warning": 50000
  }
}

The health check exits with a non-zero code when issues are found, making it directly usable in monitoring pipelines. Key checks:

• ca_validity_period: Warn at 12 months remaining on intermediate, critical at 30 days.
• crl_validity_period: Warn when CRL is 80% through its validity period; critical at 95%. This is the primary CRL expiry alerting mechanism.
• role_no_store_false: Flags any role with no_store=false for review.
• too_many_certs: Alerts when the cert store grows to performance-impacting levels.

External Certificate Expiry Monitoring

For fleet-wide certificate expiry visibility:

cert-manager Prometheus metrics: cert-manager exposes certmanager_certificate_expiration_timestamp_seconds per Certificate resource. Alert on certificates expiring within your renewal buffer.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

# Prometheus alerting rule
groups:
- name: cert-manager
  rules:
  - alert: CertificateExpiringSoon
    expr: |
      certmanager_certificate_expiration_timestamp_seconds
        - time() < 86400 * 3
    for: 1h
    labels:
      severity: warning
    annotations:
      summary: "Certificate {{ $labels.name }} in {{ $labels.namespace }} expires in < 3 days"

  - alert: CertificateNotReady
    expr: certmanager_certificate_ready_status{condition="False"} == 1
    for: 10m
    labels:
      severity: critical
    annotations:
      summary: "Certificate {{ $labels.name }} in {{ $labels.namespace }} is not ready"

vault-pki-exporter: An open-source Prometheus exporter (github.com/aarnaud/vault-pki-exporter) that queries Vault PKI mounts and exposes:

• x509_cert_expiry: Time until expiry for issued certs stored in Vault.
• x509_crl_expiry: Time until CRL expiry — the primary alerting target for “CRL is about to expire and needs rotation.”

Vault audit log analysis: With no_store=true roles, the audit log is your certificate inventory. Stream audit logs to your SIEM or log platform and build queries against vault.audit.request.path matching pki_int/issue/* to track issuance volume and certificate lifetimes.

Listing Issued Certificates

If no_store=false, you can enumerate issued certificates:

1
2

vault list pki_int/certs
vault read pki_int/cert/<serial-number>

For no_store=true roles, this list will not include those certificates.

12. Security Hardening

Vault Seal Protection

The intermediate CA private key is encrypted by Vault’s seal. In a properly configured production deployment:

• Auto-unseal with cloud KMS: AWS KMS, Azure Key Vault, or GCP Cloud KMS holds the unseal key. The intermediate CA key is never exposed to a human operator.
• Audit logging: Every certificate issuance (and every Vault operation) is written to the audit log before the response is sent. There is no way to issue a certificate from Vault without an audit record.
• Seal protects at rest: If someone steals the Vault storage backend (Raft snapshots, Consul data), they cannot decrypt the intermediate private key without the unseal key from the KMS.

Vault Policies for PKI Access

The principle of least privilege for PKI:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

# Policy: service-cert-issuer
# Grants a specific application the right to issue certs from one role only

path "pki_int/issue/my-service-role" {
  capabilities = ["create", "update"]
}

path "pki_int/ca" {
  capabilities = ["read"]
}

path "pki_int/ca_chain" {
  capabilities = ["read"]
}

# Do NOT grant:
# - pki_int/root/sign-intermediate (allows creating sub-CAs)
# - pki_int/sign-verbatim (bypasses role constraints)
# - pki_int/config/* (admin path)
# - pki_int/roles/* (role management)

Separate policies for operators:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# Policy: pki-admin
# For CI/CD pipelines or operators managing the PKI mount

path "pki_int/config/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

path "pki_int/roles/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}

path "pki_int/crl/rotate" {
  capabilities = ["create", "update"]
}

path "pki_int/tidy" {
  capabilities = ["create", "update"]
}

path "pki_int/revoke" {
  capabilities = ["create", "update"]
}

# Explicitly deny the sign-verbatim endpoint even for admins
path "pki_int/sign-verbatim" {
  capabilities = ["deny"]
}

Rotating the Intermediate CA

When the intermediate CA approaches expiration (or as a proactive security rotation):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

# Step 1: Generate a new intermediate CSR in the same mount
vault write -format=json pki_int/intermediate/generate/internal \
    common_name="MyOrg Intermediate CA 2028" \
    issuer_name="myorg-intermediate-2028" \
    key_type="ec" \
    key_bits=384 \
    | jq -r '.data.csr' > new_intermediate.csr

# Step 2: Sign with the offline root (same as initial setup)
certstrap sign \
    --expires "5 year" \
    --csr new_intermediate.csr \
    --cert new_intermediate_signed.pem \
    --intermediate \
    --path-length "0" \
    --CA "MyOrg Root CA" \
    "MyOrg Intermediate CA 2028"

# Step 3: Import and name the new issuer
cat new_intermediate_signed.pem root_ca.pem > new_intermediate_chain.pem
vault write pki_int/intermediate/set-signed \
    certificate=@new_intermediate_chain.pem

# Step 4: List issuers to get the new issuer ref
vault list pki_int/issuers

# Step 5: Update roles to prefer the new issuer
vault write pki_int/roles/my-service-role \
    issuer_ref="<new-issuer-uuid>"

# Step 6: Old certificates issued by the previous issuer continue to work
# until they expire (remember, they are short-lived). No emergency action needed.

# Step 7: After all old certs have expired, archive the old issuer
vault write pki_int/issuer/<old-issuer-uuid> \
    issuer_name="myorg-intermediate-2026-archived"

The multi-issuer architecture (Vault 1.11+) means CA rotation does not require a maintenance window. Old and new issuers coexist; roles transition to the new issuer; old certs naturally expire. For 24-hour certs, the entire fleet rotates to the new issuer within one day.

Quick Reference: Full Setup Script

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90

#!/usr/bin/env bash
# Full two-tier Vault PKI setup (Vault-internal root, for lab/staging)
# For production: replace Step 1 with offline root and external signing.
set -euo pipefail

VAULT_ADDR="${VAULT_ADDR:?}"
DOMAIN="${DOMAIN:-example.internal}"

# ── Root CA ──────────────────────────────────────────────────────────────────
vault secrets enable -path=pki pki
vault secrets tune -max-lease-ttl=87600h pki

vault write -field=certificate pki/root/generate/internal \
    common_name="$DOMAIN Root CA" \
    issuer_name="root-$(date +%Y)" \
    key_type="ec" key_bits=384 \
    ttl=87600h > /tmp/root_ca.crt

vault write pki/config/urls \
    issuing_certificates="$VAULT_ADDR/v1/pki/ca" \
    crl_distribution_points="$VAULT_ADDR/v1/pki/crl"

# ── Intermediate CA ──────────────────────────────────────────────────────────
vault secrets enable -path=pki_int pki
vault secrets tune -max-lease-ttl=43800h pki_int

vault write -format=json pki_int/intermediate/generate/internal \
    common_name="$DOMAIN Intermediate CA" \
    issuer_name="intermediate-$(date +%Y)" \
    key_type="ec" key_bits=384 \
    | jq -r '.data.csr' > /tmp/pki_int.csr

vault write -format=json pki/root/sign-intermediate \
    issuer_ref="root-$(date +%Y)" \
    csr=@/tmp/pki_int.csr \
    format=pem_bundle \
    path_length=0 \
    ttl="43800h" \
    | jq -r '.data.certificate' > /tmp/intermediate_signed.pem

vault write pki_int/intermediate/set-signed \
    certificate=@/tmp/intermediate_signed.pem

vault write pki_int/config/urls \
    issuing_certificates="$VAULT_ADDR/v1/pki_int/ca" \
    crl_distribution_points="$VAULT_ADDR/v1/pki_int/crl" \
    ocsp_servers="$VAULT_ADDR/v1/pki_int/ocsp"

vault write pki_int/config/crl \
    expiry="72h" \
    auto_rebuild=true \
    auto_rebuild_grace_period="12h" \
    enable_delta=true \
    delta_rebuild_interval="15m"

vault write pki_int/config/auto-tidy \
    enabled=true \
    interval_duration="12h" \
    tidy_cert_store=true \
    tidy_revoked_certs=true \
    safety_buffer="72h"

# ── Roles ────────────────────────────────────────────────────────────────────
# High-volume service role (optimized for performance)
vault write "pki_int/roles/services-$DOMAIN" \
    issuer_ref="$(vault read -field=default pki_int/config/issuers)" \
    allowed_domains="$DOMAIN,svc.cluster.local" \
    allow_subdomains=true \
    allow_bare_domains=false \
    allowed_uri_sans="spiffe://cluster.local/*" \
    key_type="ec" key_bits=256 \
    ttl="24h" max_ttl="72h" \
    server_flag=true client_flag=true \
    require_cn=false \
    generate_lease=false \
    no_store=false   # set true for >50k certs/day

# SPIFFE-only role (service mesh workload identity)
vault write "pki_int/roles/spiffe-$DOMAIN" \
    issuer_ref="$(vault read -field=default pki_int/config/issuers)" \
    allowed_uri_sans="spiffe://cluster.local/*" \
    allowed_uri_sans_template=true \
    require_cn=false \
    key_type="ec" key_bits=256 \
    ttl="1h" max_ttl="4h" \
    server_flag=false client_flag=true \
    generate_lease=false \
    no_store=true

echo "PKI hierarchy configured at pki/ (root) and pki_int/ (intermediate)"

Common Failure Modes

“certificate signed by unknown authority”: The CA chain was not included in the TLS configuration. Use ca_chain from the Vault response, not just certificate. Distribute the root CA cert to trust stores before deploying.

CRL expiry causing validation failures: If the CRL itself expires (Vault was unreachable and could not rebuild it), clients configured to enforce CRL checking will reject all certificates from that CA. Solution: auto_rebuild=true with auto_rebuild_grace_period, and monitor CRL expiry. Consider disable=true for short-lived cert workloads where you do not need revocation.

cert-manager certificate stuck in False ready state: Check kubectl describe certificate — the events section shows the Vault error. Common causes: policy does not allow the specific role path, Kubernetes auth role’s bound_service_account_namespaces does not include cert-manager’s namespace, or the Vault CA bundle in caBundle does not match what Vault is actually serving.

Storage performance degradation: Too many stored certs from no_store=false roles. Run vault pki health-check pki_int to detect too_many_certs. Enable auto-tidy and run a manual tidy immediately. For the longer term, switch high-volume roles to no_store=true.

Intermediate approaching expiration: The vault pki health-check ca_validity_period check will catch this. Vault does not automatically rotate intermediates — this is an operator action. Build a monitoring alert on intermediate CA expiry with at least a 90-day warning horizon.

Summary

The shift to short-lived certificates is not a performance optimization — it is a fundamental improvement to your security posture. The revocation problem has been structurally unsolved for 30 years; short TTLs solve it with expiry instead. Vault’s PKI secrets engine is the operational foundation for making this work at scale: high-throughput dynamic issuance, authenticated by Vault’s auth methods, constrained by roles, audited by the audit log, and integrated with the Kubernetes certificate lifecycle via cert-manager or the Vault Agent Injector.

The two-tier hierarchy (offline root → Vault intermediate) keeps your most sensitive key material air-gapped while enabling the automated, high-frequency issuance that service mesh and zero-trust networking require. The multi-issuer capability (Vault 1.11+) makes CA rotation a non-event. The vault pki health-check command makes operational hygiene scriptable.

Build the hierarchy once. Automate issuance from day one. Set generate_lease=false. Monitor CRL expiry and CA expiry. Let certificates expire rather than revoking them. That is the production PKI posture.

Sources

• PKI secrets engine — HashiCorp Developer
• PKI secrets engine considerations — HashiCorp Developer
• Quick start: intermediate CA setup — HashiCorp Developer
• Build a CA in Vault with an offline root — HashiCorp Developer
• Build your own CA — HashiCorp Developer
• PKI rotation primitives — HashiCorp Developer
• PKI Unified CRL and OCSP with cross-cluster revocation — HashiCorp Developer
• PKI secrets engine HTTP API — HashiCorp Developer
• vault pki health-check command — HashiCorp Developer
• Vault Agent template stanza — HashiCorp Developer
• Vault Agent Injector examples — HashiCorp Developer
• Configure Vault as cert manager in Kubernetes — HashiCorp Developer
• Vault - cert-manager Documentation
• cert-manager 1.19 release notes
• HashiCorp Vault 2.0 — IBM Identity Federation (InfoQ)
• Certificate Revocation Deep Dive — Axelspire
• Better together: SPIFFE and HashiCorp Vault (Medium)
• vault-pki-exporter — GitHub
• Vault telemetry reference — HashiCorp Developer
• Vault releases — GitHub