Certificate Issues

Certificate Issues

This guide covers common TLS certificate problems in Mantis deployments. Mantis uses mTLS (mutual TLS) for internal communication between Mandible, Thorax, and Tarsus, so certificate misconfiguration is one of the most common sources of connectivity failures.

Symptoms

Symptom	Likely Cause
Tarsus fails to register	CA certificate mismatch or expired cert
gRPC health shows “unhealthy”	Thorax mTLS handshake failing
certificate verify failed in logs	CA not trusted or cert chain incomplete
certificate has expired	Certificate past its validity period
bad certificate	Client cert not signed by expected CA

Certificate Expiry

Check Certificate Expiry

# Check a PEM certificate file
openssl x509 -in /etc/mantis/certs/server.crt -noout -enddate

# Check a remote server's certificate
openssl s_client -connect mantis.example.com:3000 < /dev/null 2>/dev/null \
  | openssl x509 -noout -enddate

# Check all certificates in a directory
for cert in /etc/mantis/certs/*.crt; do
  echo "$cert: $(openssl x509 -in "$cert" -noout -enddate)"
done

Renew Certificates

If certificates have expired or are approaching expiry:

1. Generate new certificates using your CA or mantisctl
2. Replace certificate files on all affected components
3. Restart the affected services

# Restart after certificate replacement
systemctl restart mantis-mandible mantis-thorax mantis-tarsus

CA Trust Issues

Verify CA Chain

Ensure the certificate was signed by the expected CA:

# Verify certificate against CA
openssl verify -CAfile /etc/mantis/certs/ca.crt /etc/mantis/certs/server.crt

# Check the full chain
openssl verify -CAfile /etc/mantis/certs/ca.crt \
  -untrusted /etc/mantis/certs/intermediate.crt \
  /etc/mantis/certs/server.crt

Common CA Problems

Problem: unable to get local issuer certificate

The server certificate was signed by a CA that the client does not trust. Ensure all components reference the same CA certificate file.

# Each component's TLS config must reference the CA
[tls]
ca_cert_path = "/etc/mantis/certs/ca.crt"
cert_path = "/etc/mantis/certs/component.crt"
key_path = "/etc/mantis/certs/component.key"

Problem: self-signed certificate in certificate chain

If using self-signed certificates, the CA certificate must be explicitly trusted by every component. The system trust store is not used for mTLS verification.

mTLS Handshake Failures

Diagnose Handshake Issues

# Test mTLS connection to Mandible's internal gRPC port
openssl s_client \
  -connect localhost:50052 \
  -cert /etc/mantis/certs/thorax.crt \
  -key /etc/mantis/certs/thorax.key \
  -CAfile /etc/mantis/certs/ca.crt

# Check for TLS version compatibility
openssl s_client -connect localhost:3000 -tls1_3

Common mTLS Problems

Problem: no suitable certificate found

The client is not presenting a certificate. Verify the component’s TLS configuration includes both cert_path and key_path.

Problem: certificate required

The server requires a client certificate but none was provided. This is expected behavior — all internal Mantis communication requires mTLS.

Problem: wrong CN (Common Name)

Mantis extracts the service identity from the certificate’s Common Name. For Tarsus agent certificates, the CN is a free-form name matching the client_name used during registration — there is no fixed value required. The only reserved system CN is mandible, which is used exclusively by Mandible’s internal dispatch certificate. When Thorax connects to Mandible it presents the certificate configured at [mandible].tls_cert_path as its client identity — provisioned separately from the Thorax server certificate.

Certificate File Permissions

Ensure certificate files have correct permissions:

# Private keys should only be readable by the service user
chmod 600 /etc/mantis/certs/*.key
chown mantis:mantis /etc/mantis/certs/*.key

# Certificates and CA can be world-readable
chmod 644 /etc/mantis/certs/*.crt

# Verify permissions
ls -la /etc/mantis/certs/

Certificate Revocation

If a certificate has been compromised, revoke it through the Lens UI or API:

# Check if a registration (and its certificate) is revoked
curl -s -H "Authorization: Bearer $TOKEN" \
  https://mantis.example.com/api/v1/registrations/{id} | jq '.status'
# A revoked registration will show "revoked" as the status value

# Revoke a registration via the API
curl -s -X POST -H "Authorization: Bearer $TOKEN" \
  https://mantis.example.com/api/v1/registrations/{id}/revoke

Revoked certificates are rejected at the application layer: Thorax checks with Mandible on each gRPC request whether the client certificate has been revoked (results are cached for up to 60 seconds). If the certificate is revoked, Thorax returns a gRPC UNAUTHENTICATED error on that request — the TLS connection itself is not closed, but every subsequent call is rejected too once the cache reflects the revocation. After revocation, the affected component must be re-enrolled with a new certificate.

Debugging Checklist

1. Check certificate expiry dates
2. Verify the CA chain (openssl verify)
3. Confirm file permissions on private keys
4. Ensure all components reference the same CA certificate
5. Check the certificate CN matches the expected identity
6. Review component logs for specific TLS error messages
7. Test connectivity with openssl s_client

Next Steps

• Connection Problems — Broader connectivity troubleshooting
• Certificate Backup — Protect your certificates
• Key Rotation Runbook — Scheduled certificate rotation