Secret Rotation Automation

Advanced15 min

Manage and rotate Kubernetes secrets safely, including creating, updating, and verifying secret propagation to pods.

Prerequisites

-kubectl configured
-Understanding of Kubernetes secrets

Steps

List all secrets in a namespace

View existing secrets and their types.

$ kubectl get secrets -n <namespace> -o custom-columns='NAME:.metadata.name,TYPE:.type,AGE:.metadata.creationTimestamp'

Create a new secret from literal values

Create a secret with key-value pairs from the command line.

$ kubectl create secret generic db-credentials --from-literal=username=admin --from-literal=password=$(openssl rand -base64 24) --dry-run=client -o yaml | kubectl apply -f -

Using --dry-run=client -o yaml | kubectl apply allows you to update existing secrets without deleting them first.

View a secret's contents (decoded)

Decode and display the values stored in a secret.

$ kubectl get secret db-credentials -o jsonpath='{.data}' | jq 'to_entries[] | {key: .key, value: (.value | @base64d)}'

Be careful displaying decoded secrets in shared terminals or CI logs.

Restart pods to pick up rotated secrets

After updating a secret, restart the deployment so pods get the new values.

$ kubectl rollout restart deployment/<deployment-name>

Alternatively, mount secrets as volumes instead of environment variables. Volume-mounted secrets are updated automatically without pod restarts.

Verify pods have the updated secret

Check that running pods see the new secret values.

$ kubectl exec <pod-name> -- env | grep -i 'DB_\|SECRET_\|PASSWORD'

Full Script

full-script.sh

#!/bin/bash
# Secret Rotation Workflow
set -euo pipefail

SECRET_NAME=${1:?"Usage: $0 <secret-name> [namespace]"}
NS=${2:-"default"}

echo "=== Current Secret ==="
kubectl get secret "$SECRET_NAME" -n "$NS" -o jsonpath='{.metadata.creationTimestamp}' 2>/dev/null && echo "" || echo "Secret not found"

echo ""
echo "=== Secret Keys ==="
kubectl get secret "$SECRET_NAME" -n "$NS" -o jsonpath='{.data}' 2>/dev/null | jq -r 'keys[]' || echo "No keys found"

echo ""
echo "=== Pods Using This Secret ==="
kubectl get pods -n "$NS" -o json | jq -r --arg secret "$SECRET_NAME" '.items[] | select(.spec.containers[].env[]?.valueFrom.secretKeyRef.name == $secret or .spec.volumes[]?.secret.secretName == $secret) | .metadata.name' 2>/dev/null || echo "None found"

echo ""
echo "=== Rotation Steps ==="
echo "1. Update:  kubectl create secret generic $SECRET_NAME --from-literal=key=newvalue --dry-run=client -o yaml | kubectl apply -f -"
echo "2. Restart: kubectl rollout restart deployment/<name> -n $NS"
echo "3. Verify:  kubectl exec <pod> -- env | grep KEY"

FAQ

Discussion

Loading comments...

#!/bin/bash # Secret Rotation Workflow set -euo pipefail SECRET_NAME=${1:?"Usage: $0 <secret-name> [namespace]"} NS=${2:-"default"} echo "=== Current Secret ===" kubectl get secret "$SECRET_NAME" -n "$NS" -o jsonpath='{.metadata.creationTimestamp}' 2>/dev/null && echo "" || echo "Secret not found" echo "" echo "=== Secret Keys ===" kubectl get secret "$SECRET_NAME" -n "$NS" -o jsonpath='{.data}' 2>/dev/null | jq -r 'keys[]' || echo "No keys found" echo "" echo "=== Pods Using This Secret ===" kubectl get pods -n "$NS" -o json | jq -r --arg secret "$SECRET_NAME" '.items[] | select(.spec.containers[].env[]?.valueFrom.secretKeyRef.name == $secret or .spec.volumes[]?.secret.secretName == $secret) | .metadata.name' 2>/dev/null || echo "None found" echo "" echo "=== Rotation Steps ===" echo "1. Update: kubectl create secret generic $SECRET_NAME --from-literal=key=newvalue --dry-run=client -o yaml | kubectl apply -f -" echo "2. Restart: kubectl rollout restart deployment/<name> -n $NS" echo "3. Verify: kubectl exec <pod> -- env | grep KEY"