If you're attending KubeCon Chicago from 6 - 9 November, don’t forget to visit us at Booth A14. We're excited to offer you a free DevEx Assessment to elevate your DevOps and Cloud-Native game. This comprehensive review is designed to pinpoint areas for growth and improvement within your existing practices and tools infrastructure. We look forward to meeting you!


Introduction

Secrets management is an important component of any application’s security posture, particularly when deploying applications in a Kubernetes environment. Kubernetes provides a native Secrets object, but ensuring the effective management and security of these secrets requires careful consideration and planning. This blog post will guide you through the best practices and tools you can use to enhance secrets management in Kubernetes.

Creating a Kubernetes Secrets

Kubernetes Secrets allow you to store and manage sensitive information such as passwords, OAuth tokens, and SSH keys. By default, Secrets are stored as Base64-encoded strings, making them not immediately human-readable. However, it’s crucial to note that this is not a form of encryption and does not provide robust security on its own.

Creating a Secret

You can create a Kubernetes Secret using kubectl. Here’s an example:

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
type: Opaque
data:
  username: dXNlcm5hbWU=
  password: cGFzc3dvcmQ=

Save the yaml file and apply it using kubectl apply -f secret.yaml

Best Practices for Secrets Management

  1. Encryption at Rest

Ensure that Secrets are encrypted at rest. Kubernetes supports encryption at rest for Secrets, but it’s not enabled by default. You can enable it by configuring the API server with the --encryption-provider-config flag, pointing to a configuration file that specifies an encryption provider.

2. Use Secrets Management Tools

There are several external tools and services designed to enhance secrets management in Kubernetes:

  • HashiCorp Vault: Integrates with Kubernetes and provides a robust mechanism for secrets management.
  • AWS Secrets Manager or Azure Key Vault: If you’re running on AWS or Azure, using their respective secrets management services can provide seamless integration and robust security.

3. Rotate Secrets Regularly

Implement a routine for rotating secrets regularly and ensure that applications can handle secret rotation without downtime.

4. Least Privilege Access

Assign the least privilege necessary for your applications to access Secrets. You can also implement Kubernetes RBAC to control access to Secrets.

Integrating HashiCorp Vault with Kubernetes

HashiCorp Vault stands out for its ability to handle secrets in a dynamic and secure manner. Here’s how you can get started with HashiCorp Vault with Kubernetes.

Install and Configure Vault

First, install Vault in your Kubernetes cluster. You can use the Helm chart for this:

helm repo add hashicorp https://helm.releases.hashicorp.com
helm install vault hashicorp/vault

Initialize and unseal Vault

After installing, you need to initialize and unseal Vault. This process generates encryption keys and sets up the initial access credentials.

kubectl exec vault-0 -- vault operator init
kubectl exec vault-0 -- vault operator unseal

Configure Kubernetes Authentication

Enable and configure Kubernetes authentication in Vault:

kubectl exec vault-0 -- vault auth enable kubernetes
kubectl exec vault-0 -- vault write auth/kubernetes/config \
  token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
  kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443" \
  kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt

Create Policies and Roles

Define policies in Vault that specify what secrets can be accessed:

path "secret/data/my-app/*" {
  capabilities = ["read"]
}

Create a role in Vault that connects the Kubernetes service account to this policy:

kubectl exec vault-0 -- vault write auth/kubernetes/role/my-app \
  bound_service_account_names=my-app \
  bound_service_account_namespaces=default \
  policies=my-app-policy \
  ttl=1h

Accessing Secrets from Your Application

In your application, use the Vault API to access secrets. Here’s an example using curl:

VAULT_TOKEN=$(kubectl exec my-app-pod -- curl --request POST --data '{"jwt": "'$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)'", "role": "my-app"}' http://vault:8200/v1/auth/kubernetes/login | jq -r .auth.client_token)

SECRET=$(kubectl exec my-app-pod -- curl --header "X-Vault-Token: $VAULT_TOKEN" http://vault:8200/v1/secret/data/my-app/my-secret | jq -r .data.data)


Conclusion

Implementing robust secrets management in Kubernetes is important for maintaining the security and integrity of your applications. By following best practices, using tools like HashiCorp Vault, and regularly auditing and rotating secrets, you can create a secure and resilient environment for handling sensitive information.