Building a Secure, Intent-Driven Kubernetes Platform

🚀 Building a Secure, Intent-Driven Kubernetes Platform

By Mark Kendall | January 2026

In modern cloud-native engineering, the "how" of deployment is just as important as the "what." Today, we are setting a new architectural standard for our platform—moving away from theory and hype toward a robust, scalable reality.

This document outlines our new standard for Secrets & Environment Management, aligning Dev, DevOps, Security, and Platform teams under one clean, auditable model.

1. Our Vision: The End of "Secret Sprawl"

The goals of this new architecture are simple but absolute:

* ❌ No secrets in Git.

* ❌ No secrets in Helm or Kustomize overlays.

* ❌ No manual handling of production credentials.

Instead, we are moving toward:

* ✅ Centralized secret storage.

* ✅ Environment-specific injection.

* ✅ Clean governance and ownership.

* ✅ Auditable, reproducible deployments.

2. Core Principles (Non-Negotiable)

To ensure the integrity of our platform, we are adopting a strict "Never/Always" policy:

| 🚫 What We NEVER Do | ✅ What We ALWAYS Do |

|---|---|

| Store secrets in Git repositories | Store secrets in a secure external vault |

| Commit secrets in YAML or Helm values | Reference secret names and keys only |

| Bake secrets into Docker images | Inject secrets at deploy-time or runtime |

| Place sensitive data in ConfigMaps | Isolate secrets by environment (Dev/UAT/Prod) |

3. The Technical Foundation

Environment Overlay Model

We use Kustomize to manage environment-specific configurations. Each microservice follows a standardized folder structure:

packages/<service>/k8s/

overlays/

dev/ (kustomization.yaml, deployment-patch.yaml)

uat/ (kustomization.yaml, deployment-patch.yaml)

prod/ (kustomization.yaml, deployment-patch.yaml)

Decoupled Secrets

Applications never "see" secret values in the codebase. They reference a Kubernetes Secret name and key. This allows the same YAML patch to function across all environments without modification.

4. Roadmap to Maturity

We are implementing this architecture in three distinct phases:

Phase 1: Immediate Efficiency

GitLab CI Variables → Kubernetes Secrets

Currently, secrets are stored as scoped, masked variables in GitLab. Our pipelines inject these into the cluster during the deployment step, creating a "generic secret" on the fly before applying Kustomize overlays.

Phase 2: The Target Standard

External Secrets Operator (ESO)

We will move secrets into AWS Secrets Manager. The External Secrets Operator within our cluster will automatically synchronize these values into Kubernetes Secrets, removing the need for sensitive data to ever pass through the CI/CD pipeline.

Phase 3: Future State

OIDC & IRSA (Zero-Trust)

Our long-term goal is the elimination of static secrets. By using IAM Roles for Service Accounts (IRSA), pods will assume identities and authenticate directly with AWS services without needing stored credentials.

5. Ownership & Governance

This model enforces a healthy separation of concerns:

* Shared Services: Owns the secret store (AWS SSM/Secrets Manager), IAM, and encryption.

* Application Teams: Own the overlay definitions and define which secret keys their service requires.

* CI/CD: Owns the routing (mapping branches to environments) without ever needing to "know" the secret values.

Summary: Why This Matters

This architecture makes our platform secure by default. It provides a clear path for promotion from Dev to Prod, ensures full auditability for our security partners, and—most importantly—gives our developers the autonomy to move fast without risking a credential leak.

> "This foundation makes audits boring—which is the highest compliment you can pay to enterprise architecture."

>