Here are three example questions every Kubernetes platform or security administrator might like to be able to answer on demand:

  1. Which of my namespaces — across every cluster — are missing a default-deny NetworkPolicy?

  2. Is a given app’s namespace and pod-security posture actually consistent across dev, staging, and prod?

  3. Who can read Secrets, fleet-wide?

If your configuration lives in Helm charts, Kustomize overlays, cdk8s TypeScript, or Jsonnet, none of these is a query you can run against your authoritative configuration files — those are configuration as code: programs, likely spread across many git repositories, that render manifests for one workload, for one cluster, on the fly during deployment or GitOps synchronization. There’s no store of “all my namespaces, across all my clusters” to query.

The obvious workaround is to skip the configuration source entirely and read the live clusters — kubectl every namespace’s NetworkPolicies, Roles, and Namespace objects and analyze the fetched resources. That’s roughly what a typical policy tool or a homegrown script do, and for the read half it works.

But once you’ve found a problem, fixing it is where configuration as code lands you between a rock and a hard place.

  1. Patch the cluster directly and you’ve drifted from the source. Your next ArgoCD/Flux GitOps reconcile may revert it.

  2. Go back to the template instead. You first have to map a rendered resource in some cluster back to the chart, values file, TypeScript, or Jsonnet repo, directory, and files that produced it, translate the change to the as-code format in an ad hoc, workload-specific, format-specific way, and perform the git commit-and-push ceremony. If you don’t own the helm chart, you may need to wrap it with an umbrella chart or add a kustomize overlay to inject the additional policy resources.

The place you find the problem (a live cluster) and the place you must fix it (a program) are different systems, and the translation between them is complicated enough to require a human or AI agent. AI agents work well enough now to fix a single configuration, but the risk of hallucination, inconsistencies, misinterpretations, invalid assumptions, out-of-date information, and other mistakes increases with the number of occurrences of the issue.

ConfigHub takes another path: configuration as data. Every Kubernetes resource is stored as fully materialized YAML — literal values, no templates. Because it’s data, not code, every field of every resource in every namespace across every cluster is queryable and mutable — through server-side functions that produce clean, audited revisions.

The key isn’t just that you can query it — a cluster scraper can query too. It’s that the data you query is the same data you fix, and it’s the system of record for the configuration data applied to your clusters. Read and write, on the same records, for the whole fleet.

That difference means you can build tools that read and modify policy across the entire fleet. To make the point, we built several policy tools. Here I’m going to discuss 3 CLI-based agent-friendly tools (open-source repo), each about a half a day of work (which I’m sure we can reduce): cub-namespace, cub-netpol, and cub-rbac. I previously wrote about a web app form factor of the RBAC manager.

1. Diagnose the whole fleet in one command

Start with inventory. Not “what’s in this chart” — what’s actually deployed, everywhere:

$ cub-namespace snapshot
CLUSTER NS NETPOL RBAC WORKLOADS UNITS GATED UNAPPLIED
cluster-worker-kubernetes-yaml-cluster 1 1 1 1 6 0 1
dev-cluster 3 12 10 19 33 0 20
prod-cluster 3 17 10 19 38 0 24
...
10 clusters, 11 namespaces, 36 network policies, 40 rbac, 41 workloads, 114 units

Now ask the governance question — which namespaces are missing their policy envelope? This is a property of the whole set of resources in each namespace, joined across types. No per-resource admission controller and no per-cluster tenancy controller can see it, because each sees one object, or one cluster, at a time.

$ cub-namespace findings
SEVERITY ANALYZER CLUSTER NAMESPACE MESSAGE
medium missing-pod-security dev-cluster apptique namespace "apptique" has no pod-security.kubernetes.io/enforce label
medium missing-pod-security prod-cluster apptique namespace "apptique" has no pod-security.kubernetes.io/enforce label
medium missing-pod-security dev-cluster appvote namespace "appvote" has no pod-security.kubernetes.io/enforce label
...
low missing-baseline-rbac prod-cluster appchat namespace "appchat" has no baseline RBAC (RoleBinding)
...
14 findings (0 high, 7 medium, 7 low)

Then the consistency question — is each app’s namespace identical across all its environment/region variants? This is the invariant that quietly drifts in a configuration-as-code world, because each environment is a different values file:

$ cub-namespace consistency
COMPONENT VARIANTS NAMESPACES POD-SECURITY CONSISTENT ISSUES
appchat 2 appchat - yes -
apptique 3 apptique baseline yes -
appvote 2 appvote - yes -
...
7 components (7 consistent, 0 inconsistent)

Seven components, checked across all their variants, in one query. If apptique’s namespace had drifted to apptique-prod in one environment, this would say so — and point at the exact configuration.

Now the basic security question — who can read Secrets, everywhere? Try answering this from a directory of Helm charts.

$ cub-rbac who-can get secrets
CLUSTER SUBJECT SCOPE ROLE BINDING
cluster-worker-kubernetes-yaml-cluster ServiceAccount:confighub/confighub-worker cluster-wide cluster-admin (builtin) confighub-worker-admin
rbac-demo-dev Group:oidc:oncall cluster-wide cluster-admin (builtin) oncall-breakglass
rbac-demo-dev Group:oidc:platform-operators cluster-wide rbac-manager-operator operator
rbac-demo-prod Group:oidc:platform-operators cluster-wide rbac-manager-operator operator
rbac-demo-staging Group:oidc:platform-operators cluster-wide rbac-manager-operator operator

5 grants

Five grants across the fleet — including an on-call break-glass group with cluster-admin.

The NetworkPolicy tool answers the network twin of that question — who can reach a given workload? This isn’t reading one policy; it’s the effective reachability computed from the additive-OR combination of every NetworkPolicy in the namespace (isolation, pod-selectors, ingress rules together). Answering it from templates would mean simulating every selector across every policy.

$ cub-netpol who-can-reach cartservice --cluster prod-cluster
Deployment prod-cluster/cartservice (apptique) can be reached by 2 workload(s):
apptique Deployment checkoutservice
apptique Deployment frontend

Exactly two workloads — checkoutservice and frontend — and nothing else. That’s the answer a security reviewer actually wants for a sensitive service, and it falls out of the same fleet snapshot. (Its inverse, reachable-from <workload>, answers “if this pod is compromised, what can it talk to?”)

Back on the RBAC side, the hygiene scan finds the usual over-privilege, ranked:

$ cub-rbac findings
SEVERITY ANALYZER CLUSTER KIND RESOURCE MESSAGE
HIGH cluster-admin-bindings rbac-demo-dev ClusterRoleBinding oncall-breakglass Grants superuser (cluster-admin) to: Group:oidc:oncall.
HIGH wildcard-rules rbac-demo-dev ClusterRole legacy-admin Wildcard permissions (rule 0: wildcard resources; wildcard apiGroups)...
MEDIUM risky-grants rbac-demo-dev ClusterRole rbac-manager-operator Sensitive access: pod exec/attach, secrets read.
MEDIUM orphaned-bindings rbac-demo-dev RoleBinding monitoring/grafana-viewers References Role "grafana-viewer", which does not exist...
LOW unbound-service-accounts dev-cluster ServiceAccount apptique/adservice ServiceAccount has no role bindings...

These commands read the ConfigHub system of record — the same data that gets applied — so the answers are about your authoritative configuration.

2. Find a gap → fix it as data → across the fleet, one command

Diagnosis is half the value. The other half is that, because config is data, the fix is a write to that data — not a manual cluster edit that immediately drifts. (Though that’s not a problem in ConfigHub, either, but it does bypass validation, review, etc.) Not only that, but it’s easy enough to make the change in ConfigHub (and with the right tools it can even be easier) that the motivation to make changes directly in clusters is greatly reduced.

Let’s manufacture a gap. Someone deletes a default-deny NetworkPolicy — say for the appvote app, in both clusters it runs in, exactly the kind of omission that hides in a values file with a conditional change. (This could be easily prevented in ConfigHub; it’s just for illustration purposes.)

$ cub unit delete default-deny-appvote --space appvote-dev
Successfully deleted unit default-deny-appvote
$ cub unit delete default-deny-appvote --space appvote-prod
Successfully deleted unit default-deny-appvote

The NetworkPolicy tool catches it immediately — in both clusters, with the exposed workloads named:

$ cub-netpol coverage
CLUSTER NAMESPACE POLICY DD-INGRESS WORKLOADS UNCOVERED-IN
dev-cluster appvote no no 5 5
prod-cluster appvote no no 5 5

$ cub-netpol findings --severity high
SEVERITY ANALYZER CLUSTER NAMESPACE RESOURCE MESSAGE
high missing-default-deny-ingress dev-cluster appvote - namespace "appvote" has 5 workload(s) but no default-deny ingress NetworkPolicy
high uncovered-ingress dev-cluster appvote vote Deployment "vote" ... is not selected by any ingress NetworkPolicy
high uncovered-ingress prod-cluster appvote db Deployment "db" ... is not selected by any ingress NetworkPolicy
...

Now fix it. Not “edit the chart, render to check the output, and perform the git ceremony.” One command generates the missing policy as data for every uncovered namespace in the fleet — dry-run first:

$ cub-netpol fleet default-deny # dry-run by default
CLUSTER NAMESPACE SPACE UNIT ACTION
dev-cluster appvote appvote-dev default-deny-appvote create
prod-cluster appvote appvote-prod default-deny-appvote create

2 namespace(s) would get a default-deny (0 skipped). Re-run with --commit --change-desc "…".

$ cub-netpol fleet default-deny --commit --change-desc "Restore missing default-deny for appvote across the fleet"
CLUSTER NAMESPACE SPACE UNIT ACTION
dev-cluster appvote appvote-dev default-deny-appvote created
prod-cluster appvote appvote-prod default-deny-appvote created

Created 2 default-deny Unit(s) (0 skipped; not applied — apply when ready).

Confirm:

$ cub-netpol coverage
CLUSTER NAMESPACE POLICY DD-INGRESS WORKLOADS UNCOVERED-IN
dev-cluster appvote yes yes 5 0
prod-cluster appvote yes yes 5 0

$ cub-netpol findings --severity high
0 findings (0 high, 0 medium, 0 low)

Two things to notice. First, ”created, not applied.” The tool wrote new versioned configuration, but nothing will hit a cluster until someone deliberately releases it — you can think of it like the equivalent of merging a PR in the case of GitOps, but without the ceremony. The edit is auditable (every mutation carries a change description and produces a revision) and reversible — the opposite of a kubectl edit that drifts the moment it lands. Second, that was one command for the whole fleet. The tool found the uncovered namespaces itself and generated a correct policy for each, in the right place.

The same shape applies to fixing namespace envelopes (cub-namespace apply-envelope, backfill) and RBAC (cub-rbac fleet-edit). And you can make any of these findings enforced rather than advisory: each tool installs a guardrail pack of Warn=true validation triggers, so a regression shows up as a warning on the offending configuration before it ever applies.

Why configuration as code can’t do this

The problem with Helm, cdk8s, and other configuration-as-code tools is that they create an ongoing obstacle to essential security administration tasks.

Configuration as code (Helm, Kustomize, cdk8s):

  • Templates / programs that render manifests, sprawled across git repos

  • To assess potential security problems, scrape live clusters and analyze

  • To fix, patch the cluster (drifts) or map the finding back to the template

  • There’s no uniform data model to build tools on

Configuration as data (ConfigHub):

  • Fully-materialized YAML, versioned, in a database

  • To assess potential security problems, query over the system of record

  • To fix, run functions over the same configuration data

  • Standard KRM + an SDK → fleet tools in an afternoon

The last bullet is the real win. One cannot build “modify RBAC across my fleet” on top of Helm charts, because there is no fleet — only a sprawl of templates and values and repos. On a uniform data model, a platform team can build exactly the tool their org needs, such as to query and fix security policy across a whole fleet of clusters. We built some tools to demonstrate it.

Your policy already wants to be data — a set of facts about what should be true across your clusters. Store it that way, and querying and fixing it fleet-wide stops being a research project and becomes a command line.

Have you been frustrated by security tools that either cause drift or just provide recommendations, which you then need to implement? How do you patch Helm charts? Using Kustomize? Using an umbrella chart? By forking it? Have you wanted to build Kubernetes tools that could make changes, but abandoned the idea because you expected everyone to write configuration generators using Helm, Kustomize, cdk8s, or another configuration-as-code tool? Do you use cluster-scraping tools, such as Cloud Query, in order to assess the state of your fleet? How do you keep track of all of the git repositories where the configuration generators live?

Feel free to email us at hello@confighub.com, or send me a message on LinkedIn, X/Twitter, or Bluesky.

You could also try out ConfigHub, which is now in preview.

If you found this interesting, you may be interested in other posts in my Kubernetes series.