There are a lot of helm charts out there for third-party components — software someone else wrote. That’s essentially what helm was originally created for — a software installer for Kubernetes (e.g., see helm install). Helm calls itself a package manager, but it has a much heavier focus on an area that most package managers don’t: software configuration.
Very large amounts of configuration. To the point where helm is mainly thought of as a configuration tool. Many helm charts for off-the-shelf applications have hundreds of parameters. They are pushed in the direction of extreme flexibility by requests from their many users — the more users and use cases, the more parameters.
Take the Traefik chart as an example. To be clear, I don’t intend to pick on Traefik or its chart specifically. We had a need to run Traefik, so we started with the chart. I’m just using it here to illustrate the challenges with the installer-as-universal-template paradigm.
This chart has more than 2700 lines of complex templates.
% wc -l *.yaml *.tpl rbac/*.yaml
58 daemonset.yaml
58 deployment.yaml
4 extra-objects.yaml
62 gateway.yaml
14 gatewayclass.yaml
35 hpa.yaml
112 hub-admission-controller.yaml
19 hub-apiportal.yaml
12 ingressclass.yaml
43 ingressroute.yaml
23 poddisruptionbudget.yaml
28 prometheusrules.yaml
12 provider-file-cm.yaml
26 pvc.yaml
78 requirements.yaml
33 service-metrics.yaml
86 service.yaml
69 servicemonitor.yaml
39 tlsoption.yaml
12 tlsstore.yaml
261 _helpers.tpl
935 _podtemplate.tpl
25 _service-metrics.tpl
85 _service.tpl
280 rbac/clusterrole.yaml
17 rbac/clusterrolebinding.yaml
68 rbac/podsecuritypolicy.yaml
207 rbac/role.yaml
26 rbac/rolebinding.yaml
14 rbac/serviceaccount.yaml
2741 total
Here’s a small taste:
{{- $services := .Values.service.additionalServices -}}
{{- $services = set $services "default" (omit .Values.service "additionalServices") }}
{{- range $name, $service := $services -}}
{{- if ne $service.enabled false -}}
{{- $fullname := include "traefik.service-name" (dict "root" $ "name" $name) }}
{{- $tcpPorts := dict -}}
{{- $udpPorts := dict -}}
{{- $exposedPorts := false -}}
{{- range $portName, $config := $.Values.ports -}}
{{- if $config -}}
{{- if ($config.http3).enabled -}}
{{- if (not $config.tls.enabled) -}}
{{- fail "ERROR: You cannot enable http3 without enabling tls" -}}
{{- end -}}
{{ $udpConfig := deepCopy $config -}}
{{ $_ := set $udpConfig "protocol" "UDP" -}}
{{ $_ := set $udpConfig "exposedPort" (default $config.exposedPort $config.http3.advertisedPort) -}}
{{- if (not $service.single) }}
{{ $_ := set $udpPorts (printf "%s-http3" $portName) $udpConfig -}}
{{- else }}
{{ $_ := set $tcpPorts (printf "%s-http3" $portName) $udpConfig -}}
{{- end }}
{{- end -}}
{{- if eq (toString $config.protocol) "UDP" -}}
{{ $_ := set $udpPorts $portName $config -}}
{{- end -}}
{{- if eq (toString (default "TCP" $config.protocol)) "TCP" -}}
{{ $_ := set $tcpPorts $portName $config -}}
{{- end -}}
{{- if (index (default dict $config.expose) $name) -}}
{{- $exposedPorts = true -}}
{{- end -}}
{{- end -}}
{{- end -}}
Clear enough?
It also more than 3000 lines across values.yaml, values.schema.json, and VALUES.md.
% wc -l values.yaml
1105 values.yaml
% wc -l values.schema.json
1982 values.schema.json
% wc -l VALUES.md
383 VALUES.md
That’s quite a bit to read and understand. More than 350 input parameters.
Traefik also has >15k lines of CRDs, though it’s unlikely anyone would read them directly.
% wc -l crds/*.yaml
10345 crds/gateway-standard-install.yaml
368 crds/hub.traefik.io_accesscontrolpolicies.yaml
332 crds/hub.traefik.io_aiservices.yaml
132 crds/hub.traefik.io_apibundles.yaml
186 crds/hub.traefik.io_apicatalogitems.yaml
103 crds/hub.traefik.io_apiplans.yaml
139 crds/hub.traefik.io_apiportals.yaml
168 crds/hub.traefik.io_apiratelimits.yaml
250 crds/hub.traefik.io_apis.yaml
249 crds/hub.traefik.io_apiversions.yaml
207 crds/hub.traefik.io_managedsubscriptions.yaml
26 crds/kustomization.yaml
384 crds/traefik.io_ingressroutes.yaml
247 crds/traefik.io_ingressroutetcps.yaml
111 crds/traefik.io_ingressrouteudps.yaml
1151 crds/traefik.io_middlewares.yaml
87 crds/traefik.io_middlewaretcps.yaml
139 crds/traefik.io_serverstransports.yaml
120 crds/traefik.io_serverstransporttcps.yaml
114 crds/traefik.io_tlsoptions.yaml
97 crds/traefik.io_tlsstores.yaml
668 crds/traefik.io_traefikservices.yaml
15623 total
When we were first trying to use this chart, we were trying to understand how to set the values to what we needed. In particular, we wanted to expose a non-standard port. So we added something like this to the input values:
ports:
auxsecure:
port: 9443
Did that work? Well, no it didn’t. We did helm install, and tried to send traffic to port 9443, and it didn’t work.
So, we rendered the chart using helm template. Only the standard ports, 80 and 443, appeared in the Kubernetes Service in the rendered output, shown below. Speaking of the rendered output, it resulted in only about 300 lines of standard Kubernetes YAML. Compared with the unique-to-this-chart 1100-line values.yaml file, of which about 12 non-contiguous lines were relevant to this issue.
# Source: traefik/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: all-traefik
namespace: default
labels:
app.kubernetes.io/name: traefik
app.kubernetes.io/instance: all-default
helm.sh/chart: traefik-35.1.0
app.kubernetes.io/managed-by: Helm
spec:
type: LoadBalancer
selector:
app.kubernetes.io/name: traefik
app.kubernetes.io/instance: all-default
ports:
- port: 80
name: web
targetPort: web
protocol: TCP
- port: 443
name: websecure
targetPort: websecure
protocol: TCP
We concluded that we were missing a couple lines needed to expose the port. We also found that we had specified some other values incorrectly. There were no warnings or errors because it turns out that additionalProperties was true by default in the values.schema.json file. We set that to false, corrected the misspecified values, ranhelm template again, verified that the output was what we intended, ran helm upgrade, and confirmed that it worked. Yeah!
This is why the rendered manifest pattern, where the configuration is rendered (generated) prior to deployment, is attractive to some Kubernetes users. The output is so much more understandable than the input. The output is specific to my use case. The input needs to handle a wide variety of use cases, such as HTTP3 and UDP.
The rendered output can also be run through Kubernetes linting tools like kubeconform or checked by your IDE, and run through policy tools, such as CEL or Open Policy Agent. It also can be searched for specific values using tools like yq, and edited with simple tools like sed.
Now how could we make such a simple mistake? Well, the standard ports were exposed by default already, and it wasn’t immediately obvious which parameters caused that. The example in the documentation didn’t explicitly expose the ports it specified, nor did the reference guide, which also confusingly displayed a YAML configuration file format that was similar to but slightly different from the helm values. There were 12 parameters in VALUES.md that contain the string expose, but the relevant parameters included the port name in their paths, such as ports.websecure.expose.default.
Unlike application installers for desktop software, the helm tool itself doesn’t provide help with setting the input parameter values. There are some GUIs that provide forms for entering helm values, but with hundreds of parameters, that’s not much assistance.
I analyzed the chart to identify sources of complexity. Only 6 values were used in more than 2 template files:
.Values.additionalArguments 3
.Values.hub.apimanagement.enabled 4
.Values.hub.token 4
.Values.metrics.prometheus 4
.Values.ports 3
.Values.providers.file.enabled 4
A number of values were used in exactly two files due to being used in multiple alternatives. Some chart options allowed the user installing the chart to choose between different Kubernetes resource types, and others whether to include some optional resources, including:
-
Deployment vs DaemonSet
-
Ingress vs Gateway
-
ClusterRole[Binding] vs Role[Binding]
-
HorizontalPodAutoscaler
-
PodDisruptionBudget
-
Various Prometheus resources
We had to make a few such high-level choices. Mostly default choices were acceptable. Many options were irrelevant to our current needs. This chart is pretty thorough. Others may require more customization, even post-rendering, to use alternatives for namespace provisioning, RBAC configuration, secrets, container registry, autoscaling, monitoring, logging, ingress, certificate management, and/or storage.
The vast majority of values were each just used in a single template file. Many of the parameters just set standard Kubernetes resource attributes.
There were also about 170 command-line flags (if my grep incantation was correct), which Traefik apparently uses for application configuration. That’s about half of all helm values. Application configuration often accounted for a large proportion of Borg configuration also. I find it unfortunate that the native application format needs to be wrapped in a more complicated syntax that exposes all of the settings in a different manner than what is documented or that the application team understands. (This is one reason I prefer configuration files over command-line flags. Tools like kustomize can embed native configurations in ConfigMaps.)
I tried to task Claude Code with the configuration change we were trying to make. It was able to construct a correct values.yaml file to expose the additional port, configure Let’s Encrypt, and enable logging with one brief prompt. It also informed me which parts of the values.yaml file were relevant:
The key configuration sections are at:
- Ports: traefik/values.yaml:616-754
- ACME/certificates: traefik/values.yaml:831-834
- Logging: traefik/values.yaml:356-396
I then asked Claude about security: “This directory contains a Kubernetes helm chart for Traefik, which is a reverse proxy. What values should I set to implement Kubernetes security best practices?” I know how to set securityContext values in pod templates, but I didn’t want to reverse engineer the chart to figure out which values I needed to set.
What Claude told me is that the chart had good defaults set.
podSecurityContext:
runAsGroup: 65532
runAsNonRoot: true
runAsUser: 65532
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: [ALL]
readOnlyRootFilesystem: true
The securityContext was just inserted as a block, which was relatively straightforward, and I could see it in the output:
{{- with .Values.securityContext }}
securityContext:
{{- toYaml . | nindent 10 }}
{{- end }}
Claude also detailed 11 other groups of security and privacy settings I hadn’t asked about, such as exposing a dashboard, usage reporting, host networking, and so on, which was helpful.
Definitely AI is easier to use than a simple GUI form for navigating this magnitude of complexity, and easier to build than a step-by-step installation decision workflow.
Even so, this experience reinforced that I don’t want the helm chart to be my interface to Traefik or any other off-the-shelf component on a daily basis. Once the initial installation choices are made, I’d rather just deal with standard Kubernetes resources like Service, Deployment, etc., which are literally 10x shorter, syntactically much simpler to read, mechanically easier to change, easier to tweak and extend, well documented, and have lots of tools built around them.
Updating image tags later is easy, and simple tools can make it even easier. If I need to change an application setting, like a flag or an environment variable, I’d hope the application would document them, in --help if nothing else. If it doesn’t, I’m not likely to change the values via a helm chart, either. Moreover, pulling in who-knows-what changes on a helm upgrade isn’t appealing. I’ve heard of many cases where upgrades broke working components, and others where upgrades didn’t complete successfully, such as due to attempting to change immutable fields.
Changing and applying WET configuration is much more predictable. I’m going to stick to that.
Do you find the outputs of helm charts easier to understand than the inputs? Do you use the rendered manifest pattern? If so, how do you use the rendered resource files? Do you lint and/or validate them? Do you review them? Do you use them as the source of truth instead of the original helm chart? Are there tools that help you with that? Or tools that you wish you had so that you could? When a project has a plain YAML installation (e.g., cert-manager, headlamp), do you choose that? Would you like to?
Feel free to email us at hello@confighub.com, or send me a message on LinkedIn, X/Twitter, or Bluesky.
If you found this interesting, you may be interested in other posts in my Infrastructure as Code and Declarative Configuration series and/or Kubernetes series.