Showing posts with label Helm. Show all posts
Showing posts with label Helm. Show all posts

Thursday, 16 July 2026

Helm and Infrastructure

 


Does Helm provision aws infrastructure?



The short answer is no, Helm does not natively provision AWS cloud infrastructure.

Helm is a package manager designed specifically for Kubernetes. Its job is to template, package, and deploy Kubernetes-native YAML manifests (like Deployments, Services, ConfigMaps, and Ingresses) onto an already-running cluster. It does not natively understand or manage AWS resources like VPCs, EC2 instances, RDS databases, or S3 buckets.  However, because the cloud native ecosystem is highly integrated, the line can sometimes look blurry. Here is how that boundary gets crossed and how the two worlds interact.

The "Grey Area": When Helm Indirectly Triggers AWS Provisioning

While Helm itself only talks to the Kubernetes API, certain Kubernetes operators and controllers can watch for Helm-deployed resources and provision AWS infrastructure in response:

1. Cloud-Provider Load Balancers

If your Helm chart deploys a Kubernetes Service of type: LoadBalancer:
  • Helm simply submits that Service YAML to the EKS control plane.
  • The cloud-controller-manager inside EKS sees this and automatically provisions an AWS Classic Load Balancer or Network Load Balancer (NLB) in your AWS account.
  • Similarly, deploying an Ingress resource alongside the AWS Load Balancer Controller (often installed via Helm itself) triggers the creation of an AWS Application Load Balancer (ALB).  

2. Custom Resource Definitions (CRDs) & Operators

You can use Kubernetes operators that are explicitly designed to manage cloud infrastructure through Kubernetes. If you install these operators (often using Helm), you can then use Helm to deploy manifests that provision AWS resources:
  • AWS Controllers for Kubernetes (ACK): This framework allows you to define AWS resources (like an S3 bucket or an RDS instance) directly as Kubernetes custom resources. If you bundle an ACK custom resource inside a Helm chart, running helm install will tell the ACK operator to provision that infrastructure in AWS.
  • Crossplane: Similar to ACK, Crossplane turns your cluster into a control plane. You can package Crossplane claims inside Helm charts to spin up databases, networks, and queues.


The Standard Best Practice: "Separation of Concerns"

Because managing cloud infrastructure via Kubernetes manifests can get complex quickly, the industry-standard pattern is to keep these layers strictly separated:


┌────────────────────────────────────────────────────────┐
│                   App Layer (Helm)                     │
│  - Deployments, Services, Ingress, ConfigMaps, etc.    |
└───────────────────────────┬────────────────────────────┘
                            │ Runs inside
┌───────────────────────────▼────────────────────────────┐
│                Kubernetes Cluster (EKS)                │
└───────────────────────────┬────────────────────────────┘
                            │ Provisioned by
┌───────────────────────────▼────────────────────────────┐
│               Infra Layer (IaC - Terraform)            │
│  - VPC, Subnets, EKS Cluster, IAM, RDS, S3, etc.       │
└────────────────────────────────────────────────────────┘



  • Infrastructure as Code (Terraform, OpenTofu, CloudFormation, Pulumi): Used to provision the underlying AWS infrastructure—the VPC, subnets, IAM roles, EKS cluster, and managed node groups.
  • Helm: Used once the cluster is up and running to deploy and configure applications, ingress controllers, logging agents, and service meshes on top of that infrastructure.

Note: If you want a single tool to bridge the gap, Terraform actually has a Helm Provider. This allows you to use Terraform to provision your EKS cluster first, and then immediately call Helm charts within the same Terraform code to bootstrap your cluster applications.


How to use Terraform to install a Helm chart?


To bridge the gap between infrastructure and application deployment, you can use the HashiCorp Helm Provider directly inside your Terraform configuration.

This approach ensures strict ordering: Terraform first provisions the AWS network and EKS cluster, and only after the cluster is fully active does it initialize Helm to install your charts.

Here is a clean, production-ready example showing how to connect your AWS provider, configure the Kubernetes/Helm providers dynamically using the cluster's output token, and deploy a chart.


# 1. PROVIDERS CONFIGURATION
# We configure Kubernetes and Helm providers dynamically using tokens 
# generated from the newly created AWS EKS cluster.

provider "aws" {
  region = "eu-west-2" # London
}

provider "kubernetes" {
  host                   = aws_eks_cluster.main.endpoint
  cluster_ca_certificate = base64decode(aws_eks_cluster.main.certificate_authority[0].data)
  token                  = data.aws_eks_cluster_auth.main.token
}

provider "helm" {
  kubernetes {
    host                   = aws_eks_cluster.main.endpoint
    cluster_ca_certificate = base64decode(aws_eks_cluster.main.certificate_authority[0].data)
    token                  = data.aws_eks_cluster_auth.main.token
  }
}

# Data source to fetch authentication tokens for EKS dynamically
data "aws_eks_cluster_auth" "main" {
  name = aws_eks_cluster.main.name
}

# 2. AWS EKS CLUSTER PROVISIONING (Simplified)
resource "aws_eks_cluster" "main" {
  name     = "production-core-cluster"
  role_arn = aws_iam_role.eks_cluster.arn

  vpc_config {
    subnet_ids = ["subnet-12345678", "subnet-87654321"] # Replace with your VPC subnet IDs
  }
}

# 3. HELM RELEASE PROVISIONING
# Terraform treats this Helm chart as a resource managed within its state file.

resource "helm_release" "ingress_nginx" {
  name             = "ingress-nginx"
  repository       = "https://kubernetes.github.io/ingress-nginx"
  chart            = "ingress-nginx"
  version          = "4.10.0"
  namespace        = "ingress-system"
  create_namespace = true

  # Explicit dependency guarantees the cluster exists before Helm tries to talk to it
  depends_on = [aws_eks_cluster.main]

  # Passing configuration values into the Helm chart
  set {
    name  = "controller.replicaCount"
    value = "3"
  }

  set {
    name  = "controller.service.annotations.service\\.beta\\.kubernetes\\.io/aws-load-balancer-type"
    value = "nlb"
  }
}

Why this pattern works so well:
  • Token-Based Auth Lifecycle: By using the aws_eks_cluster_auth data source, Terraform fetches a short-lived administrative token on the fly during the apply phase, ensuring Helm can authenticate without storing static kubeconfig files on disk or in version control.
  • Escaping Helm Keys: Note the double backslash (\\) in the annotation key. Helm uses dot notation to traverse nested values, so if an AWS annotation key contains a dot (like aws-load-balancer-type), you must escape it in Terraform so Helm interprets it as a single key string rather than a nested map.
  • State Management: When you run terraform destroy, Terraform is smart enough to reverse the order: it calls Helm to cleanly uninstall the charts, deletes any cloud-provider resources (like NLBs) created by those charts, and then tears down the EKS cluster.



Thursday, 26 February 2026

Where to keep Helm chart values in Terraform projects


If we use Terraform to deploy Helm charts, we might be using one of these strategies to keep chart values:

  1. Values are in inline YAML string
  2. Values in separate .yaml file
  3. Values in separate YAML Template files (.yaml.tpl)
  4. Use Helm's set for Dynamic Values
  5. Multiple Values Files

(1) Values in inline YAML string


This is not ideal as problems with Inline YAML in Terraform include:
  • No syntax highlighting or validation - Easy to break YAML formatting
  • Hard to review in diffs - Changes are messy in PRs
  • Can't use standard tooling - No yamllint, Pluto, or other YAML tools
  • Mixing concerns - Infrastructure code mixed with application config
  • Escaping nightmares - Terraform string interpolation conflicts with Helm templating

Example:

resource "helm_release" "app" {
  values = [<<-EOT
    replicaCount: ${var.replicas}
    image:
      repository: myapp
      tag: ${var.tag}
    service:
      type: LoadBalancer
  EOT
  ]
}


(2) Separate Values Files 


Keep values in YAML files, reference them in Terraform.
This is a better approach because:
  • Clean separation
  • Easy to validate with standard tools
  • Better diffs
  • Can use Pluto directly: pluto detect-files -d .

Example:

main.tf:

resource "helm_release" "my_app" {
  name       = "my-app"
  chart      = "my-chart"
  repository = "https://charts.example.com"
  
  values = [
    file("${path.module}/helm-values.yaml")
  ]
}


(3) Templated Values Files


Use Terraform's templatefile() to inject dynamic values:


helm-values.yaml.tpl:

replicaCount: ${replica_count}
image:
  repository: ${image_repo}
  tag: ${image_tag}
ingress:
  enabled: ${enable_ingress}
  host: ${hostname}

main.tf:

resource "helm_release" "my_app" {
  name  = "my-app"
  chart = "my-chart"
  
  values = [
    templatefile("${path.module}/helm-values.yaml.tpl", {
      replica_count  = var.replica_count
      image_repo     = var.image_repository
      image_tag      = var.image_tag
      enable_ingress = var.enable_ingress
      hostname       = var.hostname
    })
  ]
}

Pros:

  • Still gets variable injection
  • Can be validated as YAML (with placeholders)
  • Clean and readable


(4) Use Helm's set for Dynamic Values


Keep static config in files, override specific values:


resource "helm_release" "my_app" {
  name       = "my-app"
  chart      = "my-chart"
  
  # Base values from file
  values = [
    file("${path.module}/helm-values.yaml")
  ]
  
  # Override specific values dynamically
  set {
    name  = "image.tag"
    value = var.image_tag
  }
  
  set {
    name  = "replicaCount"
    value = var.replica_count
  }
  
  set_sensitive {
    name  = "secret.password"
    value = var.db_password
  }
}

Pros:
  • Clear what's dynamic vs static
  • Base values file can be validated
  • Sensitive values handled properly

Here is the example how we can migrate inline YAML from the above to templated file:

helm-values.yaml:

image:
  repository: myapp
service:
  type: LoadBalancer


main.tf:

resource "helm_release" "app" {
  values = [
    file("${path.module}/helm-values.yaml")
  ]
  
  set {
    name  = "replicaCount"
    value = var.replicas
  }
  
  set {
    name  = "image.tag"
    value = var.tag
  }
}

Now we can run: 

% pluto detect-files -f helm-values.yaml



(5) Multiple Values Files


We can layer our configuration:

resource "helm_release" "my_app" {
  name  = "my-app"
  chart = "my-chart"
  
  values = [
    file("${path.module}/helm-values-base.yaml"),
    file("${path.module}/helm-values-${var.environment}.yaml")
  ]
}

---

Friday, 16 May 2025

How to use Helm charts

Case study: we want to install Elasticsearch via Helm chart. 

From  Elastic Stack Helm chart | Elastic Docs we can see that Elastic offers a repository of Helm charts: https://helm.elastic.co

Inspecting the local Helm repositories


Before adding some repo to our local system, we can check if that repo has already been added:

% helm repo list

NAME    URL
stable  https://charts.helm.sh/stable
bitnami https://charts.bitnami.com/bitnami

Each entry includes:
  • NAME: The local alias you’ve given to the repo.
  • URL: The actual remote chart repository URL.

Adding a new Helm repository


We first need to add Elastic Helm repository to our local Helm repository list:

% helm repo add elastic https://helm.elastic.co

We can choose an arbitrary local name for the repository we're adding. We used elastic as repository is provided by Elastic.


The next step is to update information of available charts locally from all added chart repositories, or from the one we've just added:

% helm repo update elastic                                              
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "elastic" chart repository
Update Complete. ⎈Happy Helming!⎈

helm repo update basically downloads all Helm charts from a given repo to our local registry.


To update repo index (fetch latest chart versions) for all local repositories:

% helm repo update


Inspecting charts in a local Helm repository


Let's now list all charts in elastic repository:

% helm search repo elastic
NAME                          CHART VERSION APP VERSION DESCRIPTION                                       
elastic/eck-elasticsearch    0.15.0                    Elasticsearch managed by the ECK operator         
elastic/elastic-agent        9.0.1        9.0.1      Elastic-Agent Helm Chart                          
elastic/elasticsearch        8.5.1        8.5.1      Official Elastic helm chart for Elasticsearch     
elastic/apm-attacher          1.1.3                    A Helm chart installing the Elastic APM Kuberne...
elastic/apm-server            8.5.1        8.5.1      Official Elastic helm chart for Elastic APM Server
elastic/eck-agent            0.15.0                    Elastic Agent managed by the ECK operator         
elastic/eck-apm-server        0.15.0                    Elastic APM Server managed by the ECK operator    
elastic/eck-beats            0.15.0                    Elastic Beats managed by the ECK operator         
elastic/eck-enterprise-search 0.15.0                    Elastic Enterprise Search managed by the ECK op...
elastic/eck-fleet-server      0.15.0                    Elastic Fleet Server as an Agent managed by the...
elastic/eck-kibana            0.15.0                    Kibana managed by the ECK operator                
elastic/eck-logstash          0.15.0                    Logstash managed by the ECK operator              
elastic/eck-operator          3.0.0        3.0.0      Elastic Cloud on Kubernetes (ECK) operator        
elastic/eck-operator-crds    3.0.0        3.0.0      ECK operator Custom Resource Definitions          
elastic/eck-stack            0.15.0                    Elastic Stack managed by the ECK Operator         
elastic/filebeat              8.5.1        8.5.1      Official Elastic helm chart for Filebeat          
elastic/kibana                8.5.1        8.5.1      Official Elastic helm chart for Kibana            
elastic/kube-state-metrics    5.30.1        2.15.0      Install kube-state-metrics to generate and expo...
elastic/logstash              8.5.1        8.5.1      Official Elastic helm chart for Logstash          
elastic/metricbeat            8.5.1        8.5.1      Official Elastic helm chart for Metricbeat        
elastic/pf-host-agent        8.14.3        8.14.3      Hyperscaler software efficiency. For everybody.   
elastic/profiling-agent      9.0.0        9.0.0      Hyperscaler software efficiency. For everybody.   
elastic/profiling-collector  9.0.0        9.0.0      Universal Profiling. Hyperscaler software effic...
elastic/profiling-symbolizer 9.0.0        9.0.0      Universal Profiling. Hyperscaler software effic...



Another way of checking all charts is to download index.yaml file from the remote repository. It contains information of ALL versions of ALL charts in the repo:


% curl https://helm.elastic.co/index.yaml
...
  - apiVersion: v2
    appVersion: 8.15.0
    created: "2024-08-08T09:05:09.582088545Z"
    description: 'Universal Profiling. Hyperscaler software efficiency. For everybody. '
    digest: 9f6a78ed179cda2792259ad7c73db32c2753bf5e3317135fca52fbfb48a8063c
    icon: https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt6ec3007768940247/63337a1f4d11fa0cfdb55244/illustration-deployment-3-arrows.png
    kubeVersion: '>= 1.22.0-0'
    name: profiling-symbolizer
    urls:
    - https://helm.elastic.co/helm/profiling-symbolizer/profiling-symbolizer-8.15.0.tgz
    version: 8.15.0
  - apiVersion: v2
    appVersion: 8.14.3
    created: "2024-07-11T13:35:06.289007371Z"
    description: 'Universal Profiling. Hyperscaler software efficiency. For everybody. '
    digest: 9d1656e80f9c96c3cf7fa2d0692c7318e34e20ff6ad1da13a6b4dae1c82bc990
    icon: https://static-www.elastic.co/v3/assets/bltefdd0b53724fa2ce/blt6ec3007768940247/63337a1f4d11fa0cfdb55244/illustration-deployment-3-arrows.png
    kubeVersion: '>= 1.22.0-0'
    name: profiling-symbolizer
    urls:
    - https://helm.elastic.co/helm/profiling-symbolizer/profiling-symbolizer-8.14.3.tgz
    version: 8.14.3
...

To see only versions of some particular chart e.g. eck-elasticsearch:

% curl -s https://helm.elastic.co/index.yaml | grep eck-elasticsearch
  eck-elasticsearch:
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.15.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.14.1.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.14.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.13.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.12.1.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.12.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.11.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.10.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.9.1.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.9.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.8.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.7.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.7.0-SNAPSHOT.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.6.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.4.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.3.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.2.0.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.1.1.tgz
    name: eck-elasticsearch
    - https://helm.elastic.co/helm/eck-elasticsearch/eck-elasticsearch-0.1.0.tgz
    - condition: eck-elasticsearch.enabled
      name: eck-elasticsearch
    - condition: eck-elasticsearch.enabled
      name: eck-elasticsearch
    ...

As the response is YAML document, we can use yq tool to extract exactly what we need:

% curl -s https://helm.elastic.co/index.yaml | yq '.entries | to_entries | .[].value[] | select(.name == "eck-elasticsearch") | "Name: " + .name + "\nVersion: " + .version + "\n\n"'

Name: eck-elasticsearch
Version: 0.15.0

Name: eck-elasticsearch
Version: 0.14.1

Name: eck-elasticsearch
Version: 0.14.0

Name: eck-elasticsearch
Version: 0.13.0

Name: eck-elasticsearch
Version: 0.12.1

...


Let's say we want to install elastic/eck-elasticsearch chart (note that elastic is the local name for elastic chart repo and eck-elasticsearch is the name of the chart). How can we find its default values?

% helm show values elastic/eck-elasticsearch 
---
# Default values for eck-elasticsearch.
# This is a YAML-formatted file.

# Overridable names of the Elasticsearch resource.
# By default, this is the Release name set for the chart,
# followed by 'eck-elasticsearch'.
#
# nameOverride will override the name of the Chart with the name set here,
# so nameOverride: quickstart, would convert to '{{ Release.name }}-quickstart'
#
# nameOverride: "quickstart"
#
# fullnameOverride will override both the release name, and the chart name,
# and will name the Elasticsearch resource exactly as specified.
#
# fullnameOverride: "quickstart"

# Version of Elasticsearch.
#
version: 9.0.0

# Elasticsearch Docker image to deploy
#
# image:

# Labels that will be applied to Elasticsearch.
#
labels: {}

# Annotations that will be applied to Elasticsearch.
#
annotations: {}

# Settings for configuring Elasticsearch users and roles.
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-users-and-roles.html
#
auth: {}

# Settings for configuring stack monitoring.
# ref: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-stack-monitoring.html
#
monitoring: {}
  # metrics:
  #   elasticsearchRefs:
  #   - name: monitoring
  #     namespace: observability
  # logs:
  #   elasticsearchRefs:
  #   - name: monitoring
  #     namespace: observability
...
...
 
We can save this document into a local yaml file which we can then modify and adjust to our needs:

% helm show values elastic/eck-elasticsearch > eck-elasticsearch-values.yaml


Export only overrides:

helm get values tempo -n grafana-tempo -o yaml > tempo-values.yaml

Export full computed values:

helm get values tempo -n grafana-tempo --all -o yaml > tempo-values-full.yaml

The --all version includes defaults Helm is currently using

If Terraform shows drift, switch to the --all version


How to find what is the latest version of some chart?


Example: We want to find the latest version of chart "tempo" from repository "https://grafana.github.io/helm-charts".

1. Add the Grafana Helm repo (if not already added)

% helm repo add grafana https://grafana.github.io/helm-charts
"grafana" has been added to your repositories

2. Update local index of charts

% helm repo update grafana

Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "grafana" chart repository
Update Complete. ⎈Happy Helming!⎈

3. List all versions of the tempo chart

% helm search repo grafana/tempo --versions
NAME                            CHART VERSION   APP VERSION     DESCRIPTION                                       
grafana/tempo                   1.24.4          2.9.0           Grafana Tempo Single Binary Mode                  
grafana/tempo                   1.24.3          2.9.0           Grafana Tempo Single Binary Mode                  
grafana/tempo                   1.24.2          2.9.0           Grafana Tempo Single Binary Mode                  
...                
grafana/tempo                   0.6.0           v0.6.0          Grafana Tempo Single Binary Mode                  
grafana/tempo                   0.5.0           v0.5.0          Grafana Tempo Single Binary Mode                  
grafana/tempo-distributed       1.61.3          2.9.0           Grafana Tempo in MicroService mode                
grafana/tempo-distributed       1.61.2          2.9.0           Grafana Tempo in MicroService mode                    
...              
grafana/tempo-distributed       0.6.0           0.6.0           Grafana Tempo in MicroService mode                
grafana/tempo-vulture           0.10.1          2.9.0           Grafana Tempo Vulture - A tool to monitor Tempo...
...
grafana/tempo-vulture           0.1.0           0.7.0           Grafana Tempo Vulture - A tool to monitor Tempo...

How to find which resources will Helm chart deploy?


We have few options:

1) Dry-run install the chart and inspect output


% helm install my-fleet-server elastic/eck-fleet-server --dry-run --debug

This will render the templates using default values (or our custom --values file) and print all the generated Kubernetes YAML to stdout.

We need to look for Look for: Deployment, Service, Secret, ConfigMap, Pod or any custom resources (Agent, etc.).

2) Download the chart locally and inspect the templates


% helm pull elastic/eck-fleet-server --untar
% cd eck-fleet-server

Now we can inspect the files under templates/ and values.yaml.

We'll see:
  • All the resource templates (deployment.yaml, service.yaml, etc.)
  • Which fields can be customized.

3) Search the Helm chart source code on GitHub

We can inspect:
  • templates/ folder – actual YAML templates
  • values.yaml – configurable inputs
  • Chart.yaml – metadata


Installing Helm chart


To deploy Helm chart into the Kubernetes cluster by using our own values:

% helm install \
   my-elasticsearch \
   elastic/eck-elasticsearch \
   -f eck-elasticsearch-values.yaml \
   -n elastic-system \
   --create-namespace

We chose to deploy it in a custom namespace which we named elastic-system.


By default, helm install installs the chart into the Kubernetes cluster our kubectl is currently configured to use. Helm relies on the kubeconfig file (typically located at ~/.kube/config) to know which cluster to interact with.

helm install:
  • Reads the kubeconfig file used by kubectl.
  • Connects to the current Kubernetes context (cluster and namespace).
  • Installs the Helm chart to that cluster, unless you override the context or namespace.


We can control the target cluster and namespace using the following:

helm install my-release elastic/eck-elasticsearch --kube-context=my-cluster-context


To list contexts:

kubectl config get-contexts


To switch context:

kubectl config use-context my-cluster-context


Before we attempt to target a remote Kubernetes cluster, we need to ensure that:
  • Our ~/.kube/config contains valid credentials and cluster info.
  • We can interact with it using kubectl (test with kubectl get nodes or kubectl get pods).


To remove the repo from the local system:

% helm repo remove elastic

---

Wednesday, 8 January 2025

How to locally run Helm from a Docker container


Instead of managing a local installation of Helm, I prefer using its latest version via Docker container: alpine/helm - Docker Image | Docker Hub.

% docker run -it --rm  -v ~/.helm:/root/.helm -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm alpine/helm
The Kubernetes package manager

Common actions for Helm:

- helm search:    search for charts
- helm pull:      download a chart to your local directory to view
- helm install:   upload the chart to Kubernetes
- helm list:      list releases of charts

Environment variables:

| Name                               | Description                                                                                                |
|------------------------------------|------------------------------------------------------------------------------------------------------------|
| $HELM_CACHE_HOME                   | set an alternative location for storing cached files.                                                      |
| $HELM_CONFIG_HOME                  | set an alternative location for storing Helm configuration.                                                |
| $HELM_DATA_HOME                    | set an alternative location for storing Helm data.                                                         |
| $HELM_DEBUG                        | indicate whether or not Helm is running in Debug mode                                                      |
| $HELM_DRIVER                       | set the backend storage driver. Values are: configmap, secret, memory, sql.                                |
| $HELM_DRIVER_SQL_CONNECTION_STRING | set the connection string the SQL storage driver should use.                                               |
| $HELM_MAX_HISTORY                  | set the maximum number of helm release history.                                                            |
| $HELM_NAMESPACE                    | set the namespace used for the helm operations.                                                            |
| $HELM_NO_PLUGINS                   | disable plugins. Set HELM_NO_PLUGINS=1 to disable plugins.                                                 |
| $HELM_PLUGINS                      | set the path to the plugins directory                                                                      |
| $HELM_REGISTRY_CONFIG              | set the path to the registry config file.                                                                  |
| $HELM_REPOSITORY_CACHE             | set the path to the repository cache directory                                                             |
| $HELM_REPOSITORY_CONFIG            | set the path to the repositories file.                                                                     |
| $KUBECONFIG                        | set an alternative Kubernetes configuration file (default "~/.kube/config")                                |
| $HELM_KUBEAPISERVER                | set the Kubernetes API Server Endpoint for authentication                                                  |
| $HELM_KUBECAFILE                   | set the Kubernetes certificate authority file.                                                             |
| $HELM_KUBEASGROUPS                 | set the Groups to use for impersonation using a comma-separated list.                                      |
| $HELM_KUBEASUSER                   | set the Username to impersonate for the operation.                                                         |
| $HELM_KUBECONTEXT                  | set the name of the kubeconfig context.                                                                    |
| $HELM_KUBETOKEN                    | set the Bearer KubeToken used for authentication.                                                          |
| $HELM_KUBEINSECURE_SKIP_TLS_VERIFY | indicate if the Kubernetes API server's certificate validation should be skipped (insecure)                |
| $HELM_KUBETLS_SERVER_NAME          | set the server name used to validate the Kubernetes API server certificate                                 |
| $HELM_BURST_LIMIT                  | set the default burst limit in the case the server contains many CRDs (default 100, -1 to disable)         |
| $HELM_QPS                          | set the Queries Per Second in cases where a high number of calls exceed the option for higher burst values |

Helm stores cache, configuration, and data based on the following configuration order:

- If a HELM_*_HOME environment variable is set, it will be used
- Otherwise, on systems supporting the XDG base directory specification, the XDG variables will be used
- When no other location is set a default location will be used based on the operating system

By default, the default directories depend on the Operating System. The defaults are listed below:

| Operating System | Cache Path                | Configuration Path             | Data Path               |
|------------------|---------------------------|--------------------------------|-------------------------|
| Linux            | $HOME/.cache/helm         | $HOME/.config/helm             | $HOME/.local/share/helm |
| macOS            | $HOME/Library/Caches/helm | $HOME/Library/Preferences/helm | $HOME/Library/helm      |
| Windows          | %TEMP%\helm               | %APPDATA%\helm                 | %APPDATA%\helm          |

Usage:
  helm [command]

Available Commands:
  completion  generate autocompletion scripts for the specified shell
  create      create a new chart with the given name
  dependency  manage a chart's dependencies
  env         helm client environment information
  get         download extended information of a named release
  help        Help about any command
  history     fetch release history
  install     install a chart
  lint        examine a chart for possible issues
  list        list releases
  package     package a chart directory into a chart archive
  plugin      install, list, or uninstall Helm plugins
  pull        download a chart from a repository and (optionally) unpack it in local directory
  push        push a chart to remote
  registry    login to or logout from a registry
  repo        add, list, remove, update, and index chart repositories
  rollback    roll back a release to a previous revision
  search      search for a keyword in charts
  show        show information of a chart
  status      display the status of the named release
  template    locally render templates
  test        run tests for a release
  uninstall   uninstall a release
  upgrade     upgrade a release
  verify      verify that a chart at the given path has been signed and is valid
  version     print the client version information

Flags:
      --burst-limit int                 client-side default throttling limit (default 100)
      --debug                           enable verbose output
  -h, --help                            help for helm
      --kube-apiserver string           the address and the port for the Kubernetes API server
      --kube-as-group stringArray       group to impersonate for the operation, this flag can be repeated to specify multiple groups.
      --kube-as-user string             username to impersonate for the operation
      --kube-ca-file string             the certificate authority file for the Kubernetes API server connection
      --kube-context string             name of the kubeconfig context to use
      --kube-insecure-skip-tls-verify   if true, the Kubernetes API server's certificate will not be checked for validity. This will make your HTTPS connections insecure
      --kube-tls-server-name string     server name to use for Kubernetes API server certificate validation. If it is not provided, the hostname used to contact the server is used
      --kube-token string               bearer token used for authentication
      --kubeconfig string               path to the kubeconfig file
  -n, --namespace string                namespace scope for this request
      --qps float32                     queries per second used when communicating with the Kubernetes API, not including bursting
      --registry-config string          path to the registry config file (default "/root/.config/helm/registry/config.json")
      --repository-cache string         path to the directory containing cached repository indexes (default "/root/.cache/helm/repository")
      --repository-config string        path to the file containing repository names and URLs (default "/root/.config/helm/repositories.yaml")

Use "helm [command] --help" for more information about a command.


Example: Adding a Helm chart repository

% docker run -it --rm  -v ~/.helm:/root/.helm -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm alpine/helm repo add elastic https://helm.elastic.co
"elastic" has been added to your repositories


Example: Updating a Helm chart repository

% docker run -it --rm  -v ~/.helm:/root/.helm -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm alpine/helm repo update                             
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "elastic" chart repository
Update Complete. ⎈Happy Helming!⎈



Example: View all configurable values in a chart

% docker run -it --rm  -v ~/.helm:/root/.helm -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm alpine/helm show values elastic/eck-operator
# nameOverride is the short name for the deployment. Leave empty to let Helm generate a name using chart values.
nameOverride: "elastic-operator"

# fullnameOverride is the full name for the deployment. Leave empty to let Helm generate a name using chart values.
fullnameOverride: "elastic-operator"

# managedNamespaces is the set of namespaces that the operator manages. Leave empty to manage all namespaces.
managedNamespaces: []

# installCRDs determines whether Custom Resource Definitions (CRD) are installed by the chart.
# Note that CRDs are global resources and require cluster admin privileges to install.
# If you are sharing a cluster with other users who may want to install ECK on their own namespaces, setting this to true can have unintended consequences.
# 1. Upgrades will overwrite the global CRDs and could disrupt the other users of ECK who may be running a different version.
# 2. Uninstalling the chart will delete the CRDs and potentially cause Elastic resources deployed by other users to be removed as well.
installCRDs: true

# replicaCount is the number of operator pods to run.
replicaCount: 1

image:
  # repository is the container image prefixed by the registry name.
  repository: docker.elastic.co/eck/eck-operator
  # pullPolicy is the container image pull policy.
  pullPolicy: IfNotPresent
  # tag is the container image tag. If not defined, defaults to chart appVersion.
  tag: null
  # fips specifies whether the operator will use a FIPS compliant container image for its own StatefulSet image.
  # This setting does not apply to Elastic Stack applications images.
  # Can be combined with config.ubiOnly.
  fips: false

# priorityClassName defines the PriorityClass to be used by the operator pods.
priorityClassName: ""

# imagePullSecrets defines the secrets to use when pulling the operator container image.
imagePullSecrets: []

# resources define the container resource limits for the operator.
resources:
  limits:
    cpu: 1
    memory: 1Gi
  requests:
    cpu: 100m
    memory: 150Mi

# statefulsetAnnotations define the annotations that should be added to the operator StatefulSet.
statefulsetAnnotations: {}

# statefulsetLabels define additional labels that should be added to the operator StatefulSet.
statefulsetLabels: {}

# podAnnotations define the annotations that should be added to the operator pod.
podAnnotations: {}

## podLabels define additional labels that should be added to the operator pod.
podLabels: {}

# podSecurityContext defines the pod security context for the operator pod.
podSecurityContext:
  runAsNonRoot: true

# securityContext defines the security context of the operator container.
securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL
  readOnlyRootFilesystem: true
  runAsNonRoot: true

# nodeSelector defines the node selector for the operator pod.
nodeSelector: {}

# tolerations defines the node tolerations for the operator pod.
tolerations: []

# affinity defines the node affinity rules for the operator pod.
affinity: {}

# podDisruptionBudget configures the minimum or the maxium available pods for voluntary disruptions,
# set to either an integer (e.g. 1) or a percentage value (e.g. 25%).
podDisruptionBudget:
  enabled: false
  minAvailable: 1
  # maxUnavailable: 3

# additional environment variables for the operator container.
env: []

# additional volume mounts for the operator container.
volumeMounts: []

# additional volumes to add to the operator pod.
volumes: []

# createClusterScopedResources determines whether cluster-scoped resources (ClusterRoles, ClusterRoleBindings) should be created.
createClusterScopedResources: true

# Automount API credentials for the Service Account into the pod.
automountServiceAccountToken: true

serviceAccount:
  # create specifies whether a service account should be created for the operator.
  create: true
  # Specifies whether a service account should automount API credentials.
  automountServiceAccountToken: true
  # annotations to add to the service account
  annotations: {}
  # name of the service account to use. If not set and create is true, a name is generated using the fullname template.
  name: ""

tracing:
  # enabled specifies whether APM tracing is enabled for the operator.
  enabled: false
  # config is a map of APM Server configuration variables that should be set in the environment.
  config:
    ELASTIC_APM_SERVER_URL: http://localhost:8200
    ELASTIC_APM_SERVER_TIMEOUT: 30s

refs:
  # enforceRBAC specifies whether RBAC should be enforced for cross-namespace associations between resources.
  enforceRBAC: false

webhook:
  # enabled determines whether the webhook is installed.
  enabled: true
  # caBundle is the PEM-encoded CA trust bundle for the webhook certificate. Only required if manageCerts is false and certManagerCert is null.
  caBundle: Cg==
  # certManagerCert is the name of the cert-manager certificate to use with the webhook.
  certManagerCert: null
  # certsDir is the directory to mount the certificates.
  certsDir: "/tmp/k8s-webhook-server/serving-certs"
  # failurePolicy of the webhook.
  failurePolicy: Ignore
  # manageCerts determines whether the operator manages the webhook certificates automatically.
  manageCerts: true
  # namespaceSelector corresponds to the namespaceSelector property of the webhook.
  # Setting this restricts the webhook to act only on objects submitted to namespaces that match the selector.
  namespaceSelector: {}
  # objectSelector corresponds to the objectSelector property of the webhook.
  # Setting this restricts the webhook to act only on objects that match the selector.
  objectSelector: {}
  # port is the port that the validating webhook binds to.
  port: 9443
  # secret specifies the Kubernetes secret to be mounted into the path designated by the certsDir value to be used for webhook certificates.
  certsSecret: ""

# hostNetwork allows a Pod to use the Node network namespace.
# This is required to allow for communication with the kube API when using some alternate CNIs in conjunction with webhook enabled.
# CAUTION: Proceed at your own risk. This setting has security concerns such as allowing malicious users to access workloads running on the host.
hostNetwork: false

softMultiTenancy:
  # enabled determines whether the operator is installed with soft multi-tenancy extensions.
  # This requires network policies to be enabled on the Kubernetes cluster.
  enabled: false

# kubeAPIServerIP is required when softMultiTenancy is enabled.
kubeAPIServerIP: null

telemetry:
  # disabled determines whether the operator periodically updates ECK telemetry data for Kibana to consume.
  disabled: false
  # distributionChannel denotes which distribution channel was used to install the operator.
  distributionChannel: "helm"

# config values for the operator.
config:
  # logVerbosity defines the logging level. Valid values are as follows:
  # -2: Errors only
  # -1: Errors and warnings
  #  0: Errors, warnings, and information
  #  number greater than 0: Errors, warnings, information, and debug details.
  logVerbosity: "0"

  # (Deprecated: use metrics.port: will be removed in v2.14.0) metricsPort defines the port to expose operator metrics. Set to 0 to disable metrics reporting.
  metricsPort: 0

  metrics:
    # port defines the port to expose operator metrics. Set to 0 to disable metrics reporting.
    port: "0"
    # secureMode contains the options for enabling and configuring RBAC and TLS/HTTPs for the metrics endpoint.
    secureMode:
      # secureMode.enabled specifies whether to enable RBAC and TLS/HTTPs for the metrics endpoint.
      # * This option makes most sense when using a ServiceMonitor to scrape the metrics and is therefore mutually exclusive with the podMonitor.enabled option.
      # * This option also requires using cluster scoped resources (ClusterRole, ClusterRoleBinding) to
      #   grant access to the /metrics endpoint. (createClusterScopedResources: true is required)
      #
      enabled: false
      tls:
        # certificateSecret is the name of the tls secret containing the custom TLS certificate and key for the secure metrics endpoint.
        #
        # * This is an optional setting and is only required if you are using a custom TLS certificate. A self-signed certificate will be generated by default.
        # * TLS secret key must be named tls.crt.
        # * TLS key's secret key must be named tls.key.
        # * It is assumed to be in the same namespace as the ServiceMonitor.
        #
        # example: kubectl create secret tls eck-metrics-tls-certificate -n elastic-system \
        #            --cert=/path/to/tls.crt --key=/path/to/tls.key
        certificateSecret: ""

  # containerRegistry to use for pulling Elasticsearch and other application container images.
  containerRegistry: docker.elastic.co

  # containerRepository to use for pulling Elasticsearch and other application container images.
  # containerRepository: ""

  # containerSuffix suffix to be appended to container images by default. Cannot be combined with -ubiOnly flag
  # containerSuffix: ""

  # maxConcurrentReconciles is the number of concurrent reconciliation operations to perform per controller.
  maxConcurrentReconciles: "3"

  # caValidity defines the validity period of the CA certificates generated by the operator.
  caValidity: 8760h

  # caRotateBefore defines when to rotate a CA certificate that is due to expire.
  caRotateBefore: 24h

  # caDir defines the directory containing a CA certificate (tls.crt) and its associated private key (tls.key) to be used for all managed resources.
  # Setting this makes caRotateBefore and caValidity values ineffective.
  caDir: ""

  # certificatesValidity defines the validity period of certificates generated by the operator.
  certificatesValidity: 8760h

  # certificatesRotateBefore defines when to rotate a certificate that is due to expire.
  certificatesRotateBefore: 24h

  # disableConfigWatch specifies whether the operator watches the configuration file for changes.
  disableConfigWatch: false

  # exposedNodeLabels is an array of regular expressions of node labels which are allowed to be copied as annotations on Elasticsearch Pods.
  exposedNodeLabels: [ "topology.kubernetes.io/.*", "failure-domain.beta.kubernetes.io/.*" ]

  # ipFamily specifies the IP family to use. Possible values: IPv4, IPv6 and "" (auto-detect)
  ipFamily: ""

  # setDefaultSecurityContext determines whether a default security context is set on application containers created by the operator.
  # *note* that the default option now is "auto-detect" to attempt to set this properly automatically when both running
  # in an openshift cluster, and a standard kubernetes cluster.  Valid values are as follows:
  # "auto-detect" : auto detect
  # "true"        : set pod security context when creating resources.
  # "false"       : do not set pod security context when creating resources.
  setDefaultSecurityContext: "auto-detect"

  # kubeClientTimeout sets the request timeout for Kubernetes API calls made by the operator.
  kubeClientTimeout: 60s

  # elasticsearchClientTimeout sets the request timeout for Elasticsearch API calls made by the operator.
  elasticsearchClientTimeout: 180s

  # validateStorageClass specifies whether storage classes volume expansion support should be verified.
  # Can be disabled if cluster-wide storage class RBAC access is not available.
  validateStorageClass: true

  # enableLeaderElection specifies whether leader election should be enabled
  enableLeaderElection: true

  # Interval between observations of Elasticsearch health, non-positive values disable asynchronous observation.
  elasticsearchObservationInterval: 10s

  # ubiOnly specifies whether the operator will use only UBI container images to deploy Elastic Stack applications as well as for its own StatefulSet image. UBI images are only available from 7.10.0 onward.
  # Cannot be combined with the containerSuffix value.
  ubiOnly: false

# Prometheus PodMonitor configuration
# Reference: https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#podmonitor
podMonitor:

  # enabled determines whether a podMonitor should deployed to scrape the eck metrics.
  # This requires the prometheus operator and the config.metrics.port not to be 0
  enabled: false

  # labels adds additional labels to the podMonitor
  labels: {}

  # annotations adds additional annotations to the podMonitor
  annotations: {}

  # namespace determines in which namespace the podMonitor will be deployed.
  # If not set the podMonitor will be created in the namespace where the Helm release is installed into
  # namespace: monitoring

  # interval specifies the interval at which metrics should be scraped
  interval: 5m

  # scrapeTimeout specifies the timeout after which the scrape is ended
  scrapeTimeout: 30s

  # podTargetLabels transfers labels on the Kubernetes Pod onto the target.
  podTargetLabels: []

  # podMetricsEndpointConfig allows to add an extended configuration to the podMonitor
  podMetricsEndpointConfig: {}
  # honorTimestamps: true

# Prometheus ServiceMonitor configuration
# Only used when config.enableSecureMetrics is true
# Reference: https://github.com/prometheus-operator/prometheus-operator/blob/master/Documentation/api.md#servicemonitor
serviceMonitor:
  # This option requires the following settings within Prometheus to function:
  # 1. RBAC settings for the Prometheus instance to access the metrics endpoint.
  #
  # - nonResourceURLs:
  #   - /metrics
  #   verbs:
  #   - get
  #
  # 2. If using the Prometheus Operator and your Prometheus instance is not in the same namespace as the operator you will need
  #    the Prometheus Operator configured with the following Helm values:
  #
  #   prometheus:
  #     prometheusSpec:
  #       serviceMonitorNamespaceSelector: {}
  #       serviceMonitorSelectorNilUsesHelmValues: false
  #
  # allows to disable the serviceMonitor, enabled by default for backwards compatibility
  enabled: true
  # namespace determines in which namespace the serviceMonitor will be deployed.
  # If not set the serviceMonitor will be created in the namespace where the Helm release is installed into
  # namespace: monitoring
  # caSecret is the name of the secret containing the custom CA certificate used to generate the custom TLS certificate for the secure metrics endpoint.
  #
  # * This *must* be the name of the secret containing the CA certificate used to sign the custom TLS certificate for the metrics endpoint.
  # * This secret *must* be in the same namespace as the Prometheus instance that will scrape the metrics.
  # * If using the Prometheus operator this secret must be within the `spec.secrets` field of the `Prometheus` custom resource such that it is mounted into the Prometheus pod at `caMountDirectory`, which defaults to /etc/prometheus/secrets/{secret-name}.
  # * This is an optional setting and is only required if you are using a custom TLS certificate.
  # * Key must be named ca.crt.
  #
  # example: kubectl create secret generic eck-metrics-tls-ca -n monitoring \
  #            --from-file=ca.crt=/path/to/ca.pem
  caSecret: ""
  # caMountDirectory is the directory at which the CA certificate is mounted within the Prometheus pod.
  #
  # * You should only need to adjust this if you are *not* using the Prometheus operator.
  caMountDirectory: "/etc/prometheus/secrets/"
  # insecureSkipVerify specifies whether to skip verification of the TLS certificate for the secure metrics endpoint.
  #
  # * If this setting is set to false, then the following settings are required:
  #   - certificateSecret
  #   - caSecret
  insecureSkipVerify: true

# Globals meant for internal use only
global:
  # manifestGen specifies whether the chart is running under manifest generator.
  # This is used for tasks specific to generating the all-in-one.yaml file.
  manifestGen: false
  # createOperatorNamespace defines whether the operator namespace manifest should be generated when in manifestGen mode.
  # Usually we do want that to happen (e.g. all-in-one.yaml) but, sometimes we don't (e.g. E2E tests).
  createOperatorNamespace: true
  # kubeVersion is the effective Kubernetes version we target when generating the all-in-one.yaml.
  kubeVersion: 1.21.0