Helm Chart v4: graceful migration from v3

[awoimbee]

Hello!

• Vote on this issue by adding a 👍 reaction
• If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

I'm trying to migrate from the helm Chart resource v3 to v4, but pulumi wants to redeploy the helm charts, and since it creates new resources before deleting old ones I get name conflicts.

There should be a clean migration path. Personally I would be grateful even of a pulumi stack export [...] && sed [...] && pulumi stack import solution.

Affected area/feature

kubernetes.helm.v4.Chart

[EronWright]

Sorry for the inconvenience, it is an aspect we'd like to improve upon.

A possible workaround is to use a transform function to apply an "alias" option to each child. The alias option lets you "move" a resource by defining its prior parent URN and its prior name.

[awoimbee]

Since I don't want to bloat the "codebase" with aliases everywhere I have resorted to stuff like:

pulumi -s <ENV> stack export | sed -E \
    -e 's|urn:(.*)v3:Chart(.*cert-manager.*)::(.*)|urn:\1v4:Chart\2::cert-manager:\3|g' \
    -e 's|urn:(.*)v3:Chart(.+)::(.*cert-manager.*)|urn:\1v4:Chart\2::cert-manager:\3|g' \
    -e 's|v3:Chart::cert-manager"|v4:Chart::cert-manager"|g' \
    > stack.json
pulumi -s <ENV> stack import --file stack.json

The issue is that Chart v4 adds the release name to all resources compared to Chart v3 so sed can't be used alone as a generic migration script, otherwise this technique works great (as long as you up, refresh, up to avoid drift)

[blampe]

We'd like to add some documentation around how to accomplish this using transforms since it's fairly involved.

[mortaelth]

this helped me to migrate:

in the v4 in the ComponentResourceOptions I am using something like this:

{
  parent: this,
  aliases: [{type: "kubernetes:helm.sh/v3:Chart"],
  transforms: [args => {
    return {
        props: args.props,
        opts: pulumi.mergeOptions(args.opts, {
            aliases: [{ parent: "<urn of the helm v3 resource>"}]
        })
    };
  }],
}

// quite a recent versions of @pulumi/kubernetes and @pulumi/pulumi required as of today

update: I have just noticed the "aliases" is not required, transforms are enough

[xSAVIKx]

Wanted to wonder if there's any guidelines on how to manage outputs better with the v4 Chart?

The v3 chart had getResource typed API which was very helpful.

[awoimbee]

Thanks @mortaelth, here is the generic solution I built from your snippet:

EDIT: here is the full version of what I'm using:

// lib/helm.ts
import assert from "node:assert/strict";

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";
import * as yaml from "js-yaml";
import semver from "semver";

import { notNull } from "./index.js";

async function notifyLatestChartVersion (repo: string, chart: string, version: string) {
  const helmIndexUrl = `${repo}/index.yaml`;
  const resp = await fetch(helmIndexUrl);
  const helmIndex = yaml.load(await resp.text()) as any;
  const releases = helmIndex?.entries?.[chart];
  let latestMetadata;
  if (semver.prerelease(version)) {
    latestMetadata = releases?.[0];
  } else {
    // skip prereleases
    latestMetadata = releases?.find((r: any) => semver.prerelease(r.version) === null);
  }
  const latestVersion = latestMetadata?.version as string | undefined;
  if (latestVersion === null || latestVersion === undefined) {
    console.error(`Could not fetch latest version of '${chart}' !`);
    return;
  }
  if (semver.satisfies(latestVersion, version)) {
    return;
  }
  console.warn(`New chart version available: ${chart} '${version}' => '${latestVersion}'.`);
}

/**
 * `k8s.helm.v3.Chart` with the added features:
 *   * Prints a warning when the chart is out of date
 *   * Graceful migration from v3 Chart
 * Useful for cases where resources with helm hooks should be deployed (as v4 ignores them).
 */
export class ChartV3 extends k8s.helm.v3.Chart {
  constructor (
    releaseName: string,
    config: k8s.helm.v3.ChartOpts | k8s.helm.v3.LocalChartOpts,
    opts?: pulumi.ComponentResourceOptions
  ) {
    // notify when new version is out
    if ("fetchOpts" in config && config.fetchOpts !== undefined && config.version !== undefined) {
      const repo = pulumi.output(config.fetchOpts).apply(f => notNull(f.repo));
      pulumi.all([repo, config.chart, config.version]).apply(args =>
        notifyLatestChartVersion(...args).catch(e => console.error(`Could not notify latest chart version of ${releaseName}: ${e}`))
      );
    }

    // gracefully migrate from v4
    const project = pulumi.getProject();
    const stack = pulumi.getStack();
    config.transformations ??= [];
    config.transformations.push(
      (o: any, opts: pulumi.CustomResourceOptions) => {
        const namespace = o.metadata?.namespace ? o.metadata.namespace + "/" : "";
        const apiVersion = o.apiVersion === "v1" ? "core/v1" : o.apiVersion;
        const helmV4Urn = `urn:pulumi:${stack}::${project}::kubernetes:helm.sh/v4:Chart$kubernetes:${apiVersion}:${o.kind}::${releaseName}:${namespace}${o.metadata?.name}`;
        opts.aliases ??= [];
        opts.aliases.push(helmV4Urn);
      }
    );
    super(releaseName, config, opts);
  }
}

/**
 * `k8s.helm.v4.Chart` with the added features:
 *   * Prints a warning when the chart is out of date
 *   * Graceful migration from v3 Chart
 */
export class ChartV4 extends k8s.helm.v4.Chart {
  constructor (
    releaseName: string,
    config: k8s.helm.v4.ChartArgs,
    opts?: pulumi.ComponentResourceOptions
  ) {
    // notify when new version is out
    if (config.repositoryOpts !== undefined && config.version !== undefined) {
      const repo = pulumi.output(config.repositoryOpts).apply(f => notNull(f.repo));
      pulumi.all([repo, config.chart, config.version]).apply(args =>
        notifyLatestChartVersion(...args).catch(e => console.error(`Could not notify latest chart version of ${releaseName}: ${e}`))
      );
    }

    // gracefully migrate from v3
    const project = pulumi.getProject();
    const stack = pulumi.getStack();
    opts ??= {};
    opts.transforms ??= [];
    opts.transforms.push(({ props, opts: rOpts }) => {
      if (!props.apiVersion) {
        // ignore the chart resource itself
        return;
      }
      const namespace = props.metadata?.namespace ? props.metadata.namespace + "/" : "";
      const apiVersion = props.apiVersion === "v1" ? "core/v1" : props.apiVersion;
      const helmV3Urn = `urn:pulumi:${stack}::${project}::kubernetes:helm.sh/v3:Chart$kubernetes:${apiVersion}:${props.kind}::${namespace}${props.metadata?.name}`;
      if (releaseName === "trow") {
        console.log(helmV3Urn);
      }
      return {
        props,
        opts: pulumi.mergeOptions(rOpts, {
          aliases: [helmV3Urn],
          provider: opts.provider
        })
      };
    });
    super(releaseName, config, opts);
  }
}

/** remake of ChartV3.getResource() for Chart v4 */
export function getResource<T extends pulumi.Resource=pulumi.Resource> (
  chart: k8s.helm.v4.Chart,
  match: pulumi.Input<string>
): pulumi.Output<T> {
  const resources = chart.resources.apply(
    resources => pulumi.all(
      resources.map(
        r => {
          const namespace = pulumi.output(r.metadata?.namespace).apply(n => n || "");
          return pulumi.all([
            r as pulumi.Resource,
            pulumi.interpolate`${r.apiVersion}:${r.kind}:${namespace}:${r.metadata?.name}`
          ]);
        }
      )
    )
  );
  return pulumi.all([resources, match]).apply(([resources, match]) => {
    const matchingResources = resources.filter(([_, name]) => name === match);
    const matchingResourceNames = matchingResources.map(([_, n]) => n);
    const resourceNames = resources.map(([_, name]) => name);

    assert.equal(matchingResources.length, 1, `Exactly 1 resource should match '${match}'\nMatched: '${matchingResourceNames}'\nOut of: ${resourceNames}`);
    return matchingResources[0][0] as T;
  });
}

I think everything in the snippet above should be part of pulumi-kubernetes. BTW looking at the code for helm.v3.Chart, it sets aliases to helm.v2.Chart, helm.v4.Chart should do the same towards helm.v3.Chart.

[johnsonp57]

@EronWright I'm trying to migrate to Chart.V4, but Transforms don't seem to apply when using C# - this is a snippet of what I am using to add an alias for the old v3 Chart, but Pulumi still creates new resources:

ResourceTransforms =
[
   async (args, _) =>
    {
        if (args.Type.StartsWith("kubernetes")) {
            var options = CustomResourceOptions.Merge(
                (CustomResourceOptions) args.Options,
                new CustomResourceOptions
                {
                    Aliases = [
                        new Alias
                        {
                            Type = "kubernetes:helm.sh/v3:Chart"
                        }
                    ],
                    IgnoreChanges = new List<string> {
                        "metadata.labels"
                    }
                });
            return new ResourceTransformResult(args.Args, options);
        }

        return null;
    }
]

There does not seem to be a C# example on the blog page here: https://www.pulumi.com/blog/kubernetes-chart-v4/#new-style-pulumi-transformations