live-update to a CRD without an image

[nicks]

The Problem

We're seeing more and more frameworks in Kubernetes that use CRDs that don't have images in them. Maybe the CRD builds its own image. Or maybe it forces you to choose from a limited set of pre-existing images.

As an example, let's say you want to live-update files to the pods created by Prometheus CRD: https://github.com/tilt-dev/tilt-example-frameworks/blob/master/prometheus/prometheus.yaml

Tilt's deploy pipeline works like this:

• tilt organizes kubernetes objects into groups
• tilt builds and inject images into the spec yaml, then apply the yaml to the cluster
• tilt tracks which pods belong to which objects
• tilt live-updates the containers that matches the images

Image names are an essential part of all these steps - to select objects to track, then to select containers to inject into.

Proposal

We mostly have a way to group and track objects without images. It looks like this:

k8s_resource(
  new_name='app',
  objects=[...], # object spec
  extra_pod_selectors=[{...}])

This mostly works, but is a bit manual (in a lot of cases, it's hard to figure out the labels in advance). It's also missing a way to select the containers to live-update. It maps roughly to the KubernetesDiscovery object (https://api.tilt.dev/kubernetes/kubernetes-discovery-v1alpha1.html).

I think, in the short term, we should add an API spec and function for live_update objects that "connects" to the underlying pod discovery.

live_update(
  name='app-live-update',
  kubernetes_discovery_name='app',
  steps=[...],
  containers=[...])

where the containers= argument is a system for selecting the containers to live-update, by name or by image.

As an aside - in the long-term, I'd love to have a tiltfile API that more closely matches the REST API, so you could imagine something like:

kubernetes_apply(name=..., yaml=..., ,kubernetes_discovery_template=...)
live_update(...)

That would save you the effort of manually grouping objects and setting labels. But i think we can punt on it for now.

[nicks]

Related issue: https://github.com/tilt-dev/tilt/issues/3282 (though I think the overlap between this one and that one is very implementation-dependent and not obvious)

[milas]

With #5248 it's possible to do this for the k8s_custom_deploy case by using the container_selector argument: https://docs.tilt.dev/api.html#api.k8s_custom_deploy

The more general use case of being able to set up a Live Update outside of an image build / custom deploy is still valid, however.

[marcstreeter]

@milas the more general case you're referring to would apply to scenarios like the following is my hope:

apiVersion: batch/v1
kind: Job
metadata: ...
spec:
  template:
    spec:
      labels: ...
      containers:
      - name: seeder
        image: python:3.14-slim
        env: ...
        command: ["/bin/sh", "-c"]
        args:
        - |
          echo "Running seed script..."
          python /seed/seeder.py

Would be nice if local changes to the above ^ seeder.py could reflect and trigger the job again