[Bug] Unable to call commands on host from within `workload_to_resource_function` method

[rrmistry]

Expected Behavior

def apply_resource_labels(id):
    test = local("echo hello-world", echo=False) # <<< Errors
    return id.name + "-" + test # Should return "example-hello-world"

workload_to_resource_function(apply_resource_labels)

The local function should return hello-world string and continue running the rest of apply_resource_labels function.

Current Behavior

Currently, getting this error:

/workspaces/tilt/example/Tiltfile:8:1: error applying workload_to_resource_function: error determining resource name for 'example:deployment:default:apps': Internal error Starlark not initialized correctly: start tiltfile not found

It is worse if using local_resource because Tilt itself crashes.

See crash output:

Tilt crash console output

panic: internal error: currentExecPath must be called from an active starlark thread

goroutine 15046 [running]: github.com/tilt-dev/tilt/internal/tiltfile/starkit.CurrentExecPath(...) /root/project/internal/tiltfile/starkit/path.go:26 github.com/tilt-dev/tilt/internal/tiltfile/starkit.AbsWorkingDir(0x140046b2458?) /root/project/internal/tiltfile/starkit/path.go:19 +0x90 github.com/tilt-dev/tilt/internal/tiltfile/value.valueToCmdHelper(0x14008f3bd88?, {0x10874c2e0?, 0x14008e02b60}, {0x0?, 0x0?}, 0x0?, 0x1086ff260) /root/project/internal/tiltfile/value/value.go:141 +0xdc github.com/tilt-dev/tilt/internal/tiltfile/value.ValueToHostCmd(...) /root/project/internal/tiltfile/value/value.go:123 github.com/tilt-dev/tilt/internal/tiltfile/value.ValueGroupToCmdHelper(0x0?, {0x10874c2e0?, 0x14008e02b60?}, {0x6?, 0x0?}, {0x0?, 0x0?}, 0x2?) /root/project/internal/tiltfile/value/value.go:116 +0x54 github.com/tilt-dev/tilt/internal/tiltfile.(*tiltfileState).localResource(0x14000868960, 0x140073fddc0, 0x14008bf4ea0, {0x0, 0x0, 0x0}, {0x140083a1680, 0x2, 0x2}) /root/project/internal/tiltfile/local_resource.go:95 +0x5a8 github.com/tilt-dev/tilt/internal/tiltfile/starkit.(*Environment).AddBuiltin.func1(0x14007a6ba10?, 0x23?, {0x0?, 0x0, 0x0}, {0x140083a1680, 0x2, 0x2}) /root/project/internal/tiltfile/starkit/environment.go:121 +0x1ac go.starlark.net/starlark.(*Builtin).CallInternal(0x14007a6ba10?, 0x10874d460?, {0x0?, 0x104dc6744?, 0x14008f3c288?}, {0x140083a1680?, 0x14003eba858?, 0x4?}) /root/project/vendor/go.starlark.net/starlark/value.go:793 +0x38 go.starlark.net/starlark.Call(0x140073fddc0, {0x10874d460?, 0x14008bf4ea0?}, {0x0, 0x0, 0x0}, {0x140083a1680, 0x2, 0x2}) /root/project/vendor/go.starlark.net/starlark/eval.go:1251 +0x20c go.starlark.net/starlark.(*Function).CallInternal(0x14008f53ac0, 0x140073fddc0, {0x14007415750, 0x1, 0x1}, {0x0, 0x0, 0x0}) /root/project/vendor/go.starlark.net/starlark/interp.go:368 +0x2764 go.starlark.net/starlark.Call(0x140073fddc0, {0x10874c3a0?, 0x14008f53ac0?}, {0x14007415750, 0x1, 0x1}, {0x0, 0x0, 0x0}) /root/project/vendor/go.starlark.net/starlark/eval.go:1251 +0x20c github.com/tilt-dev/tilt/internal/tiltfile.makeWorkloadToResourceFunction.func1(0x108724340?, {{0x14007a6ba10, 0x23}, {0x14003eba870, 0x9}, {0x14003eba910, 0xb}, {0x14003eba858, 0x4}}) /root/project/internal/tiltfile/k8s.go:618 +0xc0 github.com/tilt-dev/tilt/internal/tiltfile.(*tiltfileState).workloadToResourceFunctionNames(0x14000868960, {0x1400208c800, 0x6, 0x14008bf4840?}) /root/project/internal/tiltfile/k8s.go:899 +0x1cc github.com/tilt-dev/tilt/internal/tiltfile.(*tiltfileState).calculateResourceNames(0x14000868960, {0x1400208c800?, 0x1400118a000?, 0x14008f3cd18?}) /root/project/internal/tiltfile/k8s.go:880 +0x2c github.com/tilt-dev/tilt/internal/tiltfile.(*tiltfileState).assembleK8sByWorkload(0x14000868960) /root/project/internal/tiltfile/tiltfile_state.go:943 +0x348 github.com/tilt-dev/tilt/internal/tiltfile.(*tiltfileState).assembleK8s(0x14000868960) /root/project/internal/tiltfile/tiltfile_state.go:766 +0x28 github.com/tilt-dev/tilt/internal/tiltfile.(*tiltfileState).assemble(0x14000868960) /root/project/internal/tiltfile/tiltfile_state.go:604 +0x40 github.com/tilt-dev/tilt/internal/tiltfile.(*tiltfileState).loadManifests(0x14000868960, 0x1400164a6e0) /root/project/internal/tiltfile/tiltfile_state.go:246 +0x310 github.com/tilt-dev/tilt/internal/tiltfile.tiltfileLoader.Load({0x140002c45c0, {0x1087551d0, 0x140005ba840}, {0x107296b5b, 0x9}, {0x108712a28, 0x1400063d608}, {{0x1400100b2b0, 0xf}, {0x107290280, ...}, ...}, ...}, ...) /root/project/internal/tiltfile/tiltfile.go:179 +0x42c github.com/tilt-dev/tilt/internal/controllers/core/tiltfile.(*Reconciler).run(0x140005ba9a0, {0x108749428, 0x14003440c30}, {{0x0?, 0x0?}, {0x140011a7f80?, 0xa?}}, 0x1400164a6e0, 0x14007334300, 0x140088bc0e0) /root/project/internal/controllers/core/tiltfile/reconciler.go:317 +0x2ec created by github.com/tilt-dev/tilt/internal/controllers/core/tiltfile.(*Reconciler).startRunAsync in goroutine 1140 /root/project/internal/controllers/core/tiltfile/reconciler.go:293 +0x2fc go-task: Failed to run task "tilt-up": exit status 2

Steps to Reproduce

1. For any existing Tiltfile add the code snippet as shown above under Expected Behavior section
2. Run tilt up and check the Tiltfile resource within the UI or Tilt's console log stream

Context

tilt doctor Output

$ tilt doctor
Tilt: v0.33.12, built 2024-03-29
System: darwin-arm64
---
Docker
- Host: unix:///Users/rohit/.docker/run/docker.sock
- Server Version: 25.0.3
- API Version: 1.44
- Builder: 2
- Compose Version: v2.24.6-desktop.1
---
Kubernetes
- Env: k3d
- Context: k3d-my-cloud
- Cluster Name: k3d-my-cloud
- Namespace: default
- Container Runtime: containerd
- Version: v1.27.4+k3s1
- Cluster Local Registry: &RegistryHosting{Host:localhost:50046,HostFromClusterNetwork:my-cloud:5000,HostFromContainerRuntime:my-cloud:5000,Help:https://k3d.io/stable/usage/registries/#using-a-local-registry,SingleName:,}
---
Thanks for seeing the Tilt Doctor!
Please send the info above when filing bug reports. 💗

The info below helps us understand how you're using Tilt so we can improve,
but is not required to ask for help.
---
Analytics Settings
--> (These results reflect your personal opt in/out status and may be overridden by an `analytics_settings` call in your Tiltfile)
- User Mode: opt-out
- Machine: 5d069c3f9f30c36c6a380a84b3a2b18e
- Repo: 7UMXh1Xbrry+p1bIsU0kYA==

About Your Use Case

Our use case is to call kubectl to extract metadata about Kubernetes resources and then automatically set Tilt resource names + labels (groups).

E.g. We receive the K8sObjectID object in the method sent to workload_to_resource_function, within which we want to call kubectl to extract any resource labels, then call k8s_resource to return the new name + set label using k8s_resource method.

If our flow works then our Tiltfile would contain code like below:

def apply_resource_labels(id):

    resource_value = local(
        command="""
            kubectl --namespace %s get %s %s --output jsonpath='{.metadata.labels.tilt\\.dev\\/resource}'
        """ % (
            id.namespace,
            id.kind.lower(),
            id.name,
        ),
        echo_off=True, # skips printing command to log.
        quiet=True, # skips printing output to log.
    )

    if resource_value:
        resource_value = id.name

    label_value = local(
        command="""
            kubectl --namespace %s get %s %s --output jsonpath='{.metadata.labels.tilt\\.dev\\/label}'
        """ % (
            id.namespace,
            id.kind.lower(),
            id.name,
        ),
        echo_off=True, # skips printing command to log.
        quiet=True, # skips printing output to log.
    )

    if label_value == None:
        if "prometheus" in resource_value:
            label_value = "monitoring"
        elif "ingress-nginx" in resource_value:
            label_value = "ingress"
        # elif ...

    if label_value:
        # Reference: https://docs.tilt.dev/api.html#api.k8s_resource
        k8s_resource(
            workload=resource_value,
            labels=[label_value],
        )

    return resource_value

workload_to_resource_function(apply_resource_labels)

[rrmistry]

Currently working with a non-ideal workaround below:

# Intercept Tilt resource naming and also apply labels to resources
def apply_resource_labels(id):

    # Set the TILT_RESOURCE environment variable
    # This allows us to share information globally between Tiltfile functions scope and within apply_resource_labels function
    resource_identifier = "%s/%s/%s" % (id.namespace, id.kind, id.name)
    if os.getenv("TILT_RESOURCE", None) == None:
        os.environ["TILT_RESOURCE"] = resource_identifier
    else:
        if os.environ["TILT_RESOURCE"].find(resource_identifier) <= -1:
            if os.environ["TILT_RESOURCE"] != "":
                os.environ["TILT_RESOURCE"] += ","
            os.environ["TILT_RESOURCE"] += resource_identifier

    return id.name

# Reference: https://docs.tilt.dev/api#api.workload_to_resource_function
workload_to_resource_function(apply_resource_labels)

if os.getenv("TILT_RESOURCE", None) != None:
    # Will not execute on first run
    # But will execute on subsequent runs (i.e. when Tilt control loop triggers again on file/folder/resource changes)

    for r in os.environ["TILT_RESOURCE"].split(","):

        # Split the resource identifier into namespace, kind and name
        ns = r.split("/")[0]
        kd = r.split("/")[1].lower()
        nm = r.split("/")[2]

        resource_value = str(
            local(
                command="""
                    kubectl --namespace %s get %s %s --output jsonpath='{.metadata.labels.tilt\\.dev\\/resource}' --ignore-not-found=true
                """ % (
                    ns, kd, nm,
                ),
                echo_off=True, # skips printing command to log.
                quiet=True, # skips printing output to log.
            )
        )

        label_value = str(
            local(
                command="""
                    kubectl --namespace %s get %s %s --output jsonpath='{.metadata.labels.tilt\\.dev\\/label}' --ignore-not-found=true
                """ % (
                    ns, kd, nm,
                ),
                echo_off=True, # skips printing command to log.
                quiet=True, # skips printing output to log.
            )
        )

        # # Set sensible default label values
        # if label_value == None or len(label_value) <= 0:
        #     if "prometheus" in resource_value:
        #         label_value = "monitoring"
        #     elif "ingress-nginx" in resource_value:
        #         label_value = "ingress"
        #     # elif ...

        if label_value and len(label_value) > 0:
            if resource_value and len(resource_value) > 0:
                # Reference: https://docs.tilt.dev/api.html#api.k8s_resource
                k8s_resource(
                    workload=nm,
                    new_name=resource_value,
                    labels=[label_value],
                )
            else:
                # Reference: https://docs.tilt.dev/api.html#api.k8s_resource
                k8s_resource(
                    workload=nm,
                    labels=[label_value],
                )

Note: This is not ideal because it will not set resource names / labels on first pass of Tiltfile parsing. On second pass, the TILT_RESOURCE environment variable value will be accessible by rest of Tiltfile functions and so can be used to programmatically access them and detect and set Tilt resource names and labels

[nicks]

thanks for the report!