Tork worker inside script doesn't see host env vars

[ppcololo]

I faced an issue when host\k8s pod has some env values and I want to use them (mount values from k8s secrets) but tork inside task doesn't see any env values at all. If I check env command I can see only this:

TORK_OUTPUT=/tmp/tork4231472022/stdout
SHLVL=2
_=/usr/bin/env
PWD=/tmp/tork4231472022

But host itself has a lot of values I want to use. And the most strange thing: some time ago it worked fine. Maybe I'm missing something?

[runabol]

Host env vars are never automatically exposed to the container. Are you using env on the task?

[ppcololo]

I have some secrets in the POD, so I don't want to set those secrets like passwords or connection strings directly in Tork Job. I want tork worker to fetch POD env variables and pass to the Job

[runabol]

Look into the hostenv middleware: https://www.tork.run/customize#node-middleware

Might do what you want.

[ppcololo]

It could help but is it possible to insert all env vars or env vars by prefix? POD could be restarted and fetch new ENV from secret and ConfigMap and for changing ENV for middleware I have to rebuild docker image and place there new config.

Is there another way to do this?

[runabol]

You can write a custom middleware to expose all headers on host.

[ppcololo]

@runabol could you correct me? I set this in the config:

tork-worker-7998c84f98-nf96k:/# cat /etc/tork/config.toml
...
[middleware.task.hostenv]
vars = ["SOME_ENV"]

I can see this env inside host env

tork-worker-7998c84f98-nf96k:/# env | grep SOME_ENV
SOME_ENV=SOME_VALUE

But when I create simple job:

id: 8f8338223def40149fe758007b5e2a16
jobId: 1e3658b92db74958b9550f9f5bafcada
position: 1
name: my first task
state: COMPLETED
createdAt: 2024-03-28T08:11:26.337853Z
scheduledAt: 2024-03-28T08:11:26.351893Z
startedAt: 2024-03-28T08:11:26.383202Z
completedAt: 2024-03-28T08:11:26.399187Z
run: env
queue: default
nodeId: AwLFEhtVEgg6uDmmTaC9qc

I can see only this:

TORK_OUTPUT=/tmp/tork4188699852/stdout
SHLVL=1
_=/usr/bin/env
PWD=/tmp/tork4188699852
[90m8:11AM[0m [32mINF[0m Config loaded from /etc/tork/config.toml

[runabol]

@ppcololo this looks right. Are you using the latest build?

[ppcololo]

yes, 0.1.71 on coordinator and worker

[molowny]

I've tested version 0.1.99, and it seems to have a regression when passing the host environment using middleware.task.hostenv. It's not working with the Docker setup.

[runabol]

@molowny @ppcololo This should be fixed now. Mind re-testing with the latest version?