Environment variables possible unset on starlak

[abravalheri]

Hello guys,

I have been playing around with defining the CI pipeline with starlak (https://github.com/abravalheri/cirrus-starlak-helpers/commit/b06ea0fe1f2404566134134535b782712838446d and https://github.com/abravalheri/dummy-cirrusci-starlak-experiment). Congratulations btw, it looks very promising.

While the docs seem to indicate that the predefined environment variables listed in the YAML docs are available by using load("cirrus", "env"), it seems that not all of them can be used.

I did a quick experiment:

def main():
    print("CIRRUS_PR", environ.get("CIRRUS_PR"))
    print("CIRRUS_REPO_CLONE_TOKEN exists", "CIRRUS_REPO_CLONE_TOKEN" in environ)
    print("CIRRUS_REPO_FULL_NAME", environ.get("CIRRUS_REPO_FULL_NAME"))
    print("CIRRUS_CHANGE_IN_REPO", environ.get("CIRRUS_CHANGE_IN_REPO"))
    print("CIRRUS_WORKING_DIR", environ.get("CIRRUS_WORKING_DIR"))
    return [ ... ]

that seems to indicate that CIRRUS_REPO_CLONE_TOKEN and CIRRUS_WORKING_DIR are missing, for example...

1 | CIRRUS_PR None
2 | CIRRUS_REPO_CLONE_TOKEN exists False
3 | CIRRUS_REPO_FULL_NAME abravalheri/dummy-cirrusci-starlak-experiment
4 | CIRRUS_CHANGE_IN_REPO b2d53b5fa95ec2b14b0888aed657094a12586f0c
5 | CIRRUS_WORKING_DIR None
6 | Traceback (most recent call last):
7 | .cirrus.star:18:19: in main
8 | .cirrus.star:32:30: in linux_task
9 | lib.star@add-deep-clone:90:179: in github_deep_clone
10 | Error: key not found: CIRRUS_REPO_CLONE_TOKEN

Am I doing something wrong here? I could not find anything in the docs indicating that some env vars are not available in starlak.

(The use case is mainly for writing a OS-independent version of the deep clone script listed in the tips section of the docs - but I suppose knowing which variables are not available might be useful).

Expected Behavior

I would suppose all env vars available for .cirrus.yml are also available in .cirrus.star, mainly because of the link in the starlak guides.

Real Behavior

Some env vars seem to be missing, but others are available...

[fkorotkov]

Hey @abravalheri,

Right now these two variable are available at task execution time but not at the configuration parsing/evaluation phase.

The working directory part is tricky since Cirrus agent tries the default one but might pick a different one if the default one is already created for some reason.

With the token the situation is easier. We can add a read-only token for PRs and a write capable for other builds similar to how Cirrus passes a token to the agent to do the actual clone during execution.

For your use case with a custom clone, you can just generate scripts that will use CIRRUS_REPO_CLONE_TOKEN and when tasks will be executed this variable will be available.

[abravalheri]

Thanks @fkorotkov. I suppose there is no way yet to write that custom clone script without having to differentiate between POSIX shells and Windows shells...

[abravalheri]

@fkorotkov, is there a list of which vars are not set at the moment the starlak scripts run? I think that would be very useful in the docs...

[fkorotkov]

I'll update the docs a bit later today. If you are generating the whole task then you should know the instance type and therefore the platform. You can probably check whether the instance is windows_container or has field platform set to windows. 🤔

[abravalheri]

Thank you @fkorotkov. I changed the implementation to defer the env variables to runtime.

I am not sure if I understood your tip... How could I inspect the task while defining it? Is it somehow related to the ctx parameter?

For the time being I am using an explicit argument. It seems to be working, as shown in this build example. Is there a better way of doing it?

I don't know if this clone workaround is common enough to justify, but, would you guys be interested in a PR for the https://github.com/cirrus-modules/helpers repository adding the github_deep_clone function?

[fkorotkov]

I was thinking about having a function like this:

def use_deep_clone(task):
  # ...

Which will check for "windows_container" in task to determine/guess the OS and then do something like this in the end to add the clone script:

new_task = {}
new_task.update(script("clone", command))
new_task.update(task.items())
return new_task

And yes, of course we'll be interested in a PR!

[abravalheri]

Thank you very much for the help @fkorotkov, I submitted a PR with the new functions.