bottlerocket
bottlerocket copied to clipboard
bottlerocket-os
Consider adding a mechanism for running a script specified in userdata
What I'd like: The ability to specify a bootstrap script that gets run by an existing bottlerocket-provided bootstrap container.
Check out the discussion in #1989, particularly this comment
Any alternatives you've considered: Users can currently provide arbitrary bootstrap containers to accomplish the same outcomes, this would just make it easier, since it would eliminate the need to publish and maintain an additional container.
This would be a really useful addition, I want to test some dynamic configuration for nodes created by Karpenter and having a bootstrap container available would allow me to do this easily. I'm currently weighing up the cost/benefit of rolling my own container to test the behaviour knowing that even if it works I'd still need any resulting implementation to take a dependency on maintaining a custom container.
@stmcginnis I might be misunderstanding this but I'd assume that this can be resolved simply by adding a new repo to build an OCI image based on an existing ECR public image (Bash?) with the entrypoint set to the userdata target and published to ECR. Am I missing some additional complexity other than automating the image to be re-built when the FROM image changes (SSM?)?
I think you meant to tag @cbgbt :)
But yes, I think it would just need to be a bootstrap container that contains bash (or whatever your preferred interpreter is - python, etc) that is configured with the proper mounts.
I think the intent with this issue is to have an easy way, with documentation, that someone could take a script and combine it with a well known published container image to quickly have a bootstrap container that would execute their script.
@stevehipwell https://github.com/bottlerocket-os/bottlerocket/discussions/1989#discussioncomment-5291991
@James-Quigley that doesn't really help my scenario of needing to set dynamic API values based on the instance type. I'm after a bash based bootstrap container with the correct entrypoint to allow me to do this with only config inputs.
The image I linked allows for running an abitrary bash script. You define the bash script as the userdata for the bootstrap container. If you write a bash script for looking up the instance type and then running apiclient set commands, you could likely accomplish what you are looking for without needing to build your own container
Sorry @James-Quigley the link looked to be pointing at a completely different comment. I'll give your image a go to do the POC work as it'll save me making a custom image, but the real request here is to have an image in ECR which is kept up to date and can be implicitly trusted by virtue of ownership (and signature/attestation if possible).
I think https://github.com/bottlerocket-os/bottlerocket/pull/4131 covers some of this use case, but I think we're still missing some capabilities.
@stevehipwell this issue is to track the Bottlerocket bootstrap container which lives in https://github.com/bottlerocket-os/bottlerocket-bootstrap-container. The intention of the bootstrap container is to allow a Bottlerocket user to provide their own script to run bootstrap commands to setup their own configuration during runtime.
#4131 pertains to bootstrap commands. Bootstrap commands are a new systemd service that enable users to run predefined commands through the Bottlerocket API.
As you've pointed out in other issues, we're still working on proper documentation for these features; and in the case of this issue, we're still working on the implementation to integrate the bootstrap container into the bottlerocket-os/bottlerocket project :D thanks for your patience as we get all this sorted.
@ginglis13 I'm interested if any of the actions which would currently require a script could be implemented as part of the API as was done for raiding the NVMe disks? I'm guessing this would be of limited value as the use cases I have are generally based on collecting data about the VM and then setting an API value in response.
@stevehipwell, we did an apiclient "action" that will help with setting up the RAID arrays [see https://github.com/bottlerocket-os/bottlerocket-core-kit/pull/15], but as @ginglis13 we are working on better documentation for how to use them in conjunction with Bootstrap Commands.
Regarding getting data from the instance, and applying values in response, what information are you gathering? I'm asking to understand a bit more of your use case.
@arnaldo2792 for example I might want to calculate the kube-reserved memory for a node based on it's total memory using the GKE calculation logic.
@stevehipwell - yes, thanks!
It's now possible to run a script using the default bootstrap container without specifying a source. For example:
[settings.bootstrap-containers.hello-world]
mode = "always"
essential = false
user-data = "IyEvYmluL2Jhc2gKZWNobyBIRUxMTyBXT1JMRA=="
Where the user data is a base64-encoded script like this:
❯ echo 'IyEvYmluL2Jhc2gKZWNobyBIRUxMTyBXT1JMRA==' | base64 -d
#!/bin/bash
echo HELLO WORLD
