feat(install): Adds support for podman(compose)

[DuncanConroy]

Adds support for podman(compose), while maintaining compatibility with docker

Introduces a new script to detect podman vs. docker. Distinguishes between docker and podman minimum versions and substitutes uses of docker with a variable instead.

Closes https://github.com/getsentry/self-hosted/issues/369

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

[BYK]

FYI tests failed with the following error:

Detecting Docker platform FAIL: Unsupported docker architecture amd64.

[DuncanConroy]

Quite surprised (positively!) that the changes needed are so few! Added a few comments and on top of those, we definitely should have a test using podman now to make sure this actually works and does not break in the future (can help with this if you need me).

Thanks so much for taking the effort!

I agree that having a test would be great. In the best case, we can run the whole suite for both docker and podman. I'm just unsure about the environment this is run on and would welcome your input and ideas very much!

[DuncanConroy]

Have converted back to draft, as there are some tweaks to make, still.

[aldy505]

@DuncanConroy Let us know if you need help in terms of anything, we'd be happy to help you. Having this PR really made us happy :)

[BYK]

~~@DuncanConroy if you can check the "maintainers can modify the PR" checkbox for your PR, I'll push a fix for the test failure and try to run tests for Podman too.~~

Okay it's not that, your master branch is write protected. Are you okay disabling it? Otherwise I'll create a new branch from this PR and we can continue there.

[DuncanConroy]

Sorry for the late reply. There were some changes waiting in my commit list. As I was testing this in an air-gapped environment, it wasn't as straight forward. I have checked the repositroy settings and couldn't find any protection, but have invited you as a collaborator. Anyway, I hope these last commits have all the changes necessary (as well as test fix).

Unfortunately, I'm switching projects work-wise, and won't be able to come up with more input on this. :/

[BYK]

Yeah, apparently this needs more fixes for the test. I'll follow up later.

[DuncanConroy]

Sorry for the late reply. There were some changes waiting in my commit list. As I was testing this in an air-gapped environment, it wasn't as straight forward.

@DuncanConroy I wonder how did you manage to test this Podman Compose thing on air-gapped environment?

Built the docker images locally on my machine, then used docker save to export them to .tar, as mentioned in the docs. Copied them over to the machine and imported the images over there. Getting this up and running on RHEL with SeLinux was also quite a challenge with many manual steps. I've created another branch air-gapped in my forked repo, but that's far from production ready and doesn't include the manual steps for SeLinux. Feel free to have a look.

Unfortunately, we didn't quite got replays to work, which was the main reason we tried sentry. And as we had a very restricted network and policies, there was no option to use the cloud version.

[BYK]

@DuncanConroy podman compose run calls add a weird hash at the beginning (see https://github.com/getsentry/self-hosted/actions/runs/14732417876/job/41349793317?pr=3673#step:4:890)

Any idea how to suppress that?

[DuncanConroy]

@DuncanConroy podman compose run calls add a weird hash at the beginning (see https://github.com/getsentry/self-hosted/actions/runs/14732417876/job/41349793317?pr=3673#step:4:890)

Any idea how to suppress that?

I suppose you mean the hash that's written to the credentials.json file before the JSON? I don't have a clue, unfortunately. Didn't encounter that myself.

[aldy505]

Sorry for the late reply. There were some changes waiting in my commit list. As I was testing this in an air-gapped environment, it wasn't as straight forward.

@DuncanConroy I wonder how did you manage to test this Podman Compose thing on air-gapped environment?

Built the docker images locally on my machine, then used docker save to export them to .tar, as mentioned in the docs. Copied them over to the machine and imported the images over there. Getting this up and running on RHEL with SeLinux was also quite a challenge with many manual steps. I've created another branch air-gapped in my forked repo, but that's far from production ready and doesn't include the manual steps for SeLinux. Feel free to have a look.

I'm more interested on your war story with SELinux. I'd appreciate it if you share that with us later. Also, I'm glad the docker save thing did it for you, I was the one who wrote that docs 😆

Unfortunately, we didn't quite got replays to work, which was the main reason we tried sentry. And as we had a very restricted network and policies, there was no option to use the cloud version.

We'll help you on that. My hunch is related to COMPOSE_PROFILE that should be feature-complete. Session Replays is one of the easiest feature to enable.

[DuncanConroy]

Regarding SELinux, I believe I ended up with chmod 777 on all mounted directories, as well as adding some special container label to those directories recursively. In the end it wasn't too complicated, but took me while to figure out the right way. I have no playbook for that, unfortunately. But podman and everything runs rootless, so it is definitely possible.

[BYK]

FYI I put this on hold on my end folks as I spent way too much time fixing these scripts and CI.

If anyone can figure out why we are getting that hash, I think that would unblock most of this.

[doc-sheet]

@DuncanConroy podman compose run calls add a weird hash at the beginning (see getsentry/self-hosted/actions/runs/14732417876/job/41349793317?pr=3673#step:4:890)

Any idea how to suppress that?

@BYK, could you run it again with debug on?

I want to participate in fixes and believe it would be faster then recreating environment by myself from the ground.

[BYK]

@doc-sheet you mean GitHub CI debug? Because I'm fairly sure the debug for the tooling will generate output that would break things even further.

Btw the setup is fairly bare bones: Ubuntu 22.04 with podman stuff installed. Clone the repo, run install.sh. That's all

[doc-sheet]

yeah, I meant github action restart with debug option

[doc-sheet]

Just made a quick look and found a fun thing. When I added --verbose flag error is gone.

I suppose it somehow related to how python (podman-compose) parses arguments. Maybe it adds detach flag at some point which is why podman prints container id.

I'll try to dig in it tomorrow.

I also stepped on another issue:

podman compose --no-ansi --env-file .env build --build-arg http_proxy= --build-arg https_proxy= --build-arg no_proxy= '--podman-rm-args='\''--force'\''' web

  podman-compose: error: unrecognized arguments: --podman-rm-args='--force'

[BYK]

Could that issue be related to https://github.com/containers/podman-compose/issues/707 ?

[BYK]

@doc-sheet here's the run with debug enabled: https://github.com/getsentry/self-hosted/actions/runs/14732867860/job/41748307991?pr=3673

Thanks so much for your help 🙏🏻

[aldy505]

FYI I put this on hold on my end folks as I spent way too much time fixing these scripts and CI.

If anyone can figure out why we are getting that hash, I think that would unblock most of this.

Me and Amin don't have write access, sooo..... :shrug:

[doc-sheet]

Sooo. The issue is not in args.

How podman compose run works:

1. Creates pod if not exists
2. Runs container in that pod

podman compose --no-ansi --env-file .env run --rm --no-deps -T relay credentials generate --stdout if actually a set of commands:

podman pod exists pod_sentry-self-hosted
podman pod create --name=pod_sentry-self-hosted --infra=false --share=
podman run --name=sentry-self-hosted_relay_tmp61537 --rm -i --pod=pod_sentry-self-hosted --label io.podman.compose.config-hash=4f24dfaedb436ac5a872aaf9afd02b3d5da8c0913de8c33c0ee702a9700764db --label io.podman.compose.project=sentry-self-hosted --label io.podman.compose.version=1.3.0 --label [email protected] --label com.docker.compose.project=sentry-self-hosted --label com.docker.compose.project.working_dir=/home/runner/work/self-hosted/self-hosted --label com.docker.compose.project.config_files=docker-compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=relay -v ./relay:/work/.relay:ro -v ./geoip:/geoip:ro --network=sentry-self-hosted_default:alias=relay getsentry/relay:nightly credentials generate --stdout

So hash is the id of created pod written to stdout after podman pod create

[doc-sheet]

I guess fastest way to handle this is to just disable pod creation

"Ensuring Relay credentials" test passed.

However to someone who would use podman-compose to run sentry pods may be neccessary (idk, never used it).

--- a/install/dc-detect-version.sh
+++ b/install/dc-detect-version.sh
@@ -66,7 +66,7 @@ fi
 proxy_args="--build-arg http_proxy=${http_proxy:-} --build-arg https_proxy=${https_proxy:-} --build-arg no_proxy=${no_proxy:-}"
 if [[ "$CONTAINER_ENGINE" == "podman" ]]; then
   proxy_args_dc="--podman-build-args http_proxy=${http_proxy:-},https_proxy=${https_proxy:-},no_proxy=${no_proxy:-}"
-  dcr="$dc run --rm"
+  dcr="$dc --in-pod=false run --rm"
 else
   proxy_args_dc=$proxy_args
   dcr="$dc run --pull=never --rm"

[doc-sheet]

Another workaround I guess is to run dummy command like

$dcr --no-deps -T relay --version >/dev/null

just to create pod, so next command will run in existing pod and would not print it's id

[BYK]

@doc-sheet ah, great find! The $dcr shortcut is only for one-off commands so disabling pod creation for that makes the most sense to me. I'll make the change and see if it passes the tests.

[BYK]

We should also figure out volume caching story with Podman. We have 2 options:

• Use podman volume export and podman volume import. This looks neat but requires us to do very podman-specific things in the CI and probably duplicate the cache
• Move the cached Docker volumes over to Podman as suggested here: https://www.reddit.com/r/podman/comments/15j7vhn/comment/juys1of/ -- This feels a bit hacky/finnicky but if it works, I'll take it

[BYK]

Okay, now we need to make the tests aware of Podman

[doc-sheet]

If podman-comose compatible enough podman-docker package could help. It creates system-wide alias docker -> podman

[aldy505]

Okay, now we need to make the tests aware of Podman

@BYK do you still remember what you mean by this?

[BYK]

@aldy505 the tests written in Python assume everything with Docker and Docker Compose. They now need to be aware of what technology we're using. See the fail here: https://github.com/getsentry/self-hosted/actions/runs/14893466765/job/41831036774?pr=3673#step:4:2280

[aldy505]

@DuncanConroy FYI, I'll revisit this after 25.6.0 release, since it has too much big changes already :laughing: