Initial support for podman + docker-compose

[eb4x]

Some minor cleanups and support for podman/docker-compose combination,

https://www.redhat.com/sysadmin/podman-docker-compose

[codecov[bot]]

Codecov Report

Merging #15986 (a605ad3) into main (6fe2a0c) will increase coverage by 0.02%. The diff coverage is n/a.

@@            Coverage Diff             @@
##             main   #15986      +/-   ##
==========================================
+ Coverage   66.81%   66.83%   +0.02%     
==========================================
  Files         934      934              
  Lines       77761    77761              
  Branches     2296     2296              
==========================================
+ Hits        51955    51971      +16     
+ Misses      21791    21779      -12     
+ Partials     4015     4011       -4     

Flag	Coverage Δ
unittests	66.83% <ø> (+0.02%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...es/vulnerability/vulnerability-config.component.ts	55.55% <0.00%> (+4.76%)	:arrow_up:
src/common/utils/passports.go	89.74% <0.00%> (+5.12%)	:arrow_up:
src/jobservice/runner/redis.go	67.82% <0.00%> (+5.21%)	:arrow_up:
...ortal/src/app/shared/pipes/harbor-datetime.pipe.ts	75.00% <0.00%> (+25.00%)	:arrow_up:

[ChristianCiach]

Can this be reviewed, please?

The only reason this currently does not work is because Harbor currently does a hardcoded check for the docker-version 17.06.0.

This version of Docker is ancient. Another way to make it work with podman is to just remove this version check.

[m4r1k]

Hey everybody,

Apologies for not working on this anymore. Last August, I moved to a different role where my involvement with Harbor has essentially ended. Hopefully, @bpereto or @wy65701436 will be able to make it happen :-)

Best of luck!

[leonidas-o]

Hello what's the status here? I don't think it is simply solved by refactoring common.sh, install.sh or the prepare file. I tried to refactor the files a bit to see how far could I come with podman. Tried with Harbor 2.5.0 and some aliases, so something like:

1. set echo "alias docker=podman" >> .bashrc
2. set echo "alias docker-compose=podman-compose" >> .bashrc
3. reload . ~/.bashrc
4. comment out hard coded docker and docker-compose version check in install.sh
5. add shopt -s expand_aliases + both above mentioned alias lines to install.sh and prepare

It failed almost at the end because it was using --log-driver=syslog in the docker compose file. But when you look into podman run docs, it says:

--log-driver=”driver”

Logging driver for the container. Currently available options are k8s-file, journald, none and passthrough, with json-file aliased to k8s-file for scripting compatibility. (Default journald)

So syslog is not one of podmans options. This would have to be configurable via e.g. the harbor.yml file.

[eb4x]

Hello what's the status here? I don't think it is simply solved by refactoring common.sh, install.sh or the prepare file. I tried to refactor the files a bit to see how far could I come with podman.

Yeah there's more to it than just this. These changes just allow you to get further than dying at docker < 17.06. As Christian pointed out, this version is "ancient", and quite possibly the whole check should just be removed as you did.

I think these individual commits are no-brainers, and have no idea what the holdup is in merging them. Maybe "Initial support" is a bit strong wording, and "allow for" would be better, as I didn't actually get it all the way working. I've since switched to quay.

If any of the commits seem illogical, I can trim them out and rebase. (Incase they prefer part1/part2 over major/minor)

[leonidas-o]

Hello what's the status here? I don't think it is simply solved by refactoring common.sh, install.sh or the prepare file. I tried to refactor the files a bit to see how far could I come with podman.

Yeah there's more to it than just this. These changes just allow you to get further than dying at docker < 17.06. As Christian pointed out, this version is "ancient", and quite possibly the whole check should just be removed as you did.

I think these individual commits are no-brainers, and have no idea what the holdup is in merging them. Maybe "Initial support" is a bit strong wording, and "allow for" would be better, as I didn't actually get it all the way working. I've since switched to quay.

If any of the commits seem illogical, I can trim them out and rebase. (Incase they prefer part1/part2 over major/minor)

maybe it isn't merged because it's not fully working that way, as said the log-driver=syslog issue has to be solved as well before it can be used with podman. Quickly went over the generated docker-compose.yml file, I think that's all. The question is, who can make the hardcoded log-driver=syslog setting somehow configurable in harbor?

[ChristianCiach]

I think the issue with the syslog logging driver is slightly related to the issue https://github.com/goharbor/harbor/issues/16594 . If Harbor would just log to stdout/stderr like a proper container should, this issue would also be solved implicitly.

"Logging is delegated to a separate container when docker-compose achieves the exact same thing (I tore this out with no concequence)"

[github-actions[bot]]

This PR is being marked stale due to a period of inactivty. If this PR is still relevant, please comment or remove the stale label. Otherwise, this PR will close in 30 days.

[leonidas-o]

maybe it isn't merged because it's not fully working that way, as said the log-driver=syslog issue has to be solved as well before it can be used with podman. Quickly went over the generated docker-compose.yml file, I think that's all. The question is, who can make the hardcoded log-driver=syslog setting somehow configurable in harbor?

Still very relevant, at least for me. I still hope someone is able to make the log-driver configurable.

[Vad1mo]

adding flexibility here is a good thing, let's try to get that in. @leonidas-o did you try the PRs, does it work for you?

[leonidas-o]

Hello what's the status here? I don't think it is simply solved by refactoring common.sh, install.sh or the prepare file. I tried to refactor the files a bit to see how far could I come with podman. Tried with Harbor 2.5.0 and some aliases, so something like:

1. set echo "alias docker=podman" >> .bashrc
2. set echo "alias docker-compose=podman-compose" >> .bashrc
3. reload . ~/.bashrc
4. comment out hard coded docker and docker-compose version check in install.sh
5. add shopt -s expand_aliases + both above mentioned alias lines to install.sh and prepare

It failed almost at the end because it was using --log-driver=syslog in the docker compose file. But when you look into podman run docs, it says:

--log-driver=”driver”

Logging driver for the container. Currently available options are k8s-file, journald, none and passthrough, with json-file aliased to k8s-file for scripting compatibility. (Default journald)

So syslog is not one of podmans options. This would have to be configurable via e.g. the harbor.yml file.

@Vad1mo unfortunately no, back then, I have done it like described here, so really just removed the hard coded docker checks and realised that the hard coded log-driver is blocking me. I agree, adding flexibility is definitely a good thing. Do you have a setup to test the PRs? Otherwise I will have to spin up a VM for that.

[github-actions[bot]]

This PR is being marked stale due to a period of inactivty. If this PR is still relevant, please comment or remove the stale label. Otherwise, this PR will close in 30 days.

[wy65701436]

hi, thanks for the contribution. Since currently podman+docker-compose is not in the support scope, and we need time to evaluate it, like what's the impact and how do we test it.

[leonidas-o]

Hello what's the status here? I don't think it is simply solved by refactoring common.sh, install.sh or the prepare file. I tried to refactor the files a bit to see how far could I come with podman. Tried with Harbor 2.5.0 and some aliases, so something like:

1. set echo "alias docker=podman" >> .bashrc
2. set echo "alias docker-compose=podman-compose" >> .bashrc
3. reload . ~/.bashrc
4. comment out hard coded docker and docker-compose version check in install.sh
5. add shopt -s expand_aliases + both above mentioned alias lines to install.sh and prepare

It failed almost at the end because it was using --log-driver=syslog in the docker compose file. But when you look into podman run docs, it says:

--log-driver=”driver”

Logging driver for the container. Currently available options are k8s-file, journald, none and passthrough, with json-file aliased to k8s-file for scripting compatibility. (Default journald)

So syslog is not one of podmans options. This would have to be configurable via e.g. the harbor.yml file.

I wonder if it would be easier if this would be treated as a feature request for, let's say:

1. making the --log-driver=syslog somehow configurable.
2. clean up/ remove all hard coded checks for docker

Obviously you guys don't have the resource to support podman and I think this isn't necessary. Maybe laying the focus on smaller items like, optimisation and clean up. With this two changes, you would open up the door for the community to come up with a podman solution. Then it can still be: "Podman is not officially supported" but everyone who is interested, like me, can try to make it run, document the approach and maybe upload it somewhere in the harbor docs, clearly saying, no not officially supported but using this approach should make it run. And who knows, maybe one day, one find some time to look into it and make it officially supported, if not, simply keep it unofficial. I think this is definitely better than completely locking out podman by using hard coded log-driver and docker checks. What do you guys think, does it make sense what I'm saying?

[wy65701436]

@leonidas-o It makes sense to expand the support list for the harbor. To me, it's more like a feature request.

If we agree that this is a feature request or a new feature, I would like to follow this procedure.

This PR is not enough to introduce podman support as we don't even have any tests, this at least includes podman setup, pull/push (single imge, imge index), retag, etc.

cc @qnetter

[leonidas-o]

@leonidas-o It makes sense to expand the support list for the harbor. To me, it's more like a feature request.

If we agree that this is a feature request or a new feature, I would like to follow this procedure.

This PR is not enough to introduce podman support as we don't even have any tests, this at least includes podman setup, pull/push (single imge, imge index), retag, etc.

cc @qnetter

@wy65701436 created a PR, feel free to update it if something isn't right: https://github.com/goharbor/community/pull/208