python3-podman: can't utilize rootless podman

[TomasTomecek]

Is this a BUG REPORT or FEATURE REQUEST?:

[//]: # kind bug

Description

I really like the python interface you have built. Unfortunately it's not possible to be used with rootless podman due to the varlink limitation. Is there a way to solve it? (have two sockets maybe?)

srw-------. 1 root root 0 Nov  1 14:45 io.podman
srw-------. 1 root root 0 Nov  1 14:45 podman/io.podman

$ id
uid=1000(tt) gid=1000(tt)
context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

$ ipython3
Python 3.7.1 (default, Oct 23 2018, 18:19:07)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import podman

In [2]: with podman.Client() as client:
   ...:   list(map(print, client.images.list()))
   ...:
---------------------------------------------------------------------------
PermissionError                           Traceback (most recent call last)
...

/usr/lib/python3.7/site-packages/varlink/client.py in open_unix()
    459                 s = socket.socket(socket.AF_UNIX)
    460                 s.setblocking(True)
--> 461                 s.connect(address)
    462                 return s
    463

PermissionError: [Errno 13] Permission denied

$ podman images
REPOSITORY                                   TAG                       IMAGE ID       CREATED             SIZE
localhost/this-is-my-image                   latest                    d49960b9dbcd   29 minutes ago      83.2MB
<none>                                       <none>                    d3b925049e8d   29 minutes ago      83.2MB
localhost/this-is-my-image-20181701-161756   latest                    2a566e4de41f   29 minutes ago      83.2MB
docker.io/library/python                     3-alpine                  408808fb1a9e   7 days ago          82MB
registry.fedoraproject.org/fedora            29                        a5cc8ccd230a   2 weeks ago         306MB

[mheon]

Can we set up a socket for rootless in the systemd user instance? That would probably do everything we need for rootless.

Would still need to update the bindings to optionally connect to the separate rootless socket.

[TomasTomecek]

Can we set up a socket for rootless in the systemd user instance?

I think that would need to be baked directly in podman, since afaik you can't control user instance from the root environment.

Or maybe just provide docs and mention this in the error message when raising permission denied.

[rhatdan]

I actually think we should raise an issue for this. libpodman could launch a podman version locally when running as non root. and then communicate with the podman listening at a socket in the users homedir.

[owtaylor]

For systems using systemd user sessions (e.g. Fedora for quite some time), the system setup with socket activation could be directly duplicated.

[JayjeetAtGithub]

I am facing the same issue. Can someone please help?

[rhatdan]

We have support for in podman varlink for autostart now, no reason to listen on a socket.

podman remote does this now.

@jwhonce @baude Could you guys get python-podman to do this by default rather then looking for the socket?

[JayjeetAtGithub]

@rhatdan Thanks a lot !

[JayjeetAtGithub]

@rhatdan Can you tell me how to do it now with python podman now? I can also send the fix if you would tell me a bit? Thanks.

[bregman-arie]

I get the same error. Is there a workaround?

[jwhonce]

@bregman-arie I'm working on a bug in the API first. Once that works, I can document this.

[JayjeetAtGithub]

@jwhonce I need to use python-podman for rootless podman very soon. Can you please tell me any workaround to bypass this bug ? Thanks.

[JacobCallahan]

@jwhonce can you link to where you documented this? Thanks

[jonathanunderwood]

This still appears to be broken.

[jonathanunderwood]

We have support for in podman varlink for autostart now, no reason to listen on a socket.

podman remote does this now.

@jwhonce @baude Could you guys get python-podman to do this by default rather then looking for the socket?

@rhatdan can you explain this in a bit more detail, preferably with some doc/code links - I'd be interested in looking at implementing this.

[jonathanunderwood]

For anyone coming to this, here's a work around on Fedora 31:

First activate the user podman socket systemd service:

$ systemctl --user enable --now io.podman.socket

Check that it is running:

$ varlink info unix:/run/user/$(id -u)/podman/io.podman
Vendor: Atomic
Product: podman
Version: 1.6.2
URL: https://github.com/containers/libpod
Interfaces:
  org.varlink.service
  io.podman

At which point, you should be able to create a client with the Python bindings:

import os
import podman
c = podman.Client(uri=f"unix:/run/user/{os.getuid()}/podman/io.podman")

[rhatdan]

Is this a work around or a permanent solution?

[jonathanunderwood]

Is this a work around or a permanent solution?

I think the Python bindings should be smarter than this. The command line podman utility doesn't require that the io.podman user socket service is started for it to function, so I don't really understand why the Python bindings require it.

And, the Python bindings should be smarter about locating the relevant socket in any case.

So, I do think this is a workaround; there's still work to be done on the Python bindings. The Python bindings are in a fairly terrible state at the moment. Are these considered supported bindings in the podman/buildah world?

[rhatdan]

There is a big effort going on to implement the docker.sock api, right now, this might be a good time to also rework the python bindings to be able to launch a podman service on demand to satisfy this need. Similar to what is being done with podman varlink.

[JacobCallahan]

@rhatdan is there any proposed timeline for when we can expect this to be implemented?

[rhatdan]

It is being worked on now, and we have a goal of demonstrating it at Devconf.CZ at the end of the month.

[jonathanunderwood]

@rhatdan where's this work happening? I don't see it in the public podman repos? Am I looking in the wrong place?

[rhatdan]

https://github.com/containers/libpod/tree/apiv2

We hope to merge this into master later this week on next week.

[dAnjou]

Hey @jonathanunderwood, thanks for posting this workaround. I tried it myself but I get an error:

$ sudo systemctl --user enable --now io.podman.socket
Failed to connect to bus: No such file or directory

The socket is there:

$ sudo systemctl status io.podman.socket
[sudo] password for max: 
● io.podman.socket - Podman Remote API Socket
   Loaded: loaded (/usr/lib/systemd/system/io.podman.socket; disabled; vendor preset: disabled)
   Active: active (listening) since Sun 2020-01-26 15:29:37 CET; 5min ago
     Docs: man:podman-varlink(1)
   Listen: /run/podman/io.podman (Stream)
   CGroup: /system.slice/io.podman.socket

$ sudo varlink info "unix:/run/podman/io.podman"
Vendor: Atomic
Product: podman
Version: 1.7.0
URL: https://github.com/containers/libpod
Interfaces:
  org.varlink.service
  io.podman

I'm on Fedora 31 as well.

[jonathanunderwood]

@dAnjou Try without the sudo

[dAnjou]

@jonathanunderwood Nice, thanks!

[gatoniel]

Hi, I want to write a JupyterHub spawner for podman. I would like to use this python-podman API to run the commands to pull, start and stop containers. I started the io.podman.socket

systemctl --user enable io.podman.socket
systemctl start --user io.podman.socket
systemctl status --user io.podman.socket

● io.podman.socket - Podman Remote API Socket Loaded: loaded (/usr/lib/systemd/user/io.podman.socket; enabled; vendor preset: enabled) Active: active (listening) since Sat 2020-03-28 11:40:37 CET; 10s ago Docs: man:podman-varlink(1) Listen: /run/user/1000/podman/io.podman (Stream) CGroup: /user.slice/user-1000.slice/[email protected]/io.podman.socket

Mär 28 11:40:37 picard.Physik.Uni-Marburg.DE systemd[2934]: Listening on Podman Remote API Socket.

and ran these commands in a python console:

import podman
with podman.Client() as client:
    list(map(print, client.images.list()))

This is the error message:

Traceback (most recent call last): File "", line 1, in File "/opt/jupyterhub/lib64/python3.6/site-packages/podman/client.py", line 179, in init if not System(self._client).ping(): File "/opt/jupyterhub/lib64/python3.6/site-packages/podman/libs/system.py", line 38, in ping with self._client() as podman: File "/opt/jupyterhub/lib64/python3.6/site-packages/podman/client.py", line 99, in enter self._iface = self._client.open(self._context.interface) File "/opt/jupyterhub/lib64/python3.6/site-packages/varlink/client.py", line 598, in open connection = self.open_connection() File "/opt/jupyterhub/lib64/python3.6/site-packages/varlink/client.py", line 613, in open_connection return self._socket_fn() File "/opt/jupyterhub/lib64/python3.6/site-packages/varlink/client.py", line 512, in open_unix s.connect(address) PermissionError: [Errno 13] Permission denied

I am in CentOS 8. Is there any way to use the python-podman API without using this remote API Socket? I actually dont need to communicate remotely, since I want to start the containers on the same Server. My fallback solution would be, to call the podman commands via subprocess.Popen. But this seems rather complicated and unpleasant, when there is an python API around.

[rhatdan]

Note we are moving away from the varlink interface to a RestAPI version, We call it APIV2. This will support the docker-py as well as extensions for advanced features of podman. Python work is happening here: https://github.com/containers/podman-py

In the long run, you might be better off working towards this API.

[ivotron]

In the long run, you might be better off working towards this API.

thanks for the heads up @rhatdan! Should this project be archived then? Or a note added to the README?

[rhatdan]

Eventually once we release podman 2.0.