podman-py
podman-py copied to clipboard
containers
Unix-Sockets connection not working
Using podman-py we cannot connect to a local unix-socket. Irrespective of how we communicate the protocol, it will always end up as http.
The following does not work, it will end up making an http-request to http://%2frun%2fpodman.sock/v4.0.0/libpod/version instead of the unix-socket.
def podman_py():
from podman import PodmanClient
uri = "unix:///run/podman.sock"
with PodmanClient(base_url=uri) as client:
version = client.version() # fails
print("Release: ", version["Version"])
print("Compatible API: ", version["ApiVersion"])
print("Podman API: ", version["Components"][0]["Details"]["APIVersion"], "\n")
For comparison, the following, manual connection to the socket works:
def podman_requests_unixsocket():
import requests_unixsocket
url = "http+unix://%2frun%2fpodman.sock/v4.0.0/libpod/version"
with requests_unixsocket.Session() as session:
response = session.get(url)
response.raise_for_status()
version = json.loads(response.text)
print("Release: ", version["Version"])
print("Compatible API: ", version["ApiVersion"])
print("Podman API: ", version["Components"][0]["Details"]["APIVersion"], "\n")
My suspicion is that we are doing something wrong, or that something is lost in translation when hard-coding the protocol to http and then mapping to UDSAdapter down the line.
We made some manual changes to podman-py that seem to do the trick (although they look wrong to me):
# Map supported schemes to Pool Classes
_pool_classes_by_scheme = {
"http": UDSConnectionPool,
"http+unix": UDSConnectionPool,
"http+ssh": UDSConnectionPool,
}
# Map supported schemes to Pool Key index generator
_key_fn_by_scheme = {
"http": functools.partial(_key_normalizer, _PoolKey),
"http+unix": functools.partial(_key_normalizer, _PoolKey),
"http+ssh": functools.partial(_key_normalizer, _PoolKey),
}
if self.base_url.scheme == "http+unix":
self.mount("http+unix://", UDSAdapter(self.base_url.geturl(), **adapter_kwargs))
uri = urllib.parse.ParseResult(
"http+unix",
self.base_url.netloc,
urllib.parse.urljoin(path_prefix, path),
self.base_url.params,
self.base_url.query,
self.base_url.fragment,
)
We are using version 4.0.0
Hey,
I think you are passing wrong url.
In podman socket socket is located under /run/podman/podman.sock by default.
You are probably used to docker where socket was located under /run/docker.sock
Python 3.6.8 (default, Jan 14 2022, 11:04:20)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def podman_py(uri):
... from podman import PodmanClient
... with PodmanClient(base_url=uri) as client:
... version = client.version() # fails
... print("Release: ", version["Version"])
... print("Compatible API: ", version["ApiVersion"])
... print("Podman API: ", version["Components"][0]["Details"]["APIVersion"], "\n")
...
>>> for uri in ["unix:///run/podman.sock", "unix:///run/podman/podman.sock"]:
... try:
... print(podman_py(uri=uri))
... except:
... print(f'uri {uri} failed')
...
uri unix:///run/podman.sock failed
Release: 4.0.2
Compatible API: 1.40
Podman API: 4.0.2
@msisj the socket is mounted there by us. And as I demonstrate in the lower snippet, acessing the socket does work, just not with podman-py.
I am hitting some similar issues. The module was working with the following uri - f"unix:///run/user/{os.getuid()}/podman/podman.sock until I run the following command:
sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 $USER && podman system migrate.
The usermod command was used to solve an image build failure.
If you run podman natively does it work?
Nop. It stopped working as well. Is there a way to restore the settings before the usermod command?
@rhatdan I was able to revert the change such that podman cli returned working. The api is not. Hitting the error requests.exceptions.ConnectionError: ('Connection aborted.', APIError("Unable to make connection to UDS '/run/user/114164/podman/podman.sock'"))
@awildturtok can you please paste the output you get when this fails? also, please run systemctl start podman to ensure the socket it listening and that the system service is using this socket.
This most likely is not on our end, as I just fresh installed podman-py 4.2 and ran the test code, and it succeeded. I would assume the custom mounting of the socket is not right.
Just a question here... I was trying to embed this is a container to deal with image registry checking part... But it's really necessary to have the socket available to do this part? I am not following right the stuff being said about Portman. Is it was not said that podman was daemonless? It the socket is not for the API Daemon, it's for what?
podman runs well without any daemon or service. If you want to talk to the remote API via podman-py, docker-py, docker-compose ... Then we need to have a service listening for incoming connections. This is also provided via a Podman systemd socket activated service. Which will handle connections and shuts down within 10 seconds of the last connection. If you want to call this a daemon, then fine.
My main intent it's to create a mechanism to getting a registry image so I can validate if the image exists with the respective tag...
That is a embedded application deployed in a k8s without access to a docker daemon, so I thought about using podman, but I had problems about the socket.
I receive for instance: gcr.io/namespace/image:latest
I want to check if that exists or not.
So you want to pull the image into the container, or are you looking to see if k8s has pulled the image?
Neither of that, I only want to validate if the given remote repository image exists, not necessarily I need to pull and download it...
I could just use http... But checking directly via http get can be cumbersome if there is multiple possible docker registries (potential places to check: on-premisse registries, gcr.io etc)
So the whole OCI container engineer is not necessary for this application.
Cordialmente, Manoel Vilela.
Em sex., 28 de abr. de 2023 11:23, Daniel J Walsh @.***> escreveu:
So you want to pull the image into the container, or are you looking to see if k8s has pulled the image?
— Reply to this email directly, view it on GitHub https://github.com/containers/podman-py/issues/188#issuecomment-1527648403, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2J57RVERM3TDY5VIHQU2TXDPHEJANCNFSM54HFK54A . You are receiving this because you commented.Message ID: @.***>
Thank you for sharing skopeo, that's the type of utility I was looking for. Unfortunately it's written in Go and make integration a little bit worse in the kind of application will be embedded with it.
It would be nice if skopeo provided a python client library wrapper for this.
Sorry for bothering podman-py with this.
cc @luizmanke
Cordialmente, Manoel Vilela.
Em sáb., 29 de abr. de 2023 07:46, Daniel J Walsh @.***> escreveu:
Why not use skopeo inspect
— Reply to this email directly, view it on GitHub https://github.com/containers/podman-py/issues/188#issuecomment-1528755936, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB2J57TRUJRRG3QRQIZU5XTXDTWQNANCNFSM54HFK54A . You are receiving this because you commented.Message ID: @.***>
I'm doing something very similar @ryukinix, my extra pain point is that I'm doing it from within a docker container. Anyhow you need to start the podman service:
# this will run until killed
podman system service --time=0 unix:///tmp/podman.sock &
then within your python code
import podman
uri="unix:///tmp/podman.sock"
client = podman.PodmanClient(base_url=uri)
docker_image = "registry.hub.docker.com/library/busybox:latest"
manifest = client.manifests.get(docker_image)
if manifest is None:
print("image does not exist in registry")
else:
print("ok!")
Note that I'm having issues with the subuids/subguids, but to check this image, you should be ok without anything set.
