podman icon indicating copy to clipboard operation
podman copied to clipboard

Implement publishing API UNIX socket on Windows platforms

Open arixmkii opened this issue 1 year ago • 12 comments

Fixes #23408

gvproxy and win-sshproxy have capabilities to serve this type of enpoint. This change only adds one additional API enpoint publishing by appending proxy command lines.

Originally developed within https://github.com/containers/podman/issues/13006 but could be generalized to any VM using gvproxy or win-sshproxy. This has been verified to work for Hyper-V and WSL machines.

Example how it looks for WSL machine. commands

  • podman machine list (to show what kind of machine it is)
  • podman machine inspect (to demo published endpoint in info)
  • where curl (to verify it will use Windows one)
  • curl to call the enpoint
C:\qcw-utils\shells>podman machine list
NAME                     VM TYPE     CREATED         LAST UP            CPUS        MEMORY      DISK SIZE
podman-machine-default*  wsl         43 seconds ago  Currently running  2           2GiB        100GiB

C:\qcw-utils\shells>podman machine inspect
[
     {
          "ConfigDir": {
               "Path": "C:\\Users\\User\\.config\\containers\\podman\\machine\\wsl"
          },
          "ConnectionInfo": {
               "PodmanSocket": {
                    "Path": "C:\\Users\\User\\AppData\\Local\\Temp\\podman\\podman-machine-default-api.sock"
               },
               "PodmanPipe": {
                    "Path": "\\\\.\\pipe\\podman-machine-default"
               }
          },
          "Created": "2024-07-26T12:57:29.14137+03:00",
          "LastUp": "0001-01-01T00:00:00Z",
          "Name": "podman-machine-default",
          "Resources": {
               "CPUs": 2,
               "DiskSize": 100,
               "Memory": 2048,
               "USBs": []
          },
          "SSHConfig": {
               "IdentityPath": "C:\\Users\\User\\.local\\share\\containers\\podman\\machine\\machine",
               "Port": 50239,
               "RemoteUsername": "user"
          },
          "State": "running",
          "UserModeNetworking": false,
          "Rootful": false,
          "Rosetta": false
     }
]

C:\qcw-utils\shells>where curl
C:\Windows\System32\curl.exe

C:\qcw-utils\shells>curl --unix-socket C:\\Users\\User\\AppData\\Local\\Temp\\podman\\podman-machine-default-api.sock h
ttp://d/v5.0.0/libpod/info
{"host":{"arch":"amd64","buildahVersion":"1.36.0","cgroupManager":"cgroupfs","cgroupVersion":"v1","cgroupControllers":[],"conmon":{"package":"conmon-2.1.10-1.fc40.x86_64","path":"/usr/bin/conmon","version":"conmon version 2.1.10, commit: "},"cpus":4,"cpuUtilization":{"userPercent":0.86,"systemPercent":1.54,"idlePercent":97.6},"databaseBackend":"sqlite","distribution":{"distribution":"fedora","variant":"container","version":"40"},"eventLogger":"journald","freeLocks":2048,"hostname":"glider-mk2","idMappings":{"gidmap":[{"container_id":0,"host_id":1000,"size":1},{"container_id":1,"host_id":524288,"size":65536}],"uidmap":[{"container_id":0,"host_id":1000,"size":1},{"container_id":1,"host_id":524288,"size":65536}]},"kernel":"5.15.153.1-microsoft-standard-WSL2","logDriver":"journald","memFree":16180596736,"memTotal":16773939200,"networkBackend":"netavark","networkBackendInfo":{"backend":"netavark","version":"netavark 1.11.0","package":"netavark-1.11.0-1.fc40.x86_64","path":"/usr/libexec/podman/netavark","dns":{"version":"aardvark-dns 1.11.0","package":"aardvark-dns-1.11.0-1.fc40.x86_64","path":"/usr/libexec/podman/aardvark-dns"}},"ociRuntime":{"name":"crun","package":"crun-1.15-1.fc40.x86_64","path":"/usr/bin/crun","version":"crun version 1.15\ncommit: e6eacaf4034e84185fd8780ac9262bbf57082278\nrundir: /run/user/1000/crun\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL"},"os":"linux","remoteSocket":{"path":"/run/user/1000/podman/podman.sock","exists":true},"rootlessNetworkCmd":"pasta","serviceIsRemote":false,"security":{"apparmorEnabled":false,"capabilities":"CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT","rootless":true,"seccompEnabled":true,"seccompProfilePath":"/usr/share/containers/seccomp.json","selinuxEnabled":false},"slirp4netns":{"executable":"","package":"","version":""},"pasta":{"executable":"/usr/bin/pasta","package":"passt-0^20240624.g1ee2eca-1.fc40.x86_64","version":"pasta 0^20240624.g1ee2eca-1.fc40.x86_64\nCopyright Red Hat\nGNU General Public License, version 2 or later\n  \u003chttps://www.gnu.org/licenses/old-licenses/gpl-2.0.html\u003e\nThis is free software: you are free to change and redistribute it.\nThere is NO WARRANTY, to the extent permitted by law.\n"},"swapFree":4294967296,"swapTotal":4294967296,"uptime":"0h 1m 29.00s","variant":"","linkmode":"dynamic"},"store":{"configFile":"/home/user/.config/containers/storage.conf","containerStore":{"number":0,"paused":0,"running":0,"stopped":0},"graphDriverName":"overlay","graphOptions":{},"graphRoot":"/home/user/.local/share/containers/storage","graphRootAllocated":1081101176832,"graphRootUsed":790700032,"graphStatus":{"Backing Filesystem":"extfs","Native Overlay Diff":"true","Supports d_type":"true","Supports shifting":"false","Supports volatile":"true","Using metacopy":"false"},"imageCopyTmpDir":"/var/tmp","imageStore":{"number":0},"runRoot":"/run/user/1000/containers","volumePath":"/home/user/.local/share/containers/storage/volumes","transientStore":false},"registries":{"search":["docker.io"]},"plugins":{"volume":["local"],"network":["bridge","macvlan","ipvlan"],"log":["k8s-file","none","passthrough","journald"],"authorization":null},"version":{"APIVersion":"5.1.2","Version":"5.1.2","GoVersion":"go1.22.5","GitCommit":"","BuiltTime":"Wed Jul 10 03:00:00 2024","Built":1720569600,"OsArch":"linux/amd64","Os":"linux"}}

Does this PR introduce a user-facing change?

Additionally provide API access via UNIX sockets to Podman running in Podman Machine on Windows

arixmkii avatar Jul 26 '24 10:07 arixmkii

I can't find any tests for sockets or pipes in e2e tests of the machine, so, I can't extend these tests for additional verification. There is limited value trying to test gvproxy or win-sshproxy command builders as it would be close to testing getters/setters.

arixmkii avatar Jul 26 '24 10:07 arixmkii

@l0rd any opinions on this?

baude avatar Aug 07 '24 13:08 baude

@l0rd any opinions on this?

I am fine with the original issue. I think that's useful. Now a few considerations:

  1. AF_UNIX requires build 17061 or higher. Podman requires build 19041 or higher. We are fine on that side.
  2. We need automated tests to make sure that we don't break this on both hyper-v / wsl
  3. That's an optional feature that can break the machine start. Therefore we should have a config option to disable it

l0rd avatar Aug 07 '24 16:08 l0rd

Therefore we should have a config option to disable it

I will work on this. If you someone can share a good example of a feature toggle I can use as inspiration this would speed up things definitely.

We need automated tests to make sure that we don't break this on both hyper-v / wsl

I will try to sketch some tests and then they could be improved during review process.

Having this accepted as an idea for a feature is good enough for me now to continue improving this changeset.

arixmkii avatar Aug 08 '24 19:08 arixmkii

Added tests for the API. Client for Unix sockets and Named Pipes. Curl for Unix sockets. To not skip tests of Client on Windows it is needed to have a fix in go-connections utilities first https://github.com/docker/go-connections/issues/116

Will work on the config changes for the feature toggle.

arixmkii avatar Aug 31 '24 13:08 arixmkii

@l0rd @baude PTAL

rhatdan avatar Sep 03 '24 19:09 rhatdan

@arixmkii thank you for updating the PR. I didn't had the chance to review it yet but I plan to do it tomorrow.

l0rd avatar Sep 04 '24 16:09 l0rd

code changes LGTM ... @l0rd are you good with merging this?

baude avatar Sep 17 '24 20:09 baude

There is no way to disable the exposure of the Unix API socket yet but in the current state the PR already adds some value so +1 for me to merge if @arixmkii is ok.

l0rd avatar Sep 17 '24 22:09 l0rd

@baude @l0rd would be nice to merge this if possible. I don't see reasonable code improvements for this feature, but feature toggle will be an additional feature itself.

arixmkii avatar Sep 20 '24 11:09 arixmkii

/lgtm

l0rd avatar Sep 20 '24 13:09 l0rd

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: arixmkii, l0rd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci[bot] avatar Oct 11 '24 10:10 openshift-ci[bot]