pcp icon indicating copy to clipboard operation
pcp copied to clipboard

Published 20 hours ago • verified publisher icon performancecopilot

podman pmda and pcp containers in general only find root-managed containers

Open Fodoj opened this issue 4 years ago • 15 comments

As a normal user:

5f87707b1cef  docker.io/library/redis:alpine       docker-entrypoint...  2 hours ago  Up 2 hours ago  0.0.0.0:8000->8065/tcp, 0.0.0.0:8001->8066/tcp, 0.0.0.0:3001->3001/tcp  redis
f0ad97f94c2d  docker.io/library/postgres:9.6.9     docker-entrypoint...  2 hours ago  Up 2 hours ago  0.0.0.0:8000->8065/tcp, 0.0.0.0:8001->8066/tcp, 0.0.0.0:3001->3001/tcp  postgres

As root (sudo podman ps):

2297307970dc  localhost/langtool:latest  /bin/sh -c /start...  6 days ago  Up 6 days ago  0.0.0.0:8081->8081/tcp  exciting_swirles

pminfo shows only the one running as root:

pminfo --fetch podman.container.name

result:

podman.container.name
    inst [0 or "2297307970dcaed7b0c1e95c7cd6ae5532bd9d902f7dfcb8bf17c8c138b35844"] value "exciting_swirles"

and for containers:

pminfo --fetch containers

result:

containers.engine
    inst [0 or "2297307970dcaed7b0c1e95c7cd6ae5532bd9d902f7dfcb8bf17c8c138b35844"] value "podman"

containers.name
    inst [0 or "2297307970dcaed7b0c1e95c7cd6ae5532bd9d902f7dfcb8bf17c8c138b35844"] value "exciting_swirles"

containers.pid
    inst [0 or "2297307970dcaed7b0c1e95c7cd6ae5532bd9d902f7dfcb8bf17c8c138b35844"] value 138562

containers.cgroup
    inst [0 or "2297307970dcaed7b0c1e95c7cd6ae5532bd9d902f7dfcb8bf17c8c138b35844"] value "/libpod-2297307970dcaed7b0c1e95c7cd6ae5532bd9d902f7dfcb8bf17c8c138b35844.scope"

containers.state.running
    inst [0 or "2297307970dcaed7b0c1e95c7cd6ae5532bd9d902f7dfcb8bf17c8c138b35844"] value 1

containers.state.paused
    inst [0 or "2297307970dcaed7b0c1e95c7cd6ae5532bd9d902f7dfcb8bf17c8c138b35844"] value 0

containers.state.restarting
    inst [0 or "2297307970dcaed7b0c1e95c7cd6ae5532bd9d902f7dfcb8bf17c8c138b35844"] value 0

I did not do any specific pcp configuration, just installed default packages on my Fedora 32 Workstation.

Fodoj avatar May 16 '20 20:05 Fodoj

@Fodoj there's no way to query podman for other users on a system (AFAIK). The PMDA runs as root, so as you see it can access only the root users containers. We could add an option to the PMDA to run as an unprivileged user - but you will still only be able to see that user's containers.

natoscott avatar May 17 '20 23:05 natoscott

Actually, we could possibly do better than that and make pmdapodman behave like pmdaproc, such that it performs podman queries as the user running the client monitoring tool connected to pmcd.

natoscott avatar May 17 '20 23:05 natoscott

+1 to this. We are encouraging users to run rootless containers these days, so this pmda is less useful if it only reports on containers created by the root user

apitanga avatar Feb 07 '21 23:02 apitanga

Actually, we could possibly do better than that and make pmdapodman behave like pmdaproc, such that it performs podman queries as the user running the client monitoring tool connected to pmcd.

What about container monitoring with the pmproxy REST API/Grafana? Ideally I would like to see all containers of all users. As a regular linux user I also see all processes of other users (unless hidepid is set).

andreasgerstmayr avatar Feb 11 '21 17:02 andreasgerstmayr

Related (thanks Marko): https://podman.io/blogs/2020/08/01/deprecate-and-remove-varlink-notice.html

natoscott avatar Apr 20 '21 22:04 natoscott

Hello @natoscott , I am using pcp-pmda-podman , version 5.3.7 together with pcp version 5.3.7 and I am unable to find non-root manged containers.

Isnt this issue resolved by: https://github.com/performancecopilot/pcp/pull/1284 ?

sercanyaldiz avatar May 02 '23 21:05 sercanyaldiz

@natoscott yep - check you have /run/user/*/podman/podman.sock file(s) for the non-root user(s) - pmdapodman communicates over these sockets to the podman REST API. Also check there's no selinux or other security module preventing access.

natoscott avatar May 03 '23 06:05 natoscott

@natoscott the sock file is present: srw-rw----. 0 Apr 20 09:28 /run/user/1000/podman/podman.sock

Also the podman.socket is enabled and active. ● podman.socket - Podman API Socket Loaded: loaded (/usr/lib/systemd/system/podman.socket; enabled; vendor preset: disabled) Active: active (listening) since Wed 2023-04-19 20:24:17 CEST; 1 week 6 days ago Until: Wed 2023-04-19 20:24:17 CEST; 1 week 6 days ago Triggers: ● podman.service Docs: man:podman-system-service(1) Listen: /run/podman/podman.sock (Stream) CGroup: /system.slice/podman.socket Apr 19 20:24:17 home systemd[1]: Listening on Podman API Socket.

$ pminfo --fetch containers containers.engine No value(s) available! containers.name No value(s) available! containers.pid No value(s) available! containers.cgroup No value(s) available! containers.state.running No value(s) available! containers.state.paused No value(s) available! containers.state.restarting No value(s) available!

I have the following running containers

podman ps -q cf24d60248a5 7862962360ac 7b97311b3892 6b347bfb4e03 3118aeb8964b

sercanyaldiz avatar May 03 '23 11:05 sercanyaldiz

OK, so let's check the APIs the PCP code is using. Can you see what results you get from...

$ curl -s --unix-socket /run/user/1000/podman/podman.sock http://d/v1.0.0/libpod/info | jq

And if that's working, try these endpoints too (substitute in the above command from v1 onwards)

/v3.0.0/libpod/pods/json /v3.0.0/libpod/containers/json /v3.0.0/libpod/containers/stats?stream=false

The above is for the podman PMDA (as referenced in the PR you linked earlier). To activate those metrics, its:

# touch /var/lib/pcp/pmdas/podman/.NeedInstall
# systemctl restart pmcd

For the 'root' PMDA which services container.* we look in a containers.json file below /var/lib/containers/storage/overlay-containers - I assume that must exist because you're getting the root metrics. But, hmm, I'm not sure this code will find user containers unless they're in that location too. Can you see any file in the users HOME (maybe below .containers or something) that looks similar?

cheers.

natoscott avatar May 04 '23 06:05 natoscott

@natoscott thank you sir! Really appreciate for helping me.

I know for sure that I installed podman pmdas by executing ./Install after dnf install pmdas-podman confused why it didnt work right away.

But I followed your instructions below and I now I do see the metrics coming in.

# touch /var/lib/pcp/pmdas/podman/.NeedInstall
# systemctl restart pmcd

● pmcd.service - Performance Metrics Collector Daemon
     Loaded: loaded (/usr/lib/systemd/system/pmcd.service; disabled; vendor preset: disabled)
     Active: active (running) since Thu 2023-05-04 20:51:46 CEST; 17min ago
       Docs: man:pmcd(1)
   Main PID: 670596 (pmcd)
      Tasks: 8 (limit: 406740)
     Memory: 25.6M
        CPU: 906ms
     CGroup: /system.slice/pmcd.service
             ├─670596 /usr/libexec/pcp/bin/pmcd -A
             ├─670600 /var/lib/pcp/pmdas/root/pmdaroot
             ├─670601 /var/lib/pcp/pmdas/proc/pmdaproc -d 3
             ├─670603 /var/lib/pcp/pmdas/xfs/pmdaxfs -d 11
             ├─670605 /var/lib/pcp/pmdas/linux/pmdalinux
             ├─670606 /var/lib/pcp/pmdas/kvm/pmdakvm -d 95
             ├─670607 python3 /var/lib/pcp/pmdas/libvirt/pmdalibvirt.python
             └─670922 /var/lib/pcp/pmdas/podman/pmdapodman -d 33
May 04 20:51:46 systemd[1]: Starting Performance Metrics Collector Daemon...
May 04 20:51:46 systemd[1]: Started Performance Metrics Collector Daemon.
May 04 20:51:46 pmcd[670645]: Installing podman PMDA ...

Is .NeedInstall a special way of installation?

When I want to fetch the container names , I do got wrong values.

$ pminfo --fetch podman.container.name
podman.container.name
    inst [0 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"
    inst [1 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"
    inst [2 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"
    inst [3 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"
    inst [4 or "**cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba**"] value "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"

When I request the data with curl I do get the correct values;

 curl -s --unix-socket /run/user/1000/podman/podman.sock http://d/v3.0.0/libpod/containers/json | jq
[
  {
    "AutoRemove": false,
    "Command": null,
    "Created": "2023-03-30T08:34:24.799258169+02:00",
    "CreatedAt": "",
    "Exited": false,
    "ExitedAt": 1681940209,
    "ExitCode": 0,
    "Id": "**cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba**",
    "Image": "registry.redhat.io/rhel9/grafana:latest",
    "ImageID": "fd8fc14372b7b67a2bf96c620fb35cbffab4499e4c380d7d431b566fde01cf4f",
    "IsInfra": false,
    "Labels": {
      "architecture": "x86_64",
      "build-date": "2023-02-22T14:01:27",
      "com.redhat.component": "grafana-container",
      "com.redhat.license_terms": "https://www.redhat.com/agreements",
      "description": "Grafana is an open source, feature rich metrics dashboard and graph editor for Graphite, Elasticsearch, OpenTSDB, Prometheus, InfluxDB and Performance Co-Pilot.",
      "distribution-scope": "public",
      "help": "cat /README.md",
      "io.buildah.version": "1.27.3",
      "io.k8s.description": "Grafana is an open source, feature rich metrics dashboard and graph editor for Graphite, Elasticsearch, OpenTSDB, Prometheus, InfluxDB and Performance Co-Pilot.",
      "io.k8s.display-name": "Grafana",
      "io.openshift.expose-services": "3000:grafana",
      "io.openshift.tags": "grafana,monitoring,dashboard",
      "maintainer": "Grafana Maintainers <[email protected]>",
      "name": "rhel9/grafana",
      "release": "110",
      "summary": "Grafana is an open source, feature rich metrics dashboard and graph editor",
      "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/rhel9/grafana/images/7-110",
      "usage": "podman run -d --name grafana -p 3000:3000 -v grafana-data:/var/lib/grafana registry.redhat.io/rhel9/grafana",
      "vcs-ref": "9d2153d82ddf485a5c15318366cf5a98affde1d4",
      "vcs-type": "git",
      "vendor": "Red Hat, Inc.",
      "version": "7"
    },
    "Mounts": [
      "/var/lib/grafana"
    ],
    "**Names**": [
      "grafana"
    ],
    "Namespaces": {},
    "Networks": [],
    "Pid": 79670,
    "Pod": "",
    "PodName": "",
    "Ports": [
      {
        "host_ip": "",
        "container_port": 3000,
        "host_port": 3000,
        "range": 1,
        "protocol": "tcp"
      }
    ],
    "Size": null,
    "StartedAt": 1681983068,
    "State": "running",
    "Status": ""
  },

Seems like I found a bug. After some time , I am getting the following errors in journalog

May 04 21:30:07 systemd-coredump[681449]: [🡕] Process 681394 (pmdapodman) of user 0 dumped cor> ░░ Subject: Process 681394 (pmdapodman) dumped core ░░ Defined-By: systemd ░░ Support: https://access.redhat.com/support ░░ Documentation: man:core(5) ░░ ░░ Process 681394 (pmdapodman) crashed and dumped core. ░░ ░░ This usually indicates a programming error in the crashing program and ░░ should be reported to its vendor as a bug. May 04 21:31:06 systemd-coredump[681698]: [🡕] Process 681645 (pmdapodman) of user 0 dumped cor> ░░ Subject: Process 681645 (pmdapodman) dumped core ░░ Defined-By: systemd ░░ Support: https://access.redhat.com/support ░░ Documentation: man:core(5) ░░ ░░ Process 681645 (pmdapodman) crashed and dumped core. ░░ ░░ This usually indicates a programming error in the crashing program and ░░ should be reported to its vendor as a bug.

sercanyaldiz avatar May 04 '23 19:05 sercanyaldiz

Interesting, thanks @sercanyaldiz - looks like podman is returning different output to previously (do you know what that '**' notation there means OOC? how many containers do you actually have running there) - I'll take a closer look when time comes available.

natoscott avatar May 10 '23 07:05 natoscott

@natoscott, Once again thank you for your time. I tried to highlight the container names with '**'. :)

I have 5 containers running. The APIs the PCP code is using , does returns the correct values.

But pmdapodman is reporting the following 'null' value metrics;

 $ pminfo --fetch podman
podman.container.name
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value "null"
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value "null"
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value "null"
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value "null"
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value "null"
podman.container.command
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value "null"
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value "null"
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value "null"
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value "null"
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value "null"
podman.container.status
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value ""
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value ""
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value ""
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value ""
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value ""
podman.container.running
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 1
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 1
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 1
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 1
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 1
podman.container.image
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value "null"
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value "null"
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value "null"
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value "null"
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value "null"
podman.container.pod
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value ""
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value ""
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value ""
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value ""
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value ""
podman.container.stats.net_input
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 695640
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 109698
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 0
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 74963968
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 45788541
podman.container.stats.net_output
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 59772764
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 1458006
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 0
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 674355954
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 591452790
podman.container.stats.block_input
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 0
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 0
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 0
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 0
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 0
podman.container.stats.block_output
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 0
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 0
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 0
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 0
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 0
podman.container.stats.cpu
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 0.06421809345043963
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 0.1965012084765377
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 0.007704793088730479
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 0.3726398512761362
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 0.2408749830775645
podman.container.stats.cpu_nano
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 1117751738000
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 3420215434000
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 134106044000
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 1852064794000
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 1419310272000
podman.container.stats.system_nano
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 530467023
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 1362437787
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 39442641
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 818883485
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 937246286
podman.container.stats.mem_usage
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 84647936
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 126242816
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 15482880
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 171147264
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 101720064
podman.container.stats.mem_limit
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 66709114880
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 66709114880
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 66709114880
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 66709114880
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 66709114880
podman.container.stats.mem_perc
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 0.1268911094867161
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 0.1892437281278465
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 0.02320954194618149
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 0.2565575398622348
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 0.1524830065321352
podman.container.stats.pids
    inst [0 or "cf24d60248a5e8d7909ec5dd0d44d50913331c98c11e0033bab5f653bdf89eba"] value 44
    inst [1 or "7862962360acc1621aef51b3f23e1eed8626963000a7dd4a7d49ad579f38e1e1"] value 33
    inst [2 or "7b97311b3892557db9fa098aa0943a7987f5bd875e24d833ef259f151d3c8781"] value 10
    inst [3 or "6b347bfb4e03f1d4db2d4b0b9d3763f6cd1389044e437b9a5c4e22dd55272069"] value 12
    inst [4 or "626bc7b270ab95c2040a2373b24fcc870ed4da0904023fc80c621890a6061c4c"] value 40
podman.pod.name
No value(s) available!
podman.pod.cgroup
No value(s) available!
podman.pod.status
No value(s) available!
podman.pod.containers
No value(s) available!

sercanyaldiz avatar May 10 '23 13:05 sercanyaldiz

@sercanyaldiz ah, I understand now - my email client didn't render the '**' in the same way as github, heh. Thanks for explaining - from your earlier output it looks like command is indeed null but the Name: field has now become a Names list ... and PCP code is unaware.

natoscott avatar May 10 '23 23:05 natoscott

I've got some containers running w/ a non-root user named ansible (UID 1001) and I can't seem to get this to work.

I've tried

# touch /var/lib/pcp/pmdas/podman/.NeedInstall
# systemctl restart pmcd

Is that what the ./Install script should be doing? What is the ./Install script for -- or is this a workaround for a bug?

I've confirmed that commands like the following output sensible info:

curl -s --unix-socket /run/user/1001/podman/podman.sock http://d/v1.0.0/libpod/images/json
curl -s --unix-socket /run/user/1001/podman/podman.sock http://d/v1.0.0/libpod/info

But the following show an empty list, despite the fact that there are multiple containers running:

curl -s --unix-socket /run/user/1001/podman/podman.sock http://d/v1.0.0/libpod/containers/json

There was some mention of containers.json under an overlay storage directory. For me I found that in /home/ansible/.local/share/containers/storage/overlay-containers/containers.json

Any ideas about something I could do differently, or is this just not yet a feature?

kdelee avatar Jan 09 '24 21:01 kdelee

Any of .NeedInstall and ./Install should do the basic installation.

Article https://www.redhat.com/ja/blog/examining-container-performance-rhel-8-pcp-and-pdma-podman is introducing into pmda-podman usage (not specifically considering root/root-less containers though).

christianhorn avatar Jan 10 '24 01:01 christianhorn