glances
glances copied to clipboard
nicolargo
Podman containers monitoring support
Is your feature request related to a problem? Please describe. When using podman instead of docker, glances fails with an exception.
Describe the solution you'd like Glances should not fail to start. Podman containers should be visible in a similar manner as docker containers. Podman supports Docker API v1.40: https://docs.podman.io/en/latest/markdown/podman-system-service.1.html
Describe alternatives you've considered n/a
Additional context n/a
Steps to reproduce
- Install podman
- Enable podman service `systemctl start --user podman.service
- Set
DOCKER_HOSTenvironment variable:
export XDG_RUNTIME_DIR=/run/user/$(id -u) # this should not be needed - XDG_RUNTIME_DIR should be already set on your system
export DOCKER_HOST="unix://$XDG_RUNTIME_DIR/podman/podman.sock"
- Start glances - it hangs with the following error (needs to be ctrl-c'd):
glances
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 268, in _raise_for_status
response.raise_for_status()
File "/usr/lib/python3.10/site-packages/requests/models.py", line 953, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.40/images/latest/json
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/glances", line 33, in <module>
sys.exit(load_entry_point('Glances==3.2.4', 'console_scripts', 'glances')())
File "/usr/lib/python3.10/site-packages/glances/__init__.py", line 150, in main
start(config=core.get_config(), args= core.get_args())
File "/usr/lib/python3.10/site-packages/glances/__init__.py", line 109, in start
mode = GlancesMode(config=config, args=args)
File "/usr/lib/python3.10/site-packages/glances/standalone.py", line 80, in __init__
self.stats.update()
File "/usr/lib/python3.10/site-packages/glances/stats.py", line 230, in update
self._plugins[p].update()
File "/usr/lib/python3.10/site-packages/glances/plugins/glances_plugin.py", line 1151, in wrapper
ret = fct(self, *args, **kw)
File "/usr/lib/python3.10/site-packages/glances/plugins/glances_plugin.py", line 1166, in wrapper
ret = fct(*args, **kw)
File "/usr/lib/python3.10/site-packages/glances/plugins/glances_docker.py", line 250, in update
container_stats['Image'] = container.image.tags
File "/usr/lib/python3.10/site-packages/docker/models/containers.py", line 40, in image
return self.client.images.get(image_id.split(':')[1])
File "/usr/lib/python3.10/site-packages/docker/models/images.py", line 314, in get
return self.prepare_model(self.client.api.inspect_image(name))
File "/usr/lib/python3.10/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/usr/lib/python3.10/site-packages/docker/api/image.py", line 251, in inspect_image
return self._result(
File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 274, in _result
self._raise_for_status(response)
File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "/usr/lib/python3.10/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.40/images/latest/json: Not Found ("failed to find image latest: latest: No such image")
^CException ignored in: <module 'threading' from '/usr/lib/python3.10/threading.py'>
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1560, in _shutdown
lock.acquire()
File "/usr/lib/python3.10/site-packages/glances/__init__.py", line 69, in __signal_handler
end()
File "/usr/lib/python3.10/site-packages/glances/__init__.py", line 84, in end
sys.exit(0)
SystemExit: 0
Hi,
please you send us the Glances and Python Docker Client version (pip freeze | grep docker) ?
I have similar problem using podman on CentOS Stream 9
# glances
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/docker/api/client.py", line 268, in _raise_for_status
response.raise_for_status()
File "/usr/lib/python3.9/site-packages/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.40/images/v1.0.11/json
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/glances", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.9/site-packages/glances/__init__.py", line 150, in main
start(config=core.get_config(), args=core.get_args())
File "/usr/local/lib/python3.9/site-packages/glances/__init__.py", line 109, in start
mode = GlancesMode(config=config, args=args)
File "/usr/local/lib/python3.9/site-packages/glances/standalone.py", line 80, in __init__
self.stats.update()
File "/usr/local/lib/python3.9/site-packages/glances/stats.py", line 230, in update
self._plugins[p].update()
File "/usr/local/lib/python3.9/site-packages/glances/plugins/glances_plugin.py", line 1158, in wrapper
ret = fct(self, *args, **kw)
File "/usr/local/lib/python3.9/site-packages/glances/plugins/glances_plugin.py", line 1173, in wrapper
ret = fct(*args, **kw)
File "/usr/local/lib/python3.9/site-packages/glances/plugins/glances_docker.py", line 250, in update
container_stats['Image'] = container.image.tags
File "/usr/local/lib/python3.9/site-packages/docker/models/containers.py", line 40, in image
return self.client.images.get(image_id.split(':')[1])
File "/usr/local/lib/python3.9/site-packages/docker/models/images.py", line 314, in get
return self.prepare_model(self.client.api.inspect_image(name))
File "/usr/local/lib/python3.9/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/docker/api/image.py", line 251, in inspect_image
return self._result(
File "/usr/local/lib/python3.9/site-packages/docker/api/client.py", line 274, in _result
self._raise_for_status(response)
File "/usr/local/lib/python3.9/site-packages/docker/api/client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "/usr/local/lib/python3.9/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.40/images/v1.0.11/json: Not Found ("failed to find image v1.0.11: v1.0.11: No such image")
^CException ignored in: <module 'threading' from '/usr/lib64/python3.9/threading.py'>
Traceback (most recent call last):
File "/usr/lib64/python3.9/threading.py", line 1470, in _shutdown
lock.acquire()
File "/usr/local/lib/python3.9/site-packages/glances/__init__.py", line 69, in __signal_handler
end()
File "/usr/local/lib/python3.9/site-packages/glances/__init__.py", line 84, in end
sys.exit(0)
SystemExit: 0
# dnf list installed | grep podman
podman.x86_64 2:3.4.5-0.7.el9 @appstream
podman-catatonit.x86_64 2:3.4.5-0.7.el9 @appstream
podman-docker.noarch 2:3.4.5-0.7.el9 @appstream
# glances --version
Glances v3.2.4.2 with PsUtil v5.9.0
Log file: /tmp/glances-root.log
# pip freeze | grep docker
docker==5.0.3
Same problem on MicroOS
# glances
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 268, in _raise_for_status
response.raise_for_status()
File "/usr/lib/python3.10/site-packages/requests/models.py", line 960, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.40/images/latest/json
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/glances", line 33, in <module>
sys.exit(load_entry_point('Glances==3.2.5', 'console_scripts', 'glances')())
File "/usr/lib/python3.10/site-packages/glances/__init__.py", line 189, in main
start(config=core.get_config(), args=core.get_args())
File "/usr/lib/python3.10/site-packages/glances/__init__.py", line 118, in start
mode = GlancesMode(config=config, args=args)
File "/usr/lib/python3.10/site-packages/glances/standalone.py", line 80, in __init__
self.stats.update()
File "/usr/lib/python3.10/site-packages/glances/stats.py", line 221, in update
self._plugins[p].update()
File "/usr/lib/python3.10/site-packages/glances/plugins/glances_plugin.py", line 1157, in wrapper
ret = fct(self, *args, **kw)
File "/usr/lib/python3.10/site-packages/glances/plugins/glances_plugin.py", line 1174, in wrapper
ret = fct(*args, **kw)
File "/usr/lib/python3.10/site-packages/glances/plugins/glances_docker.py", line 258, in update
container_stats['Image'] = container.image.tags
File "/usr/lib/python3.10/site-packages/docker/models/containers.py", line 40, in image
return self.client.images.get(image_id.split(':')[1])
File "/usr/lib/python3.10/site-packages/docker/models/images.py", line 314, in get
return self.prepare_model(self.client.api.inspect_image(name))
File "/usr/lib/python3.10/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/usr/lib/python3.10/site-packages/docker/api/image.py", line 251, in inspect_image
return self._result(
File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 274, in _result
self._raise_for_status(response)
File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "/usr/lib/python3.10/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.40/images/latest/json: Not Found ("failed to find image latest: docker.io/library/latest: No such image")
^CException ignored in: <module 'threading' from '/usr/lib64/python3.10/threading.py'>
Traceback (most recent call last):
File "/usr/lib64/python3.10/threading.py", line 1560, in _shutdown
lock.acquire()
File "/usr/lib/python3.10/site-packages/glances/__init__.py", line 75, in __signal_handler
end()
File "/usr/lib/python3.10/site-packages/glances/__init__.py", line 90, in end
sys.exit(0)
SystemExit: 0
# glances --version
Glances v3.2.5 with PsUtil v5.9.0
Don't have pip installed so
# rpm -qa|grep docker
python310-docker-5.0.3-2.1.noarch
Manually testing pulling data from socket returns info without problems
curl --unix-socket /var/run/docker.sock http://localhost/v1.40/images/json|jq
@daqqad I presume you also have podman, then shouldn't you be using that unix socket?
Also I did look into the API a bit, some fields seem to be receiving fixed values(IO & Network), while memory and cpu usage were fine. Getting the images list fails for some reason. Need more research on it.
Another alternative would be to make another plugin: If we do choose that option this might be helpful: https://github.com/containers/podman-py (https://pypi.org/project/podman/)
@RazCrimson I updated service file and renamed the socket for compatibility, but I am using podman.
+1 for the @RazCrimson proposal. If the Podman API is not fully compliant with the Docker one, the best solution is to create a new Podman plugin for Glances. Under the wood, if Podman is detected, the Docker plugin should be disable automatically. For the dev part, we should avoid duplication code between the two plugins...
Added in the Glances version 4 roadmap (because the Python Podman Lib is only available on Python 3).
@nicolargo I think we should not automatically disable the docker plugin too. If we do, we should atleast provide the option to enable it.
There might be edge cases where someone would want to monitor both docker and podman containers. 🙂
There might be edge cases where someone would want to monitor both docker and podman containers. slightly_smiling_face
Absolutely, I've found the most natural way to get started with podman is to gradually migrate and test services in case of incompatibilities. Having the option to monitor both makes sense. Could a non-required recommendation by offered to disable docker when enabling podman, or vice versa.
Error during first test: https://github.com/containers/podman-py/issues/179
Quick and dirty sandbox:
Start some pod:
podman pod create --name podtest
podman pod list
podman run -dt --pod podtest ubuntu top
podman ps -a --pod
Execute the Python code (https://gist.github.com/nicolargo/b9e6bd079b07fe64cb8c9b798a219cb0):
Release: 3.4.4
Compatible API: 1.40
Podman API: 3.4.4
<Image: 'docker.io/library/alpine:latest'> e66264b98777e12192600bf9b4d663655c98a090072e1bab49e233d7531d1294
<Image: 'k8s.gcr.io/pause:3.5'> ed210e3e4a5bae1237f1bb44d72a05a2f1e5c6bfe7a7e73da179e2534269c459
<Image: 'docker.io/library/ubuntu:latest'> 216c552ea5ba7b0e3f6e33624e129981c39996021403518019d19b8843c27cbc
================================================================================
AutoRemove False
Command None
Created 2022-10-22T14:23:03.120912374+02:00
CreatedAt
ExitCode 0
Exited False
ExitedAt -62135596800
Id 9491515251edcd5bb5dc17205d7ee573c0be96fe0b08b0a12a7e2cea874565ea
Image k8s.gcr.io/pause:3.5
ImageID ed210e3e4a5bae1237f1bb44d72a05a2f1e5c6bfe7a7e73da179e2534269c459
IsInfra True
Labels None
Mounts []
Names ['8d0f1c783def-infra']
Namespaces {}
Networks None
Pid 181221
Pod 8d0f1c783def193085bf68d2f245971e1c33562d96a0ee6952396865de3cb4ec
PodName podtest
Ports None
Size None
StartedAt 1666441640
State running
Status
================================================================================
AutoRemove False
Command ['top']
Created 2022-10-22T14:27:20.000770538+02:00
CreatedAt
ExitCode 0
Exited False
ExitedAt -62135596800
Id 1da5bc154a086d39eb2fce44a0ab11b89aab7bae86f5752cac0eb3393e9de9de
Image docker.io/library/ubuntu:latest
ImageID 216c552ea5ba7b0e3f6e33624e129981c39996021403518019d19b8843c27cbc
IsInfra False
Labels None
Mounts []
Names ['hungry_lalande']
Namespaces {}
Networks None
Pid 181228
Pod 8d0f1c783def193085bf68d2f245971e1c33562d96a0ee6952396865de3cb4ec
PodName podtest
Ports None
Size None
StartedAt 1666441640
State running
Status
Stop the pod:
podman stop podtest
Main issue: no information concerning the CPU / RAM / DiskIO via this lib but should be exposed by the API because the podman stats command line display the following information:
Another way:
podman stats --no-stream --format=json -a
Return:
[
{
"id": "1da5bc154a08",
"name": "hungry_lalande",
"cpu_time": "675.345ms",
"cpu_percent": "4.05%",
"avg_cpu": "4.05%",
"mem_usage": "876.5kB / 7.836GB",
"mem_percent": "0.01%",
"net_io": "-- / --",
"block_io": "-- / --",
"pids": "1"
},
{
"id": "9491515251ed",
"name": "8d0f1c783def-infra",
"cpu_time": "5.15ms",
"cpu_percent": "3.09%",
"avg_cpu": "3.09%",
"mem_usage": "221.2kB / 7.836GB",
"mem_percent": "0.00%",
"net_io": "-- / --",
"block_io": "-- / --",
"pids": "1"
}
]
Could have a look on this repository https://github.com/m-erhardt/check-container-stats/blob/master/check_container_stats_podman.py for a similar approach (but JSON format is better...).
Create a new branch for the ongoing development: https://github.com/nicolargo/glances/tree/issue1985
@nicolargo How do you want the Podman to be supported? The current changes in https://github.com/nicolargo/glances/tree/issue1985 seems to modify the existing plugin to support podman containers too.
So would you prefer a new podman plugin or make the current one support podman too?
In the latter case, should we have a column to indicate the container engine from which a container was found?
API corrected after a restart of the Podman service.
$ python
>>> from podman import PodmanClient
>>> uri = "unix:///run/user/1000/podman/podman.sock"
>>> PodmanClient(base_url=uri).version()
{'Platform': {'Name': 'linux/amd64/ubuntu-22.04'}, 'Components': [{'Name': 'Podman Engine', 'Version': '3.4.4', 'Details': {'APIVersion': '3.4.4', 'Arch': 'amd64', 'BuildTime': '1970-01-01T01:00:00+01:00', 'Experimental': 'false', 'GitCommit': '', 'GoVersion': 'go1.17.3', 'KernelVersion': '5.15.0-58-generic', 'MinAPIVersion': '3.1.0', 'Os': 'linux'}}, {'Name': 'Conmon', 'Version': 'conmon version 2.0.25, commit: unknown', 'Details': {'Package': 'conmon: /usr/bin/conmon'}}, {'Name': 'OCI Runtime (crun)', 'Version': 'crun version 0.17\ncommit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL', 'Details': {'Package': 'crun: /usr/bin/crun'}}], 'Version': '3.4.4', 'ApiVersion': '1.40', 'MinAPIVersion': '1.24', 'GitCommit': '', 'GoVersion': 'go1.17.3', 'Os': 'linux', 'Arch': 'amd64', 'KernelVersion': '5.15.0-58-generic', 'BuildTime': '1970-01-01T01:00:00+01:00'}
Get images list:
>>> for image in client.images.list():
... print(image, image.id, "\n")
...
<Image: 'docker.io/library/alpine:latest'> e66264b98777e12192600bf9b4d663655c98a090072e1bab49e233d7531d1294
<Image: 'k8s.gcr.io/pause:3.5'> ed210e3e4a5bae1237f1bb44d72a05a2f1e5c6bfe7a7e73da179e2534269c459
<Image: 'docker.io/library/ubuntu:latest'> 216c552ea5ba7b0e3f6e33624e129981c39996021403518019d19b8843c27cbc
Images and containers infos:
>>> print(json.dumps(client.df(), indent=4))
{
"Images": [
{
"Repository": "docker.io/library/alpine",
"Tag": "latest",
"ImageID": "e66264b98777e12192600bf9b4d663655c98a090072e1bab49e233d7531d1294",
"Created": "2022-05-23T19:19:31.970967174Z",
"Size": 5815366,
"SharedSize": 0,
"UniqueSize": 5815366,
"Containers": 1
},
{
"Repository": "k8s.gcr.io/pause",
"Tag": "3.5",
"ImageID": "ed210e3e4a5bae1237f1bb44d72a05a2f1e5c6bfe7a7e73da179e2534269c459",
"Created": "2021-03-16T13:16:57.822648569Z",
"Size": 689969,
"SharedSize": 0,
"UniqueSize": 689969,
"Containers": 1
},
{
"Repository": "docker.io/library/ubuntu",
"Tag": "latest",
"ImageID": "216c552ea5ba7b0e3f6e33624e129981c39996021403518019d19b8843c27cbc",
"Created": "2022-10-04T23:35:20.857335994Z",
"Size": 80360801,
"SharedSize": 0,
"UniqueSize": 80360801,
"Containers": 2
}
],
"Containers": [
{
"ContainerID": "1da5bc154a086d39eb2fce44a0ab11b89aab7bae86f5752cac0eb3393e9de9de",
"Image": "216c552ea5ba7b0e3f6e33624e129981c39996021403518019d19b8843c27cbc",
"Command": [
"top"
],
"LocalVolumes": 0,
"Size": 77837008,
"RWSize": 34,
"Created": "2022-10-22T14:27:20.000770538+02:00",
"Status": "configured",
"Names": "hungry_lalande"
},
{
"ContainerID": "28cb9c28fef7dc64c8ef000f436fb867c355d0a85861ae5cab106ab26126ec7d",
"Image": "e66264b98777e12192600bf9b4d663655c98a090072e1bab49e233d7531d1294",
"Command": [
"/bin/sh"
],
"LocalVolumes": 0,
"Size": 5529096,
"RWSize": 104,
"Created": "2022-05-29T10:15:53.954943081+02:00",
"Status": "exited",
"Names": "test"
},
{
"ContainerID": "9491515251edcd5bb5dc17205d7ee573c0be96fe0b08b0a12a7e2cea874565ea",
"Image": "ed210e3e4a5bae1237f1bb44d72a05a2f1e5c6bfe7a7e73da179e2534269c459",
"Command": [],
"LocalVolumes": 0,
"Size": 682696,
"RWSize": 12,
"Created": "2022-10-22T14:23:03.120912374+02:00",
"Status": "running",
"Names": "8d0f1c783def-infra"
},
{
"ContainerID": "a1206796715fc73abb1146bd0f5a4027305dcf0227bba762d660cd62462ff8ad",
"Image": "216c552ea5ba7b0e3f6e33624e129981c39996021403518019d19b8843c27cbc",
"Command": [
"top"
],
"LocalVolumes": 0,
"Size": 77837008,
"RWSize": 34,
"Created": "2023-01-21T11:55:38.718496053+01:00",
"Status": "running",
"Names": "strange_lewin"
}
],
"Volumes": []
}
@RazCrimson
i just push some change in the dedicated branche: https://github.com/nicolargo/glances/tree/issue1985
But i am not satisfied of the current implementation and some features should be implemented for the Podman containers.
Here is the current status in the console UI:
(the container in the middle is a Docker, others are Podman).
What we should do:
- [ ] refactor the code in order (currently in glances_docker.py) to make the plugin compliant with multiple containers engines. So rename glances_docker.py to glances_containers.py and put the engines implementation in a subfolder. For example containers/glances_docker.py or glances_podman.py...).
- [ ] the UI (both console and WebUI) should be change to display the engine type (a new column with the engine name will be displayed only if more than one engine is used)
- [ ] some engine will have specifics informations (Pod name for Podman for example)... How to display (or not) those information (to be discussed)
- [ ] concerning Podman, some stats are not display (uptime and IO. For IO there is an open issue https://github.com/containers/podman/issues/11695)
@RazCrimson any free time to contribute to this enhancement request ?
@nicolargo Just an architectural question for the implementation.
I was thinking if we could try moving towards more typed-style of Python and more structured data objects instead of dicts (maybe use dataclasses) as that would probably reduce lot of errors due to missing keys or having typos in keys and would also make the code more readable.
Currently, planning to use typing features that are available in python3.6(so won't be breaking compatibility for python3) for the containers plugin and maintain the existing dict interface so that nothing breaks.
What are your thoughts on this?
I plan tu remove Python 2 support in Glances 4.0. There is a glances4 branche with part of thé wok done (juste removing Python 2 specific code buth with refactoring like dataclass). Release 3.x should bé complanté with Python 2. So if you want to use metaclass you should use glance4 branche and marge the feature branche of this issue into it.
Le ven. 10 févr. 2023, 18:26, RazCrimson @.***> a écrit :
@nicolargo https://github.com/nicolargo Just an architectural question for the implementation.
I was thinking if we could try moving towards more typed-style of Python and more structured data objects instead of dicts (maybe use dataclasses https://docs.python.org/3/library/dataclasses.html) as that would probably reduce lot of errors due to missing keys or having typos in keys and would also make the code more readable.
Currently, planning to use typing features that are available in python3.6 for the containers plugin and maintain the existing dict interface so that nothing breaks.
What are your thoughts on this?
— Reply to this email directly, view it on GitHub https://github.com/nicolargo/glances/issues/1985#issuecomment-1426118083, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF5UKYI52NM5T6X5ERZVXTWWZ22TANCNFSM5LDTYDZQ . You are receiving this because you were mentioned.Message ID: @.***>
@nicolargo A Basic version of the podman plugin is ready. It still requires improvements, but if possible I would like your comments on the current implementation.
The podman-py library also needs some minor changes, I'll try and get them upstreamed as soon as possible.
To run the current version of the PR, you need to install podman as follows:
pip install "git+https://github.com/RazCrimson/podman-py.git@glances-test"
Questions for you:
-
I have not merged the
glancesv4branch into this yet. Shall I go ahead and do that? So we change the milestone of this ticket to glances 4.0.0? -
Could you confirm if the current layout for the
Enginename andPodare fine? I'll do the WebUI changes after that.
Also I think there might be some issues with the stats that podman returns for not just NetIO, but BlockIO too.
Hi @RazCrimson !
I have merge your PR in the issue1985 branche.
Unfortunatly, after the merge, Glances console UI do not start anymore.
Here is the debug log file: https://pastebin.com/z8u0mf7Q
Any idea ?
PS: my deps are up to date. PS2: Glances starts correctly when i disable the containers plugin (./venv/bin/python -m glances -C ./conf/glances.conf --disable-plugin containers)
@nicolargo Do you have the patched podman lib mentioned above? If not you will face the hanging issue that you mentioned in the podman-py issue tracker
I have not updated the requirements file to use the patched lib, so it requires manual removal of the original podman lib followed by the install of the patched version.
Can you try the below after activating your venv and then run glances?
pip uninstall podman
pip install "git+https://github.com/RazCrimson/podman-py.git@glances-test"
On another note, the initial stream object generation should probably be moved inside the threaded worker rather than the main glances process. Will make those changes along with the cleanups
@RazCrimson my bad, i think that the venv-upgrade target of the Makefile did the trick.
It works now, i will have a look on the code.
@nicolargo Sure no problems :)
Btw the code still needs to be refactored. Need the below to be addressed to finish the refactor.
Podman issues podman-py does not expose certain features of the Podman API that would let us stream stats of all containers as opposed to just one container at a time (issue) What do you think about this? Having one thread grab all stats opposed to a thread for each container sounds less of an overhead to me. Shall I go ahead and switch over to this approach?
Other Questions
- Any plans for which version this feature is gonna be released in? glances 4.0 or 3.4? Refactoring could be better if we don't need to ensure py2 compatibility.
- Assuming we release in glances 4.0. What is the minimum py3 version that glances 4.0 will target? Python 3.4, 3.5 and 3.6 have quite low support for typing which I think would be quite nice to have in a large code base such as glances.
- Any ideas on other container management systems like Kubernetes (#2026) and other engines (#2226)?
Will they become a part of the
containersplugin or we currently don't plan to implement them?
Concerning the Podman issue, having one thread to grab all the stats in one shot should be the target solution when the Podman-Py issue https://github.com/containers/podman-py/issues/239 will be release.
About your others questions:
- I think that the best approach will be to have a first version for the release 3.4 (release date june 2023). But we also have to be sure that the Podman-Py issue will be release in the upstream. Concerning the refactoring, it is up to you... a first round could be done for the release 3.4 and a second one for the version 4.0 (with a support for Python 3 only)
- Several points should be taken into account to define the minimal Python 3 version: Python EOL (https://endoflife.date/python), minimal Python version used in all deps, Python version embedded in LTS Linux Operating system (Ubuntu and Redhat). For the moment, taken into account those points and a release date of january 2024 for Glances version 4) i will say Python 3.8
- Kubernetes support will be a great improvement for the containers plugin
Looks like the Podman-Py issue has been taken into account in the version 4.4.1 but i can not find any release note :(
Yea, they didn't mention the bug fix in the changelog.
We still need some other changes for feature like https://github.com/containers/podman-py/issues/239 and other problems related to pods stats mentioned in the same issue.
I'll try to get those changes done in the next week or so and upstream them.
@RazCrimson Hi ! The last Podman-Py version 4.5 has been released.
On my system, when i test the issue1985 branch, i have the following error:
./venv/bin/python -m glances -C ./conf/glances.conf
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/nicolargo/dev/glances/glances/__main__.py", line 18, in <module>
glances.main()
File "/home/nicolargo/dev/glances/glances/__init__.py", line 180, in main
start(config=core.get_config(), args=core.get_args())
File "/home/nicolargo/dev/glances/glances/__init__.py", line 108, in start
mode = GlancesMode(config=config, args=args)
File "/home/nicolargo/dev/glances/glances/standalone.py", line 74, in __init__
self.stats.update()
File "/home/nicolargo/dev/glances/glances/stats.py", line 216, in update
self._plugins[p].update()
File "/home/nicolargo/dev/glances/glances/plugins/glances_plugin.py", line 1114, in wrapper
ret = fct(self, *args, **kw)
File "/home/nicolargo/dev/glances/glances/plugins/glances_plugin.py", line 1131, in wrapper
ret = fct(*args, **kw)
File "/home/nicolargo/dev/glances/glances/plugins/glances_containers.py", line 160, in update
stats_podman = self.update_podman() if self.podman_client else {}
File "/home/nicolargo/dev/glances/glances/plugins/glances_containers.py", line 186, in update_podman
version, containers = self.podman_client.update(all_tag=self._all_tag())
File "/home/nicolargo/dev/glances/glances/plugins/containers/glances_podman.py", line 285, in update
pod_stats = self.pod_fetcher.activity_stats
File "/home/nicolargo/dev/glances/glances/plugins/containers/glances_podman.py", line 101, in activity_stats
cpu_stats = self._get_cpu_stats(stat)
File "/home/nicolargo/dev/glances/glances/plugins/containers/glances_podman.py", line 127, in _get_cpu_stats
cpu_usage = string_value_to_float(stats["CPU"].rstrip("%"))
TypeError: string indices must be integers
make: *** [Makefile:153: run] Error 1
The stats variable is equal to the string "CPU"...
Can you have a look on it ?
Thank and have a nice day !
Sorry for the late reply @nicolargo Bit busy this week. 😅
I found what the issue is. Will try to get it fixed by this weekend.