prometheus-api-client-python

prometheus-api-client-python copied to clipboard

Allow closing the connection/session

[{"_id":"66cd8deeb3dc7ee8eb101243","body":"@FRosner with the latest release, you can pass a custom session to `PrometheusConnect`. Does that work for your usecase? ","issue_id":1709750358481,"origin_id":1957986128,"user_origin_id":22333506,"create_time":1708550832,"update_time":1708550832,"id":1724747246049,"updated_at":"2024-08-27T08:27:26.048000Z","created_at":"2024-08-27T08:27:26.048000Z"},{"_id":"66cd8deeb3dc7ee8eb101244","body":"@4n4nd thanks! I guess it still leaves it to us to close the session, but it's for sure better than accessing the private variable. Do you think there's value in providing a close method that closes the session (may it be provided or not), and maybe even the context manager methods?","issue_id":1709750358481,"origin_id":1963748149,"user_origin_id":3427394,"create_time":1708942067,"update_time":1708942067,"id":1724747246054,"updated_at":"2024-08-27T08:27:26.053000Z","created_at":"2024-08-27T08:27:26.053000Z"}]

**Is your feature request related to a problem? Please describe.** We can open a client, but we cannot close the session. We have to work around it providing our own...

FRosner

Is httmock a runtime dependency or only required for tests?

[{"_id":"66cd8e21b3dc7ee8eb101253","body":"It's required only for tests","issue_id":1709750358483,"origin_id":1957996369,"user_origin_id":22333506,"create_time":1708551106,"update_time":1708551106,"id":1724747297733,"updated_at":"2024-08-27T08:28:17.733000Z","created_at":"2024-08-27T08:28:17.733000Z"},{"_id":"66cd8e21b3dc7ee8eb101254","body":"Sorry for the late reply, missed the notification mail. Thanks for clarifying.\r\n\r\nMaybe the requirements.txt could be split up into a requirements.txt and a requirements-test.txt like in other projects?","issue_id":1709750358483,"origin_id":2058343454,"user_origin_id":26167225,"create_time":1713249603,"update_time":1713249611,"id":1724747297750,"updated_at":"2024-08-27T08:28:17.750000Z","created_at":"2024-08-27T08:28:17.750000Z"}]

Is httmock a runtime dependency or only required for tests? https://github.com/4n4nd/prometheus-api-client-python/blob/49362b07cb3bec835a8211d81becd5ed205ace18/requirements.txt#L6 vs. https://github.com/4n4nd/prometheus-api-client-python/blob/master/setup.py#L40

johanneskastl

add pool_size setting

[{"_id":"66cd8e2bb3dc7ee8eb101257","body":"Hey @rbihore, with the latest changes in #274 you can provide a custom session with `PrometheusConnect`. Do you think it still makes sense to add this flag?","issue_id":1709750358489,"origin_id":1957970641,"user_origin_id":22333506,"create_time":1708550354,"update_time":1708550354,"id":1724747307886,"updated_at":"2024-08-27T08:28:27.886000Z","created_at":"2024-08-27T08:28:27.886000Z"}]

In the case of multithreaded use, it could be useful to be able to adjust the max_poolsize Added pool_size option in PrometeusConnect which sets a max_poolsize on the HTTPAdapter request...

rbihore

Bump pillow from 10.0.1 to 10.2.0

[]

Bumps [pillow](https://github.com/python-pillow/Pillow) from 10.0.1 to 10.2.0. Release notes Sourced from pillow's releases. 10.2.0 https://pillow.readthedocs.io/en/stable/releasenotes/10.2.0.html Changes Add keep_rgb option when saving JPEG to prevent conversion of RGB colorspace #7553 [@​bgilbert] Trim...

dependabot[bot]

Documentation Broken: `get_metric_aggregation` and `MetricRangeDataFrame`

[{"_id":"66cd8e45b3dc7ee8eb10125b","body":"Updated title to reflect another bug:\r\n\r\n\r\n\r\nLink to documentation: [here](https:\/\/prometheus-api-client-python.readthedocs.io\/en\/master\/source\/prometheus_api_client.html#prometheus_api_client.metric_range_df.MetricRangeDataFrame).\r\n\r\nThe key snippet of code is:\r\n\r\n```python\r\nmetric_data = prom.get_current_metric_value(metric_name='up', label_config=my_label_config)\r\nmetric_df = MetricRangeDataFrame(metric_data)\r\n```\r\n\r\nHowever `MetricRangeDataFrame` only works on ranges and therefore `prom.get_current_metric_value` is the wrong function, should be I think: `get_metric_range_data` (well possibly).","issue_id":1709750358498,"origin_id":1872618098,"user_origin_id":99686292,"create_time":1703975286,"update_time":1703975286,"id":1724747333737,"updated_at":"2024-08-27T08:28:53.737000Z","created_at":"2024-08-27T08:28:53.737000Z"}]

The formatting of the documentation for the function [`get_metric_aggregation`](https://prometheus-api-client-python.readthedocs.io/en/master/source/prometheus_api_client.html#prometheus_api_client.prometheus_connect.PrometheusConnect.get_metric_aggregation) appears to be broken as ilustrated below:  The description for the parameter `query` contains a link, which I suspect is...

miah0x41

Most of the tests are failing when packaging this for openSUSE: Failing due to missing network connectivity?

[]

**Describe the bug** I am trying to package this for openSUSE, but many tests are failing. The build is running in a build environment without internet connectivity. All of the...

johanneskastl

Handling of NaN values in metric aggregations

[{"_id":"66cd8e21784f76c77901d937","body":"Hi :)\r\n\r\nAny comments about this issue?","issue_id":1709750358508,"origin_id":1633734084,"user_origin_id":10234169,"create_time":1689234231,"update_time":1689234231,"id":1724747297786,"updated_at":"2024-08-27T08:28:17.785000Z","created_at":"2024-08-27T08:28:17.785000Z"}]

**Describe the bug** Not really sure if a bug or a design decision but I could not find this case covered in the tests. When using numpy to calculate the...

GPalaciosFdez

issue252 add controls to subplot

[{"_id":"66cd8e325a1d13a44707c812","body":"@mahtin I merged your latest PR, could you close the other two?","issue_id":1709750358523,"origin_id":1548235723,"user_origin_id":22333506,"create_time":1684170495,"update_time":1684170495,"id":1724747314412,"updated_at":"2024-08-27T08:28:34.411000Z","created_at":"2024-08-27T08:28:34.411000Z"}]

As per #252, coded and tested. Note that this controls the `subplots()` call and not the `plot_date()` call.

mahtin

request params not correctly passed to request.get in PrometheusConnect.custom_query

[{"_id":"66cd8e213be607015e090f7a","body":"Hi @GiulioGx, thanks for opening the issue! Line 346 seems to be a comment, is this referring to [line 355](https:\/\/github.com\/4n4nd\/prometheus-api-client-python\/blob\/master\/prometheus_api_client\/prometheus_connect.py#L355) instead?\r\n\r\nAnyway, I believe `params` are the parameters being passed for the query to the Prometheus server (see [this line](https:\/\/github.com\/4n4nd\/prometheus-api-client-python\/blob\/master\/prometheus_api_client\/prometheus_connect.py#L256) for an example), so the current implementation seems correct to me. I still tried replicating your suggestion as follows:\r\n```python\r\nimport requests\r\nfrom requests.adapters import HTTPAdapter\r\n\r\nurl = 'https:\/\/prometheus.demo.do.prometheus.io'\r\n\r\nsession = requests.Session()\r\nsession.mount(url, HTTPAdapter(max_retries=3))\r\n\r\nparams = {'time': 1680746497}\r\nresponse = session.get(\r\n \"{0}\/api\/v1\/query\".format(url),\r\n params={**{\"query\": \"ALERTS\"}},\r\n **params,\r\n)\r\n\r\nprint(response.content)\r\n```\r\nBut the above gave me an error, so I had to replace the call to be like the current implementation\r\n```python\r\nparams = {'time': 1680746497}\r\nresponse = session.get(\r\n \"{0}\/api\/v1\/query\".format(url),\r\n params={**{\"query\": \"ALERTS\"}, **params},\r\n)\r\n```\r\nwhich succeeded.\r\n\r\nDoes this help clarify your issue or did I misunderstand your problem?","issue_id":1709750358530,"origin_id":1498403828,"user_origin_id":15005500,"create_time":1680747949,"update_time":1680747949,"id":1724747297913,"updated_at":"2024-08-27T08:28:17.913000Z","created_at":"2024-08-27T08:28:17.913000Z"},{"_id":"66cd8e213be607015e090f7b","body":"Hi @chauhankaranraj, thanks for the quick reply!\r\n\r\nThe above code does not work because _time_ is not a parameter accepted by the _request()_ method. \r\nModifying _time_ to _timeout_ works for me.\r\n\r\n(The original issue I had was in fact trying to change the timeout to the http request to the prom server)","issue_id":1709750358530,"origin_id":1499082086,"user_origin_id":2845027,"create_time":1680788402,"update_time":1680788402,"id":1724747297917,"updated_at":"2024-08-27T08:28:17.917000Z","created_at":"2024-08-27T08:28:17.917000Z"}]

At line 346 of prometheus_connect.py we have: response = self._session.get( "{0}/api/v1/query".format(self.url), params={**{"query": query}, **params}, verify=self.ssl_verification, headers=self.headers, ) the params dict, ultimately used to pass params (for instance timeouts) to the...

GiulioGx

Ways to import `PrometheusConnect` without also importing **huge** pandas and matplotlib

[{"_id":"66cd8e1b5a1d13a44707c804","body":"> not only import `PrometheusConnect` itself, but also pandas and possibly matplotlib, which take about 50MB more unnecessary memory when I don't want to use DataFrames and plot them.\r\n\r\nHi @thetaprimeprime, that's an great observation! I did some memory profiling and can confirm that the additional pandas and matplotlib imports do indeed increase the memory usage by about ~45MB.\r\n\r\n> Is there any way to only import `PrometheusConnect` without also importing **huge** pandas and matplotlib?\r\n\r\nAt the moment, I don't think so. But I believe this would be a nice and welcome improvement :smiley: Is this something you'd like to work on, or would you rather someone from our team do it?\r\n\r\nOne way to accomplish this could be to refactor this python module into submodules, something like this:\r\n```bash\r\nprometheus_api_client\r\n\u251c\u2500\u2500 core\r\n\u2502 \u251c\u2500\u2500 __init__.py\r\n\u2502 \u2514\u2500\u2500 prometheus_connect.py\r\n\u251c\u2500\u2500 exceptions\r\n\u2502 \u251c\u2500\u2500 base_exception.py\r\n\u2502 \u2514\u2500\u2500 __init__.py\r\n\u251c\u2500\u2500 __init__.py --> only import core.* here, to avoid importing mpl, pandas\r\n\u251c\u2500\u2500 parsers\r\n\u2502 \u251c\u2500\u2500 __init__.py\r\n\u2502 \u251c\u2500\u2500 metric.py\r\n\u2502 \u251c\u2500\u2500 metric_range_df.py\r\n\u2502 \u251c\u2500\u2500 metrics_list.py\r\n\u2502 \u2514\u2500\u2500 metric_snapshot_df.py\r\n\u2514\u2500\u2500 utils\r\n \u251c\u2500\u2500 datetime_utils.py\r\n \u251c\u2500\u2500 __init__.py\r\n \u2514\u2500\u2500 print_utils.py\r\n```\r\nJust a suggestion off the top of my head, we should explore other ideas as well.\r\n\r\n\/cc @4n4nd ","issue_id":1709750358536,"origin_id":1000405443,"user_origin_id":15005500,"create_time":1640276392,"update_time":1640276392,"id":1724747291108,"updated_at":"2024-08-27T08:28:11.107000Z","created_at":"2024-08-27T08:28:11.107000Z"},{"_id":"66cd8e1b5a1d13a44707c805","body":"Issues go stale after 90d of inactivity.\nMark the issue as fresh with `\/remove-lifecycle stale`.\nStale issues rot after an additional 30d of inactivity and eventually close.\n\nIf this issue is safe to close now please do so with `\/close`.\n\n\/lifecycle stale","issue_id":1709750358536,"origin_id":1076731600,"user_origin_id":33906690,"create_time":1648063232,"update_time":1648063232,"id":1724747291113,"updated_at":"2024-08-27T08:28:11.112000Z","created_at":"2024-08-27T08:28:11.112000Z"},{"_id":"66cd8e1b5a1d13a44707c806","body":"Stale issues rot after 30d of inactivity.\nMark the issue as fresh with `\/remove-lifecycle rotten`.\nRotten issues close after an additional 30d of inactivity.\n\nIf this issue is safe to close now please do so with `\/close`.\n\n\/lifecycle rotten","issue_id":1709750358536,"origin_id":1106836007,"user_origin_id":33906690,"create_time":1650659439,"update_time":1650659439,"id":1724747291117,"updated_at":"2024-08-27T08:28:11.117000Z","created_at":"2024-08-27T08:28:11.117000Z"},{"_id":"66cd8e1b5a1d13a44707c807","body":"Rotten issues close after 30d of inactivity.\nReopen the issue with `\/reopen`.\nMark the issue as fresh with `\/remove-lifecycle rotten`.\n\n\/close","issue_id":1709750358536,"origin_id":1134013893,"user_origin_id":33906690,"create_time":1653261184,"update_time":1653261184,"id":1724747291120,"updated_at":"2024-08-27T08:28:11.120000Z","created_at":"2024-08-27T08:28:11.120000Z"},{"_id":"66cd8e1b5a1d13a44707c808","body":"@sesheta: Closing this issue.\n\n<details>\n\nIn response to [this](https:\/\/github.com\/AICoE\/prometheus-api-client-python\/issues\/225#issuecomment-1134013893):\n\n>Rotten issues close after 30d of inactivity.\n>Reopen the issue with `\/reopen`.\n>Mark the issue as fresh with `\/remove-lifecycle rotten`.\n>\n>\/close\n\n\nInstructions for interacting with me using PR comments are available [here](https:\/\/git.k8s.io\/community\/contributors\/guide\/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes\/test-infra](https:\/\/github.com\/kubernetes\/test-infra\/issues\/new?title=Prow%20issue:) repository.\n<\/details>","issue_id":1709750358536,"origin_id":1134013912,"user_origin_id":33906690,"create_time":1653261188,"update_time":1653261188,"id":1724747291122,"updated_at":"2024-08-27T08:28:11.122000Z","created_at":"2024-08-27T08:28:11.122000Z"},{"_id":"66cd8e1b5a1d13a44707c809","body":"This is also making it hard to use `prometheus-api-client-python` in alpine-based docker images, as both matplotlib & pandas need to build wheels.","issue_id":1709750358536,"origin_id":1486605476,"user_origin_id":11144180,"create_time":1679999313,"update_time":1680768642,"id":1724747291126,"updated_at":"2024-08-27T08:28:11.126000Z","created_at":"2024-08-27T08:28:11.126000Z"},{"_id":"66cd8e1b5a1d13a44707c80a","body":"@Raqbit do you have any suggestions on how we could improve this?","issue_id":1709750358536,"origin_id":1487331998,"user_origin_id":22333506,"create_time":1680024599,"update_time":1680024599,"id":1724747291130,"updated_at":"2024-08-27T08:28:11.130000Z","created_at":"2024-08-27T08:28:11.130000Z"},{"_id":"66cd8e1b5a1d13a44707c80b","body":"> @Raqbit do you have any suggestions on how we could improve this?\r\n\r\nAny way that the pandas & matplotlib dependencies can be avoided with a 'non-development' install would help with this. I'm not very familiar with how this is usually done, but I think [setuptools optional dependencies](https:\/\/setuptools.pypa.io\/en\/latest\/userguide\/dependency_management.html#optional-dependencies) will do the job.","issue_id":1709750358536,"origin_id":1487351797,"user_origin_id":11144180,"create_time":1680025492,"update_time":1680768651,"id":1724747291132,"updated_at":"2024-08-27T08:28:11.132000Z","created_at":"2024-08-27T08:28:11.132000Z"},{"_id":"66cd8e1b5a1d13a44707c80c","body":"> This is also making it hard to use `prometheus-api-client-python` in alpine-based docker images, as both mathplotlib & pandas need to build wheels.\r\n\r\nHi @Raqbit, could you please describe your issue in a bit more detail i.e. why is hard to use in alpine based container images? Are you unable to install these dependencies in the container? Or do they take a longer time to install? Or do they bloat the image?\r\n\r\nBasically, I _think_ OP\u2019s issue here is with slowness of the imports at runtime (which imo can be easily solved by a restructure). Whereas your issue sounds a bit more about installation? (which can't be solved by restructure, and is a bit more involved). So just wanted to get some clarity and prioritize accordingly :)","issue_id":1709750358536,"origin_id":1491282206,"user_origin_id":15005500,"create_time":1680238063,"update_time":1680238063,"id":1724747291135,"updated_at":"2024-08-27T08:28:11.134000Z","created_at":"2024-08-27T08:28:11.134000Z"},{"_id":"66cd8e1b5a1d13a44707c80d","body":"> > This is also making it hard to use `prometheus-api-client-python` in alpine-based docker images, as both mathplotlib & pandas need to build wheels.\r\n> \r\n> Hi @Raqbit, could you please describe your issue in a bit more detail i.e. why is hard to use in alpine based container images? Are you unable to install these dependencies in the container? Or do they take a longer time to install? Or do they bloat the image?\r\n> \r\n> Basically, I _think_ OP\u2019s issue here is with slowness of the imports at runtime (which imo can be easily solved by a restructure). Whereas your issue sounds a bit more about installation? (which can't be solved by restructure, and is a bit more involved). So just wanted to get some clarity and prioritize accordingly :)\r\n\r\nYes. Both `pandas` and `matplotlib` (Plus, `numpy` which is pulled in by `pandas`) have native components written in C. Normally this means that the Python Wheel (pre-compiled binary component) is downloaded and all is well (except for the longer import times mentioned by OP). For Alpine Linux, however, there is no such wheel available as Alpine uses the `musl` C standard library implementation instead of the more common glibc. This, in turn, will cause Pip to try to compile the C code for these native components during the installation process as it is unable to find suitable wheels in the package from pypi.\r\n\r\nThis requires a complete c compiler toolchain to be available, and in my case, was taking 20+ minutes to complete (minutes of compiling C-code with all available cores, making the computer unusable for other tasks). \r\n\r\nThe bloat of the compiler toolchain can be avoided by uninstalling them after running `pip install`, but the compilation step cannot be avoided without using a different container base-image such as debian.","issue_id":1709750358536,"origin_id":1491663522,"user_origin_id":11144180,"create_time":1680256809,"update_time":1680768677,"id":1724747291138,"updated_at":"2024-08-27T08:28:11.138000Z","created_at":"2024-08-27T08:28:11.138000Z"},{"_id":"66cd8e1b5a1d13a44707c80e","body":"> Yes. Both `pandas` and `mathplotlib` (Plus, `numpy` which is pulled in by `pandas`) have native components written in C. Normally this means that the Python Wheel (pre-compiled binary component) is downloaded and all is well (except for the longer import times mentioned by OP). For Alpine Linux, however, there is no such wheel available as Alpine uses the `musl` C standard library implementation instead of the more common glibc. This, in turn, will cause Pip to try to compile the C code for these native components during the installation process as it is unable to find suitable wheels in the package from pypi.\r\n> \r\n> This requires a complete c compiler toolchain to be available, and in my case, was taking 20+ minutes to complete (minutes of compiling C-code with all available cores, making the computer unusable for other tasks).\r\n> \r\n> The bloat of the compiler toolchain can be avoided by uninstalling them after running `pip install`, but the compilation step cannot be avoided without using a different container base-image such as debian.\r\n\r\nGot it, thanks for the details! In theory yes we could set up installation so that `pip install prometheus-api-client-python` just installs the \"core\" library components (`PrometheusConnect`). And `pandas` and `matplotlib` can be listed as dependencies in extras_require to be installed with something like `pip install prometheus-api-client-python[full]`, installing the other componenets (`Metric`, `MetricSnapshotDataFrame`, etc). Note that `numpy` is still required by `PrometheusConnect`, so it'd require additional changes to move that to extras_require.\r\n\r\nHowever, wouldn't doing so potentially break installations for existing users? I'm in favor of this change if it causes minimal disruption for users while maximizing the benefit. Or if we can map out a rollout procedure to ensure so. But I currently don't have any supporting artifacts to form an opinion on this. Maybe @4n4nd can weigh in here as well?","issue_id":1709750358536,"origin_id":1498440881,"user_origin_id":15005500,"create_time":1680751616,"update_time":1680751616,"id":1724747291141,"updated_at":"2024-08-27T08:28:11.141000Z","created_at":"2024-08-27T08:28:11.141000Z"}]

**Is your feature request related to a problem? Please describe.** I found this simple import ```python from prometheus_api_client import PrometheusConnect ``` not only import `PrometheusConnect` itself, but also pandas and...

buttonfly1000