marqo icon indicating copy to clipboard operation
marqo copied to clipboard

feat: add size to index-stats API response

Open hmacr opened this issue 1 year ago • 2 comments

  • What kind of change does this PR introduce? (Bug fix, feature, docs update, ...) feature

  • What is the current behavior? (You can also link to an open issue here) https://github.com/marqo-ai/marqo/issues/374

  • What is the new behavior (if this is a feature change)? Index size will be returned as a field named size in the index-stats API response

  • Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?) No

  • Have unit tests been run against this PR? (Has there also been any additional testing?) Yes

  • Related Python client changes (link commit/PR here) There is no index/stats client method exposed in py-marqo. Do you want it to be added as part of this change?

  • Related documentation changes (link commit/PR here)

  • Other information:

  • Please check if the PR fulfills these requirements

  • [x] The commit message follows our guidelines
  • [x] Tests for the changes have been added (for bug fixes/features)
  • [ ] Docs have been added / updated (for bug fixes / features)

hmacr avatar May 26 '23 07:05 hmacr

Hi @hmacr ! Thanks for the PR!

  1. Is it possible to rework this using the OpenSearch stats endpoint?
GET /{index}/_stats

indices.{index}.total.store.size_in_bytes
  1. Can you add some a unit test for this please? See here for the existing get_stats tests https://github.com/marqo-ai/marqo/blob/mainline/tests/tensor_search/test_get_stats.py

  2. Can you run please run tests. Read here for how to do that: https://github.com/marqo-ai/marqo/blob/d39f2312e9e289b0c54f49e14cddd49095808ad4/CONTRIBUTING.md#unit-tests You can run just the tensor_search tests, which are faster and use less resources, with the following: tox -- tests/tensor_search If you are having a lot of trouble running tests, I can create branch in our repo from your fork and run our pipelines against it, and pass you the feedback

pandu-k avatar May 31 '23 00:05 pandu-k

@pandu-k Apologies, I understand it's been a long time since I created the pull request; I'm trying to be more active on open source these days.

About the review comments, I've addressed them and ran the unit-tests as well (attaching the screenshots; there was one failure due to an unavailable remote repo). Please let me know your feedback.

Screenshot 2023-07-26 at 8 03 46 PM

Screenshot 2023-07-26 at 8 04 14 PM

hmacr avatar Jul 26 '23 15:07 hmacr