kr8s icon indicating copy to clipboard operation
kr8s copied to clipboard

Make getting resource by name more efficient by using a field selector

Open catdog2 opened this issue 1 year ago • 1 comments

In case you have a large number of resources getting a resource by name can get very inefficient, esp in terms of memory usage. Currently all resources are retrieved, then filtered for the name on the client side. This behavior is also very surprising for a function taking a single name.

catdog2 avatar Jul 21 '24 10:07 catdog2

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.03%. Comparing base (87063fc) to head (15b1a76). Report is 144 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #452      +/-   ##
==========================================
+ Coverage   94.61%   95.03%   +0.41%     
==========================================
  Files          29       30       +1     
  Lines        3141     3867     +726     
==========================================
+ Hits         2972     3675     +703     
- Misses        169      192      +23     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jul 21 '24 10:07 codecov[bot]

There there hasn't been any activity on this issue I'm going to merge it in and open an issue to track the improvements to Api.get().

jacobtomlinson avatar Sep 09 '24 13:09 jacobtomlinson

I'm a bit confused here. Why do we need to use a field selector vs. just using the direct GET by URL?

For example:

GET /api/v1/namespaces/{namespace}/configmaps/{name}

This appears to be what .exists() and .refresh() do.

sirianni avatar May 31 '25 15:05 sirianni

@sirianni I don't see any problem with changing the implementation to do it that way if you want to make a PR. Is there some problem you're running into because of the implementation here? If so could I ask you to open a new issue.

jacobtomlinson avatar Jun 02 '25 09:06 jacobtomlinson

@sirianni I don't see any problem with changing the implementation to do it that way if you want to make a PR. Is there some problem you're running into because of the implementation here? If so could I ask you to open a new issue.

Thanks for the reply. I did not see a correctness problem, but I was surprised to see in my httpx logs multiple GET calls issued for a single Object.get(). It seemed inefficient so I was curious if there was a motivation for this behavior.

sirianni avatar Jun 02 '25 13:06 sirianni

@sirianni I wouldn't expect to see multiple calls for a single resource. Only for multiple resources. Could you open an issue with steps you took to reproduce this so I can take a look?

jacobtomlinson avatar Jun 02 '25 16:06 jacobtomlinson

@sirianni I wouldn't expect to see multiple calls for a single resource. Only for multiple resources. Could you open an issue with steps you took to reproduce this so I can take a look?

Ack. This was with an older version of my implementation that has since been refactored, but I will try to reproduce the behavior.

sirianni avatar Jun 02 '25 18:06 sirianni