modal-client icon indicating copy to clipboard operation
modal-client copied to clipboard

[WRK-814] Client side impl Sandbox.resource_usage()

Open thundergolfer opened this issue 9 months ago • 5 comments

Describe your changes

  • WRK-814

Check these boxes or delete any item (or this section) if not relevant for this PR.

  • [x] Client+Server: this change is compatible with old servers
  • [x] Client forward compatibility: this change ensures client can accept data intended for later versions of itself

Note on protobuf: protobuf message changes in one place may have impact to multiple entities (client, server, worker, database). See points above.


Changelog

  • Adds new Sandbox.resource_usage() method to retrieve billable resource usage (CPU, RAM, GPU time).

thundergolfer avatar Mar 21 '25 05:03 thundergolfer

LGTM. It might be good to clarify in the comments/docstring that the returned CPU and memory values are max(used, requested)

oh missed that as I haven’t looked closely at the backend pr, but would argue that maybe suggests a different name, not just documentation (which is also good to add).

mwaskom avatar Mar 25 '25 22:03 mwaskom

I think different people might really want to know both though (what did it actually use, eg for profiling, and what did we bill for, eg for passing on costs).

mwaskom avatar Mar 25 '25 22:03 mwaskom

And ideally we wouldn’t expose multiple variants of similar information through multiple different ad hoc methods :/. Maybe an as_billed: bool parameter or something?

mwaskom avatar Mar 25 '25 22:03 mwaskom

Maybe an as_billed: bool parameter or something?

Like resource_usage(raw: bool = True)? If raw==True then pass through the usage, otherwise apply the reservation and regional modifiers.

Hmm I don't love it. I didn't realize that we applied compute resource multipliers because of region scheduling. I'd otherwise be OK with passing through the reserved value as-is, because I don't think this endpoint can ever serve as profiling data.

I'll leave this for a bit and think about it.

thundergolfer avatar Mar 26 '25 01:03 thundergolfer

Like resource_usage(raw: bool = True)? If raw==True then pass through the usage, otherwise apply the reservation and regional modifiers.

I am just suggesting that we allow users to query used or max(used, reserved). I don't think this endpoint should reflect any modifiers which I see as a sort of implementation detail.

I'd otherwise be OK with passing through the reserved value as-is,

If you do that IMO I'd say it's mandatory to name the method differently; how can we describe the billing as applying to max(used, reserved) and then call the resulting value "usage"?

I don't think this endpoint can ever serve as profiling data

Why not? I'd certainly try to use it for something like that based on how it's currently presented, and I expect other users would too.

mwaskom avatar Mar 26 '25 11:03 mwaskom