ondemand icon indicating copy to clipboard operation
ondemand copied to clipboard

Add memory request to interactive jobs card

Open johrstrom opened this issue 4 years ago • 3 comments

It came up in this discourse topic that it may be nice to add memory allocations to the interactive jobs cards.

https://discourse.osc.edu/t/ood-job-memory-reporting/977

So an example of the card may include memory like so (I'm not sure about the ordering here, it's just for example).

image

johrstrom avatar Mar 25 '21 15:03 johrstrom

I was looking into this and had a few questions:

  1. What command should we be using for Slurm? sacct with MaxRSS or sstat? sstat seems expensive to call repeatedly (unless I’m mistaken), so sacct seems like the better choice.
  2. Where in the adapter should this go? The existing methods in Job::Adapters::Slurm::Batch (submit_string, get_jobs) already do more than their names suggest, and adding more logic to them would just increase the mess. Should we add a separate method for fetching memory usage instead?
  3. Do we need this for other adapters? Or is this a Slurm-only feature for now?
  4. Is there a better way to do this that I’m missing? This feels like it needs more design than the issue description suggests.

Most of this is backend logic, so that’s where my questions are focused at the moment. Working off these docs to guide me: https://slurm.schedmd.com/sacct.html

Oglopf avatar Feb 26 '25 01:02 Oglopf

At this point in the code you're dealing with an OodCore::Job::Info object. Looking at that object, we don't have a direct way to get memory, so you can pull it from native like the activejobs extended panel does.

https://github.com/OSC/ondemand/blob/9e1727f31d93db4ca41e7a4c4102c7508fe32d26/apps/dashboard/app/models/active_jobs/jobstatusdata.rb#L122

Note that you're looking at min_memory while we have the competing #1006 to use max memory instead.

johrstrom avatar Feb 26 '25 19:02 johrstrom

This needs an ood_core update and the way Slurm reports memory is funky. Although this is a great issue, we are bumping it to the next release due to to time constraints with 4.1

moffatsadeghi avatar Sep 25 '25 20:09 moffatsadeghi