ldms plugin job_id difficulties

[morrone]

Currently, it is a requirement that all ldms plugins implement a job_id option and store the job_id in every metric set.

The first issue is that not all plugins will be running on components where a job_id makes sense. With some plugins, it may be obvious that a job would never be running on the nodes, and with others the plugin might run on nodes that run jobs, and on others it might run on components that do not run jobs. With the former, we might just relax the rules and allow them to not implement job_id. With the latter, we still have an issue, although perhaps just designating "0" in the job_id field to mean "no job" would be fine. But since job_id is a compile time option, and few people will want different builds for different nodes in the same center, the job_id will stay around uselessly.

Another issue with the current job_id implementation is that it seems to assume that only a single job is running on a node at a time. This is not the case on some of our clusters. So job_id is going to be insufficient there.

job_id is an integer, and that might be overly restrictive for some job managers. Switching to a string might make sense.

Finally, the all-macro implementation of the helpers that all plugins are required to use make too many assumptions about the plugin's internal implementation. It would be very nice to move to a proper function-based API for the implementation, which by its nature would probably eliminate the issues that the macro implementation has.

[baallan]

this is addressed with the sampler_base and job_info in v4. we don't intend to change it in v3, I believe

[morrone]

Maybe, but sampler_base comes with its own baggage issues. Too much in one API that isn't generally applicable to all plugins.

[baallan]

true but much easier to fix