benchpark icon indicating copy to clipboard operation
benchpark copied to clipboard

contribution model

Open pearce8 opened this issue 1 year ago • 2 comments

  1. Need to record maintainers: for systems (one per system config, e.g., nosite-x86_64), and experiments (one per experiment, e.g., amg2023/cuda)
  2. New systems
  • system owners should define/update the default compilers on the system
  • dashboard report of all the experiments that were tested
  1. New experiments
  • dashboard report of all the platforms tested
  • should we require a working test on a neutral platform (e.g., AWS)?
  1. Spack package.py and Ramble application.py
  • benchpark/repo is the way to test all of the components together
  • should commits to the repo trigger PRs to Spack and Ramble? Once those are upstreamed, changes to benchpark/repo need to be made, and ${BENCHMARK1} should point at the upstreamed versions instead of at the repo.

pearce8 avatar Oct 17 '23 20:10 pearce8

Meanwhile, the users may well prefer to keep their local changes in system and benchmark configs. This may introduce issues if the interfaces of either of the project change, but otherwise may be a desirable usage pattern for our users.

In my first experiments I realized that some benchmark config files (like tpl templates) may depend on/relate to machine configs which adds an additional complication.

pszi1ard avatar Feb 19 '24 17:02 pszi1ard

@pszi1ard interesting, can provide an example of a template you've modified for a specific system?

alecbcs avatar Feb 19 '24 17:02 alecbcs