Parameterized system variables

[pearce8]

Options for enabling different values for a variable within a single system config, such as different number of cores or memory per node in AWS instances. Brainstorming notes from discussion with Doug below.

Option 1:

variables.yaml

variables:
  aws_names: [full, half, quarter]
  sys_cpus_per_node: [44, 22, 11]
  sys_gpus_per_node [4, 2, 1]

zips.yaml

zips:
  aws_conf:
  - aws_names
  - sys_cpus_per_node
  - sys_gpus_per_node

ramble workspace setup --where '"{aws_names}" == "full"'

Option 2:

variables.yaml

variables:
  sys_cpus_per_node: 44
  sys_gpus_per_node 4

Should work

ramble -c variables:sys_cpus_per_node:22 -c variables:sys_gpus_per_node:2 workspace setup

Only works when they are part of a config scope.

Config scopes are listed in ramble config list

Anything within another config scope won't be merged by the config logic

i.e. command line scopes won't override variables defined in

ramble:applications:hostname:variables as they are in a different config section

section. And merging happens after the config scopes are merged.

ramble.yaml

ramble:
  variables: <-- Will work
  applications:
    hostname:
      variables: <-- Won't work
      workloads:
        local:
          variables: <-- Won't work
          experiments:
            test:
              variables:
                sys_cpus_per_node: 44
                sys_gpus_per_node 4

Won't work

ramble -c variables:sys_cpus_per_node:22 -c variables:sys_gpus_per_node:2 workspace setup