benchpark
benchpark copied to clipboard
[WIP] Refactor Command to Avoid Checking for Help and Version
Description
- [ ] Replace with: A short description of the change, including motivation and context.
- [ ] Depends on #784
- [ ] Replace with: Link(s) to relevant issue(s)
- [ ] Complete the checklist for a relevant section(s) below
- [ ] Delete sections below that are not relevant to this PR
Adding/modifying a system (docs: Adding a System)
- [ ] Add/modify
systems/system_name/system.pyfile - [ ] Add/modify a dry run unit test for
system_namein.github/workflows/run.yml - [ ] Add/modify
systems/all_hardware_descriptions/hardware_name/hardware_description.yamlwhich will appear in the docs catalogue
Adding/modifying a benchmark (docs: Adding a Benchmark)
- [ ] If modifying the source code of a benchmark: create, self-assign, and link here a follow up issue with a link to the PR in the benchmark repo.
- [ ] If package.py upstreamed to Spack is insufficient, add/modify
repo/benchmark_name/package.pyplus: create, self-assign, and link here a follow up issue with a link to the PR in the Spack repo. - [ ] If application.py upstreamed to Ramble is insufficient, add/modify
repo/benchmark_name/application.pyplus: create, self-assign, and link here a follow up issue with a link to the PR in the Ramble repo. - [ ] Tags in Ramble's
application.pyor inrepo/benchmark_name/application.pywill appear in the docs catalogue - [ ] Add/modify an
experiments/benchmark_name/experiment.pyto define a single node and multi-node experiments - [ ] Add/modify a dry run unit test in
.github/workflows/run.yml
Adding/modifying core functionality, CI, or documentation:
- [ ] Update docs
- [ ] Update
.github/workflowsand.gitlab/testsunit tests (if needed)
I wanted to just add a few options that @michaelmckinsey1 and I discussed relating to this. The 3 options we discussed (and the 3 most viable options besides what @michaelmckinsey1 has already done and without getting into some really complex and esoteric stuff IMO) are:
- Split the
setup_parserfunctions for each command into a separate file(s) so that Benchpark imports that are not needed for setting up argparse are not imported. Then, dynamically load only thecommandfunction that we need for the selected subcommand inmain.py - Require command developers to put all their Benchpark imports into the
commandfunction - Add a decorator similar to the example below that will auto import symbols and/or modules and inject them into the decorated function. Although I worked out the example out of curiosity, I really would NOT recommend this approach
from functools import wraps
from importlib import import_module
# Each path string takes the form of "module[:sym0[,sym1[,sym2...]]]"
def benchpark_imports(*import_path_strs):
def inner_decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
# Loop over the path strings provided to the decorator
for path_str in import_path_strs:
# Split the path string into the module name and symbols to import (if provided)
module_name_and_imports = path_str.split(":")
# If both module name and symbols to import were provided, ...
if len(module_name_and_imports) == 2:
# Import the module with importlib
mod = import_module(module_name_and_imports[0])
# Split the symbols to import into individual names
syms_to_import = module_name_and_imports[1].split(",")
# For each symbol to import, get the symbol from the loaded module.
# Then, save it into the decorated function's __builtins__.
# This entire process is equivalent to running the following inside the decorated function:
# >> from <module_name> import <sym0>, <sym1>, <sym2>, ...
for sym in syms_to_import:
func.__builtins__[sym] = getattr(mod, sym)
# Else, if only the module name was provided, ...
elif len(module_name_and_imports) == 1:
# Use built_mod to iteratively build the module name up
built_mod = []
# Split the module name into individual subpackages/submodules
for partial_mod in module_name_and_imports[0].split("."):
# Build the current module/package name to import
built_mod.append(partial_mod)
curr_mod = ".".join("built_mod)
# Import the current module/package
pmod = import_module(curr_mod)
# Save the current module/package into the decorated function's __builtins__.
# This entire process is equivalent to something like:
# >> import os
# >> import os.path
func.__builtins__[curr_mod] = pmod
# Otherwise, the user input is invalid
else:
raise ValueError(
"Invalid path/import specifier: {}".format(path_str)
)
# Call the actual decorated function
return func(*args, **kwargs)
return wrapper
return inner_decorator
# Using the decorator here is equivalent to putting the following
# inside of the main function:
# >> from test_print import test
@benchpark_imports("test_print:test")
def main():
print("Hit main")
test()