kokkos-tools
kokkos-tools copied to clipboard
Published
20 hours ago •
kokkos
kokkos
NGA FY22: improve build system
- [x] Develop CMake build system to build all the KokkosTools
- [x] Extent the tools cmake build system to optionally build Caliper and Apex (i.e. translate typical Kokkos options such as Kokkos_ENABLE_CUDA to the respective settings for Caliper and Apex)
- [x] Add option to build all tools as a single library, and add an initialization interface to that library, which uses the existing KokkosEventSet capability to load a specific tool in the library upon being called with the corresponding option (e.g. KokkosTools::initialize("space_time_stack"));
See status and detailed TODO on related PRs:
- [in-progress] https://github.com/kokkos/kokkos-tools/pull/131;
- [done] Makefile fixes:
- https://github.com/kokkos/kokkos-tools/pull/129;
- https://github.com/kokkos/kokkos-tools/pull/130;
@MikolajZuzek with respect to timemory, this should work within a cmake build system (as soon as timemory/PR#232 is merged):
# general setup
set(TIMEMORY_INSTALL_HEADERS OFF CACHE BOOL "Disable timemory header install" FORCE)
set(TIMEMORY_INSTALL_CONFIG OFF CACHE BOOL "Disable timemory cmake configuration install" FORCE)
set(TIMEMORY_INSTALL_ALL OFF CACHE BOOL "Disable install target depending on all target" FORCE)
set(TIMEMORY_BUILD_TOOLS OFF CACHE BOOL "Ensure timem executable is built" FORCE)
set(TIMEMORY_BUILD_EXCLUDE_FROM_ALL ON CACHE BOOL "Set timemory to only build dependencies" FORCE)
set(TIMEMORY_QUIET_CONFIG ON CACHE BOOL "Make timemory configuration quieter" FORCE)
# setting this to off will enable auto-discovery of third-party capabilities
option(TIMEMORY_REQUIRE_PACKAGES "" OFF)
# build basic kokkos support
option(TIMEMORY_BUILD_KOKKOS_TOOLS "" ON)
# build pre-configured kokkos-tools libraries
option(TIMEMORY_BUILD_KOKKOS_CONFIG "" ON)
# build kokkos application (requires Kokkos install) -- this will generate a bunch of CTests
option(TIMEMORY_BUILD_KOKKOS_SAMPLE "" ON)
# assumes the repo is cloned
add_subdirectory(timemory)
# dummy build target
add_custom_target(kokkos-tools-timemory ALL)
# timemory-kokkos-tools is dummy target which "building" will build all kokkos-tools
add_dependencies(kokkos-tools-timemory timemory-kokkos-tools)
Depending on the auto-discovery, you will have a different set of libraries but the above will have something like:
In the following, the 3rd party capabilities enabled are PAPI, gperftools, Caliper, LIKWID, OMPT
$ ls -1 <build-dir>/timemory/source/tools/kokkos-connector/kokkos-tools/
kp_timemory_caliper.so
kp_timemory_context_switch.so
kp_timemory_cpu_branching.so
kp_timemory_cpu_branching_wall_rate.so
kp_timemory_cpu_cache_misses.so
kp_timemory_cpu_cache_misses_wall_rate.so
kp_timemory_cpu_flops.so
kp_timemory_cpu_flops_wall_rate.so
kp_timemory_cpu_instructions.so
kp_timemory_cpu_instructions_wall_rate.so
kp_timemory_cpu_l1_misses.so
kp_timemory_cpu_l1_misses_wall_rate.so
kp_timemory_cpu_l1_total.so
kp_timemory_cpu_l1_total_wall_rate.so
kp_timemory_cpu_l2_misses.so
kp_timemory_cpu_l2_misses_wall_rate.so
kp_timemory_cpu_l2_total.so
kp_timemory_cpu_l2_total_wall_rate.so
kp_timemory_cpu_l3_misses.so
kp_timemory_cpu_l3_misses_wall_rate.so
kp_timemory_cpu_l3_total.so
kp_timemory_cpu_l3_total_wall_rate.so
kp_timemory_cpu_roofline.so
kp_timemory_cpu_stalls.so
kp_timemory_cpu_stalls_wall_rate.so
kp_timemory_cpu_tlb.so
kp_timemory_cpu_tlb_wall_rate.so
kp_timemory_filter.so
kp_timemory_gperftools.so
kp_timemory_likwid.so
kp_timemory_memory.so
kp_timemory_ompt.so
kp_timemory_papi.so
kp_timemory.so
kp_timemory_timers_memory.so
kp_timemory_timers.so
kp_timemory_trip_count.so
The testing workflow would thus be (after everything is built):
$ ctest -R 'kokkos-connector-test'
Test project /home/jrmadsen/devel/c++/timemory/timemory/build-timemory/Linux
Start 7: kokkos-connector-test-libtimemory
1/38 Test #7: kokkos-connector-test-libtimemory .............................. Passed 0.39 sec
Start 8: kokkos-connector-test-kp-timemory
2/38 Test #8: kokkos-connector-test-kp-timemory .............................. Passed 0.44 sec
Start 9: kokkos-connector-test-kp-timemory-filter
3/38 Test #9: kokkos-connector-test-kp-timemory-filter ....................... Passed 0.43 sec
Start 10: kokkos-connector-test-kp-timemory-trip-count
4/38 Test #10: kokkos-connector-test-kp-timemory-trip-count ................... Passed 0.39 sec
Start 11: kokkos-connector-test-kp-timemory-timers
5/38 Test #11: kokkos-connector-test-kp-timemory-timers ....................... Passed 0.43 sec
Start 12: kokkos-connector-test-kp-timemory-memory
6/38 Test #12: kokkos-connector-test-kp-timemory-memory ....................... Passed 0.42 sec
Start 13: kokkos-connector-test-kp-timemory-timers-memory
7/38 Test #13: kokkos-connector-test-kp-timemory-timers-memory ................ Passed 0.43 sec
Start 14: kokkos-connector-test-kp-timemory-context-switch
8/38 Test #14: kokkos-connector-test-kp-timemory-context-switch ............... Passed 0.38 sec
Start 15: kokkos-connector-test-kp-timemory-gperftools
9/38 Test #15: kokkos-connector-test-kp-timemory-gperftools ................... Passed 0.39 sec
Start 16: kokkos-connector-test-kp-timemory-papi
10/38 Test #16: kokkos-connector-test-kp-timemory-papi ......................... Passed 0.41 sec
Start 17: kokkos-connector-test-kp-timemory-cpu-roofline
11/38 Test #17: kokkos-connector-test-kp-timemory-cpu-roofline ................. Passed 18.13 sec
Start 18: kokkos-connector-test-kp-timemory-cpu-flops
12/38 Test #18: kokkos-connector-test-kp-timemory-cpu-flops .................... Passed 0.43 sec
Start 19: kokkos-connector-test-kp-timemory-cpu-flops-wall-rate
13/38 Test #19: kokkos-connector-test-kp-timemory-cpu-flops-wall-rate .......... Passed 0.42 sec
Start 20: kokkos-connector-test-kp-timemory-cpu-instructions
14/38 Test #20: kokkos-connector-test-kp-timemory-cpu-instructions ............. Passed 0.45 sec
Start 21: kokkos-connector-test-kp-timemory-cpu-instructions-wall-rate
15/38 Test #21: kokkos-connector-test-kp-timemory-cpu-instructions-wall-rate ... Passed 0.40 sec
Start 22: kokkos-connector-test-kp-timemory-cpu-cache-misses
16/38 Test #22: kokkos-connector-test-kp-timemory-cpu-cache-misses ............. Passed 0.44 sec
Start 23: kokkos-connector-test-kp-timemory-cpu-cache-misses-wall-rate
17/38 Test #23: kokkos-connector-test-kp-timemory-cpu-cache-misses-wall-rate ... Passed 0.44 sec
Start 24: kokkos-connector-test-kp-timemory-cpu-tlb
18/38 Test #24: kokkos-connector-test-kp-timemory-cpu-tlb ...................... Passed 0.39 sec
Start 25: kokkos-connector-test-kp-timemory-cpu-tlb-wall-rate
19/38 Test #25: kokkos-connector-test-kp-timemory-cpu-tlb-wall-rate ............ Passed 0.40 sec
Start 26: kokkos-connector-test-kp-timemory-cpu-branching
20/38 Test #26: kokkos-connector-test-kp-timemory-cpu-branching ................ Passed 0.45 sec
Start 27: kokkos-connector-test-kp-timemory-cpu-branching-wall-rate
21/38 Test #27: kokkos-connector-test-kp-timemory-cpu-branching-wall-rate ...... Passed 0.40 sec
Start 28: kokkos-connector-test-kp-timemory-cpu-stalls
22/38 Test #28: kokkos-connector-test-kp-timemory-cpu-stalls ................... Passed 0.42 sec
Start 29: kokkos-connector-test-kp-timemory-cpu-stalls-wall-rate
23/38 Test #29: kokkos-connector-test-kp-timemory-cpu-stalls-wall-rate ......... Passed 0.45 sec
Start 30: kokkos-connector-test-kp-timemory-cpu-l1-misses
24/38 Test #30: kokkos-connector-test-kp-timemory-cpu-l1-misses ................ Passed 0.45 sec
Start 31: kokkos-connector-test-kp-timemory-cpu-l1-misses-wall-rate
25/38 Test #31: kokkos-connector-test-kp-timemory-cpu-l1-misses-wall-rate ...... Passed 0.41 sec
Start 32: kokkos-connector-test-kp-timemory-cpu-l2-misses
26/38 Test #32: kokkos-connector-test-kp-timemory-cpu-l2-misses ................ Passed 0.40 sec
Start 33: kokkos-connector-test-kp-timemory-cpu-l2-misses-wall-rate
27/38 Test #33: kokkos-connector-test-kp-timemory-cpu-l2-misses-wall-rate ...... Passed 0.40 sec
Start 34: kokkos-connector-test-kp-timemory-cpu-l3-misses
28/38 Test #34: kokkos-connector-test-kp-timemory-cpu-l3-misses ................ Passed 0.43 sec
Start 35: kokkos-connector-test-kp-timemory-cpu-l3-misses-wall-rate
29/38 Test #35: kokkos-connector-test-kp-timemory-cpu-l3-misses-wall-rate ...... Passed 0.46 sec
Start 36: kokkos-connector-test-kp-timemory-cpu-l1-total
30/38 Test #36: kokkos-connector-test-kp-timemory-cpu-l1-total ................. Passed 0.43 sec
Start 37: kokkos-connector-test-kp-timemory-cpu-l1-total-wall-rate
31/38 Test #37: kokkos-connector-test-kp-timemory-cpu-l1-total-wall-rate ....... Passed 0.40 sec
Start 38: kokkos-connector-test-kp-timemory-cpu-l2-total
32/38 Test #38: kokkos-connector-test-kp-timemory-cpu-l2-total ................. Passed 0.45 sec
Start 39: kokkos-connector-test-kp-timemory-cpu-l2-total-wall-rate
33/38 Test #39: kokkos-connector-test-kp-timemory-cpu-l2-total-wall-rate ....... Passed 0.40 sec
Start 40: kokkos-connector-test-kp-timemory-cpu-l3-total
34/38 Test #40: kokkos-connector-test-kp-timemory-cpu-l3-total ................. Passed 0.43 sec
Start 41: kokkos-connector-test-kp-timemory-cpu-l3-total-wall-rate
35/38 Test #41: kokkos-connector-test-kp-timemory-cpu-l3-total-wall-rate ....... Passed 0.46 sec
Start 42: kokkos-connector-test-kp-timemory-caliper
36/38 Test #42: kokkos-connector-test-kp-timemory-caliper ...................... Passed 0.42 sec
Start 43: kokkos-connector-test-kp-timemory-likwid
37/38 Test #43: kokkos-connector-test-kp-timemory-likwid ....................... Passed 0.40 sec
Start 44: kokkos-connector-test-kp-timemory-ompt
38/38 Test #44: kokkos-connector-test-kp-timemory-ompt ......................... Passed 0.40 sec
100% tests passed, 0 tests failed out of 38
Label Time Summary:
kokkos = 67.35 sec*proc (38 tests)
timemory = 67.35 sec*proc (38 tests)
Total Test time (real) = 33.73 sec
