vt icon indicating copy to clipboard operation
vt copied to clipboard

1362 eti for collections

Open lifflander opened this issue 4 years ago • 21 comments

Fixes #1362

lifflander avatar Apr 15 '21 22:04 lifflander

PR tests (nvidia cuda 10.1, ubuntu, mpich)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

Compilation - successful

Testing - passed

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (clang-5.0, ubuntu, mpich)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

Compilation - successful

Testing - passed

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (clang-3.9, ubuntu, mpich)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

Compilation - successful

Testing - passed

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

Without digging into what git diff output shows directly, is the code in the new location.routing.impl.h strictly cut and pasted from location.impl.h, or are there additional modifications in that same commit? I saw the modification made in separate commits.

PhilMiller avatar Apr 15 '21 22:04 PhilMiller

PR tests (gcc-6, ubuntu, mpich)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

Compilation - successful

Testing - passed

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (gcc-5, ubuntu, mpich)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7



The following tests FAILED:
  153 - vt:*/TestNodeStatsDumper.test_node_stats_dumping_with_interval/*_proc_2 (Failed)

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (gcc-10, ubuntu, openmpi, no LB)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

Compilation - successful

Testing - passed

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (gcc-9, ubuntu, mpich, zoltan)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

Compilation - successful

Testing - passed

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (gcc-8, ubuntu, mpich, address sanitizer)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

Compilation - successful

Testing - passed

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (gcc-7, ubuntu, mpich, trace runtime, LB)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

FAILED: tests/index_nompi 
: && /usr/bin/g++ -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -ftemplate-backtrace-limit=100 -Werror -O3 -DNDEBUG --coverage -rdynamic -Wl,-rpath -Wl,/usr/local/lib -Wl,--enable-new-dtags tests/CMakeFiles/index_nompi.dir/Unity/unity_0_cxx.cxx.o -o tests/index_nompi  -Wl,-rpath,/usr/local/lib  lib/libfort/lib/libfort.a  lib/libgtest.a  src/libvt-release.a  -ldl  /usr/lib/x86_64-linux-gnu/libz.so  lib/fmt/libfmt.a  /build/checkpoint/install/lib/libcheckpoint.a  -lpthread  lib/libfort/lib/libfort.a  /usr/local/lib/libmpicxx.so  /usr/local/lib/libmpi.so && :
collect2: error: ld returned 1 exit status
FAILED: tests/CMakeFiles/pipe_basic.dir/Unity/unity_0_cxx.cxx.o 
/usr/bin/ccache /usr/bin/g++  -I/vt/tests/unit -I/vt/lib/CLI -I/vt/lib/libfort/lib -Irelease -I/vt/src -isystem /vt/tests/extern/googletest/googletest/include -isystem /vt/tests/extern/googletest/googletest -isystem /vt/lib/fmt/include -isystem /build/checkpoint/install/include -isystem /build/detector/install/include -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -ftemplate-backtrace-limit=100 -Werror -O3 -DNDEBUG -fdiagnostics-color=always -O0 -g --coverage -fPIC -std=c++14 -MD -MT tests/CMakeFiles/pipe_basic.dir/Unity/unity_0_cxx.cxx.o -MF tests/CMakeFiles/pipe_basic.dir/Unity/unity_0_cxx.cxx.o.d -o tests/CMakeFiles/pipe_basic.dir/Unity/unity_0_cxx.cxx.o -c tests/CMakeFiles/pipe_basic.dir/Unity/unity_0_cxx.cxx
In file included from tests/CMakeFiles/pipe_basic.dir/Unity/unity_0_cxx.cxx:8:0:
/vt/tests/unit/pipe/test_signal_cleanup.cc:130:3: fatal error: error writing to /tmp/ccJvMUcL.s: No space left on device
 }}} // end namespace vt::tests::unit
   ^
compilation terminated.
FAILED: tests/sequencer_extended 
: && /usr/bin/g++ -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -ftemplate-backtrace-limit=100 -Werror -O3 -DNDEBUG --coverage -rdynamic -Wl,-rpath -Wl,/usr/local/lib -Wl,--enable-new-dtags tests/CMakeFiles/sequencer_extended.dir/Unity/unity_0_cxx.cxx.o -o tests/sequencer_extended  -Wl,-rpath,/usr/local/lib  lib/libfort/lib/libfort.a  lib/libgtest.a  src/libvt-release.a  -ldl  /usr/lib/x86_64-linux-gnu/libz.so  lib/fmt/libfmt.a  /build/checkpoint/install/lib/libcheckpoint.a  -lpthread  lib/libfort/lib/libfort.a  /usr/local/lib/libmpicxx.so  /usr/local/lib/libmpi.so && :
collect2: error: ld returned 1 exit status
FAILED: tests/active_basic 
: && /usr/bin/g++ -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -ftemplate-backtrace-limit=100 -Werror -O3 -DNDEBUG --coverage -rdynamic -Wl,-rpath -Wl,/usr/local/lib -Wl,--enable-new-dtags tests/CMakeFiles/active_basic.dir/Unity/unity_1_cxx.cxx.o tests/CMakeFiles/active_basic.dir/Unity/unity_0_cxx.cxx.o -o tests/active_basic  -Wl,-rpath,/usr/local/lib  lib/libfort/lib/libfort.a  lib/libgtest.a  src/libvt-release.a  -ldl  /usr/lib/x86_64-linux-gnu/libz.so  lib/fmt/libfmt.a  /build/checkpoint/install/lib/libcheckpoint.a  -lpthread  lib/libfort/lib/libfort.a  /usr/local/lib/libmpicxx.so  /usr/local/lib/libmpi.so && :
collect2: error: ld returned 1 exit status
FAILED: tests/CMakeFiles/phase_basic.dir/Unity/unity_0_cxx.cxx.o 
/usr/bin/ccache /usr/bin/g++  -I/vt/tests/unit -I/vt/lib/CLI -I/vt/lib/libfort/lib -Irelease -I/vt/src -isystem /vt/tests/extern/googletest/googletest/include -isystem /vt/tests/extern/googletest/googletest -isystem /vt/lib/fmt/include -isystem /build/checkpoint/install/include -isystem /build/detector/install/include -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -ftemplate-backtrace-limit=100 -Werror -O3 -DNDEBUG -fdiagnostics-color=always -O0 -g --coverage -fPIC -std=c++14 -MD -MT tests/CMakeFiles/phase_basic.dir/Unity/unity_0_cxx.cxx.o -MF tests/CMakeFiles/phase_basic.dir/Unity/unity_0_cxx.cxx.o.d -o tests/CMakeFiles/phase_basic.dir/Unity/unity_0_cxx.cxx.o -c tests/CMakeFiles/phase_basic.dir/Unity/unity_0_cxx.cxx
In file included from tests/CMakeFiles/phase_basic.dir/Unity/unity_0_cxx.cxx:5:0:
/vt/tests/unit/phase/test_phase_management.cc:110:3: fatal error: error writing to /tmp/cckIcNXJ.s: No space left on device
 }}} // end namespace vt::tests::unit
   ^
compilation terminated.


Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (nvidia cuda 11.0, ubuntu, mpich)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

FAILED: src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o 
/usr/bin/ccache /nvcc_wrapper/build/nvcc_wrapper  -I/vt/lib/CLI -Irelease -I/vt/src -isystem /vt/lib/fmt/include -isystem /build/checkpoint/install/include -isystem /build/detector/install/include -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -ftemplate-backtrace-limit=100 -Werror -Wno-deprecated-gpu-targets -O3 -DNDEBUG -fdiagnostics-color=always -fPIC -std=c++14 -MD -MT src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o -MF src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o.d -o src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o -c src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx
Killed


Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (clang-9, ubuntu, mpich)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

Compilation - successful

Testing - passed

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (intel 18.03, ubuntu, mpich)

Build for f887032d2a866c1ad619d9ed269b487080b00d21

FAILED: src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o 
/usr/bin/ccache /opt/intel/install/bin/icpc -DFMT_HEADER_ONLY=1 -DFMT_USE_USER_DEFINED_LITERALS=0 -I/vt/lib/fmt -I/vt/lib/CLI -Irelease -I/vt/src -I/vt/lib/libfort/lib -isystem /build/checkpoint/install/include -isystem /build/detector/install/include -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -Werror -O3 -DNDEBUG -fPIC -std=c++14 -MD -MT src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o -MF src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o.d -o src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o -c src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx
icpc: error #10106: Fatal error in /opt/intel/system_studio_2018/compilers_and_libraries_2018.4.253/linux/bin/intel64/mcpcom, terminated by kill signal
compilation aborted for src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx (code 1)


Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (clang-10, ubuntu, mpich)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

Compilation - successful

Testing - passed

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (clang-8, alpine, mpich)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

Compilation - successful

Testing - passed

Build log

github-actions[bot] avatar Apr 15 '21 22:04 github-actions[bot]

PR tests (intel 19, ubuntu, mpich)

Build for 8897865d1e0fe990b4624b36b91a441852447cd7

FAILED: src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o 
/usr/bin/ccache /opt/intel/install/bin/icpc  -I/vt/lib/CLI -Irelease -I/vt/src -I/vt/lib/libfort/lib -isystem /vt/lib/fmt/include -isystem /build/checkpoint/install/include -isystem /build/detector/install/include -Wall -pedantic -Wshadow -Wno-unknown-pragmas -Wsign-compare -Werror -O3 -DNDEBUG -fPIC -std=c++14 -MD -MT src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o -MF src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o.d -o src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx.o -c src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx
icpc: error #10106: Fatal error in /opt/intel/sw_dev_tools/compilers_and_libraries_2020.1.219/linux/bin/intel64/mcpcom, terminated by kill signal
compilation aborted for src/CMakeFiles/vt.dir/Unity/unity_7_cxx.cxx (code 1)


Build log

github-actions[bot] avatar Apr 15 '21 23:04 github-actions[bot]

Without digging into what git diff output shows directly, is the code in the new location.routing.impl.h strictly cut and pasted from location.impl.h, or are there additional modifications in that same commit? I saw the modification made in separate commits.

The code pretty much looks good to me (legitimate macos clang failure aside), but I'd still like an affirmative response to this.

PhilMiller avatar Apr 16 '21 21:04 PhilMiller

Without digging into what git diff output shows directly, is the code in the new location.routing.impl.h strictly cut and pasted from location.impl.h, or are there additional modifications in that same commit? I saw the modification made in separate commits.

They were cut and paste directly. I needed all the functions that require MsgT to be included in things that use the location manager.

lifflander avatar Apr 19 '21 17:04 lifflander

I'm still trying to get to the bottom of this error. ETI covers this instantiation, but I was able to reproduce this locally on my Mac. Not sure why it doesn't show up on Azure. Maybe unity builds are affecting it?

Undefined symbols for architecture x86_64:
  "vt::location::EntityLocationCoord<vt::index::DenseIndexArray<unsigned long, (signed char)1> >::registerEntity(vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, short const&, std::__1::function<void (vt::messaging::BaseMsg*)>, bool const&)", referenced from:
      bool vt::vrt::collection::CollectionManager::insertCollectionElement<vt::tests::unit::test_index_types_::TestCol<vt::index::DenseIndexArray<unsigned long, (signed char)1> >, vt::index::DenseIndexArray<unsigned long, (signed char)1> >(vt::vrt::collection::Holder<vt::index::DenseIndexArray<unsigned long, (signed char)1> >::VirtualPtrType, vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, long long, unsigned long long const&, bool, short const&, bool const&, short const&) in test_index_types.extended.cc.o
  "vt::location::EntityLocationCoord<vt::index::DenseIndexArray<unsigned long, (signed char)1> >::entityEmigrated(vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, short const&)", referenced from:
      vt::vrt::collection::MigrateStatus vt::vrt::collection::CollectionManager::migrateOut<vt::tests::unit::test_index_types_::TestCol<vt::index::DenseIndexArray<unsigned long, (signed char)1> >, vt::index::DenseIndexArray<unsigned long, (signed char)1> >(unsigned long long const&, vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, short const&) in test_index_types.extended.cc.o
  "vt::location::EntityLocationCoord<vt::index::DenseIndexArray<unsigned long, (signed char)1> >::sendEagerUpdate(vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, short, short, short)", referenced from:
      void vt::location::EntityLocationCoord<vt::index::DenseIndexArray<unsigned long, (signed char)1> >::routeMsgNode<vt::tests::unit::test_index_types_::ColMsg<vt::index::DenseIndexArray<unsigned long, (signed char)1> > >(vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, short const&, short const&, vt::messaging::MsgSharedPtr<vt::tests::unit::test_index_types_::ColMsg<vt::index::DenseIndexArray<unsigned long, (signed char)1> > >)::'lambda'(vt::index::DenseIndexArray<unsigned long, (signed char)1> const&)::operator()(vt::index::DenseIndexArray<unsigned long, (signed char)1> const&) const in test_index_types.extended.cc.o
  "vt::location::EntityLocationCoord<vt::index::DenseIndexArray<unsigned long, (signed char)1> >::entityImmigrated(vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, short const&, short const&, std::__1::function<void (vt::messaging::BaseMsg*)>)", referenced from:
      bool vt::vrt::collection::CollectionManager::insertCollectionElement<vt::tests::unit::test_index_types_::TestCol<vt::index::DenseIndexArray<unsigned long, (signed char)1> >, vt::index::DenseIndexArray<unsigned long, (signed char)1> >(vt::vrt::collection::Holder<vt::index::DenseIndexArray<unsigned long, (signed char)1> >::VirtualPtrType, vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, long long, unsigned long long const&, bool, short const&, bool const&, short const&) in test_index_types.extended.cc.o
  "vt::location::EntityLocationCoord<vt::index::DenseIndexArray<unsigned long, (signed char)1> >::insertPendingEntityAction(vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, std::__1::function<void (short)>)", referenced from:
      void vt::location::EntityLocationCoord<vt::index::DenseIndexArray<unsigned long, (signed char)1> >::routeMsgNode<vt::tests::unit::test_index_types_::ColMsg<vt::index::DenseIndexArray<unsigned long, (signed char)1> > >(vt::index::DenseIndexArray<unsigned long, (signed char)1> const&, short const&, short const&, vt::messaging::MsgSharedPtr<vt::tests::unit::test_index_types_::ColMsg<vt::index::DenseIndexArray<unsigned long, (signed char)1> > >) in test_index_types.extended.cc.o
  "vt::location::EntityLocationCoord<vt::index::DenseIndexArray<unsigned long, (signed char)1> >::EntityLocationCoord(vt::location::collection_lm_tag_t, long long)", referenced from:
      void vt::location::LocationManager::insertCollectionLM<vt::index::DenseIndexArray<unsigned long, (signed char)1> >(unsigned long long const&) in test_index_types.extended.cc.o
ld: symbol(s) not found for architecture x86_64

lifflander avatar Apr 20 '21 16:04 lifflander

Codecov Report

Merging #1389 (463bafe) into develop (8337311) will increase coverage by 0.07%. The diff coverage is n/a.

:exclamation: Current head 463bafe differs from pull request most recent head 8897865. Consider uploading reports for the commit 8897865 to get more accurate results Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1389      +/-   ##
===========================================
+ Coverage    82.69%   82.76%   +0.07%     
===========================================
  Files          760      761       +1     
  Lines        28682    28679       -3     
===========================================
+ Hits         23719    23737      +18     
+ Misses        4963     4942      -21     
Impacted Files Coverage Δ
src/vt/topos/location/location.impl.h 90.49% <0.00%> (-3.69%) :arrow_down:
src/vt/runnable/runnable.h 83.33% <0.00%> (-2.39%) :arrow_down:
src/vt/runnable/make_runnable.h 98.50% <0.00%> (-0.15%) :arrow_down:
src/vt/vrt/collection/manager.impl.h 94.71% <0.00%> (-0.02%) :arrow_down:
tests/unit/mpi_singleton.h 100.00% <0.00%> (ø)
src/vt/context/runnable_context/lb_stats.impl.h 100.00% <0.00%> (ø)
.../unit/rdma/test_rdma_collection_handle.extended.cc 100.00% <0.00%> (ø)
tests/unit/test_helpers.h 100.00% <0.00%> (ø)
src/vt/pipe/pipe_manager_base.impl.h 80.28% <0.00%> (+0.28%) :arrow_up:
tests/unit/termination/test_term_dep_send_chain.cc 98.65% <0.00%> (+2.96%) :arrow_up:
... and 4 more

codecov[bot] avatar Apr 21 '21 00:04 codecov[bot]

@PhilMiller I think some of the failures are due to too large a compilation time/memory for the new location.eti.cc, which instantiates a ton of types now. We might need to split those up or reduce them.

lifflander avatar Apr 21 '21 02:04 lifflander