dash icon indicating copy to clipboard operation
dash copied to clipboard

Test cases for StoreHDF5 read and write for issue #692

Open anindex opened this issue 5 years ago • 5 comments

This PR proposes test cases for StoreHDF5::read and StoreHDF5::write with dash::Matrix. Test cases consist of 1D and 2D matrices with the default and unsigned index type.

anindex avatar Mar 10 '20 13:03 anindex

I met some weird internal errors in HDF5 library when setting extend variable too small in 1D matrix test case.

HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) MPI-process 3:
  #000: ../../../src/H5Dio.c line 263 in H5Dwrite(): file selection+offset not within extent
    major: Dataspace
    minor: Out of range

Otherwise, all test cases pass successfully.

anindex avatar Mar 10 '20 14:03 anindex

Codecov Report

Merging #699 into development will increase coverage by 0.53%. The diff coverage is 100%.

@@               Coverage Diff               @@
##           development     #699      +/-   ##
===============================================
+ Coverage        83.65%   84.18%   +0.53%     
===============================================
  Files              336      336              
  Lines            24968    25072     +104     
  Branches         11354    11411      +57     
===============================================
+ Hits             20887    21108     +221     
- Misses            3692     3693       +1     
+ Partials           389      271     -118
Impacted Files Coverage Δ
dart-impl/base/src/hwinfo.c 85% <100%> (ø) :arrow_up:
dash/test/io/HDF5MatrixTest.cc 100% <100%> (ø) :arrow_up:
dash/include/dash/GlobRef.h 76.99% <0%> (-13.28%) :arrow_down:
dash/include/dash/memory/UniquePtr.h 90% <0%> (-10%) :arrow_down:
dash/include/dash/view/ViewMod.h 92.94% <0%> (-7.06%) :arrow_down:
dash/include/dash/Team.h 80.41% <0%> (-5.16%) :arrow_down:
dash/include/dash/pattern/MakePattern.h 81.9% <0%> (-4.77%) :arrow_down:
...pi/include/dash/dart/mpi/dart_communication_priv.h 74.28% <0%> (-4.29%) :arrow_down:
dash/include/dash/view/IndexSet.h 94.21% <0%> (-3.31%) :arrow_down:
dash/include/dash/Dimensional.h 61.46% <0%> (-1.84%) :arrow_down:
... and 27 more

codecov[bot] avatar Mar 11 '20 13:03 codecov[bot]

I have worked around the internal HDF5 errors of 1D dash::Matrix by initializing pattern_t and create the matrix instance using pattern_t:

typedef dash::TilePattern<1> pattern_t;
typedef dash::Matrix<value_t, 1> matrix_t;

pattern_t pattern(dash::SizeSpec<1, pattern_t::size_type>(8),
                           dash::DistributionSpec<1>(dash::TILE(2)),
                           dash::TeamSpec<1, pattern_t::index_type>());

...

matrix_t mat1(pattern);
...

As mentioned before, StoreHDF::write popups internal HDF5 library error if I use this initialization:

typedef dash::Matrix<value_t, 1> matrix_t;
auto size_spec = dash::SizeSpec<1>(8);

...

matrix_t mat1(size_spec);

...

anindex avatar Mar 11 '20 14:03 anindex

@anindex

As mentioned before, StoreHDF::write popups internal HDF5 library error if I use this initialization:

typedef dash::Matrix<value_t, 1> matrix_t;
auto size_spec = dash::SizeSpec<1>(8);

matrix_t mat1(size_spec);

Which error exactly do you see there?

devreal avatar Mar 11 '20 18:03 devreal

The error I posted in the previous comment and it is also the cause for CI failing last time we saw it.

HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) MPI-process 3:
  #000: ../../../src/H5Dio.c line 263 in H5Dwrite(): file selection+offset not within extent
    major: Dataspace
    minor: Out of range

anindex avatar Mar 11 '20 19:03 anindex