[tmva] [sofie] Add support for tensors with dynamic shape

[lmoneta]

This Pull request:

This PR is continuing #13457 by Ahmat and it is based on https://github.com/root-project/root/pull/12941 and Dynamic-Op both by Neel Shah

Changes of Fixes

• Introduce intermediate tensors with parametric shapes (Dynamic tensors)
• Implement the Range operator
• Add tests for the Range operator
• Add full support for Input tensors with parametric shapes (using the Dim structure) and propagate the shapes using Dim also in the intermediate ones.
• Start changing operators to support inputs/output tensors with Dim shapes.
  • support now Gemm, Relu, Concat, LayerNormalization
• Add support for input Dim shapes in GNN classes to support fully variable number of nodes and edges

[phsft-bot]

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

[phsft-bot]

Build failed on ROOT-ubuntu2004/python3. Running on root-ubuntu-2004-1.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Errors:

• [2024-01-23T21:37:15.968Z] FAILED: tmva/sofie/test/CMakeFiles/SofieCompileModels_ONNX.util

[phsft-bot]

Build failed on ROOT-ubuntu2204/nortcxxmod. Running on root-ubuntu-2204-3.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

[github-actions[bot]]

Test Results

    12 files      12 suites   2d 2h 53m 21s :stopwatch:  2 561 tests  2 555 :white_check_mark: 0 :zzz: 6 :x: 28 810 runs  28 803 :white_check_mark: 0 :zzz: 7 :x:

For more details on these failures, see this check.

Results for commit c87417cc.

:recycle: This comment has been updated with latest results.

[phsft-bot]

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

[phsft-bot]

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

[phsft-bot]

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

[phsft-bot]

Build failed on ROOT-ubuntu2204/nortcxxmod. Running on root-ubuntu-2204-3.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Errors:

• [2024-02-08T19:13:50.278Z] /home/sftnight/build/workspace/root-pullrequests-build/root/tmva/sofie/inc/TMVA/ROperator_LayerNormalization.hxx:96:37: error: call of overloaded ‘AddIntermediateTensor(std::string&, TMVA::Experimental::SOFIE::ETensorType&, <brace-enclosed initializer list>)’ is ambiguous
• [2024-02-08T19:13:50.279Z] /home/sftnight/build/workspace/root-pullrequests-build/root/tmva/sofie/inc/TMVA/ROperator_LayerNormalization.hxx:101:37: error: call of overloaded ‘AddIntermediateTensor(std::string&, TMVA::Experimental::SOFIE::ETensorType&, <brace-enclosed initializer list>)’ is ambiguous

[phsft-bot]

Build failed on ROOT-ubuntu2004/python3. Running on root-ubuntu-2004-3.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Errors:

• [2024-02-08T19:39:34.407Z] FAILED: tmva/sofie/CMakeFiles/ROOTTMVASofie.dir/src/RFunction_MLP.cxx.o
• [2024-02-08T19:39:35.340Z] /home/sftnight/build/workspace/root-pullrequests-build/root/tmva/sofie/inc/TMVA/ROperator_LayerNormalization.hxx:96:65: error: call of overloaded ‘AddIntermediateTensor(std::string&, TMVA::Experimental::SOFIE::ETensorType&, <brace-enclosed initializer list>)’ is ambiguous
• [2024-02-08T19:39:35.340Z] /home/sftnight/build/workspace/root-pullrequests-build/root/tmva/sofie/inc/TMVA/ROperator_LayerNormalization.hxx:101:70: error: call of overloaded ‘AddIntermediateTensor(std::string&, TMVA::Experimental::SOFIE::ETensorType&, <brace-enclosed initializer list>)’ is ambiguous
• [2024-02-08T19:39:36.690Z] FAILED: tmva/sofie/CMakeFiles/G__ROOTTMVASofie.dir/G__ROOTTMVASofie.cxx.o
• [2024-02-08T19:39:37.635Z] /home/sftnight/build/workspace/root-pullrequests-build/root/tmva/sofie/inc/TMVA/ROperator_LayerNormalization.hxx:96:65: error: call of overloaded ‘AddIntermediateTensor(std::string&, TMVA::Experimental::SOFIE::ETensorType&, <brace-enclosed initializer list>)’ is ambiguous
• [2024-02-08T19:39:37.635Z] /home/sftnight/build/workspace/root-pullrequests-build/root/tmva/sofie/inc/TMVA/ROperator_LayerNormalization.hxx:101:70: error: call of overloaded ‘AddIntermediateTensor(std::string&, TMVA::Experimental::SOFIE::ETensorType&, <brace-enclosed initializer list>)’ is ambiguous

[phsft-bot]

Build failed on windows10/default. Running on null:C:\build\workspace\root-pullrequests-build See console output.

Errors:

• [2024-02-08T19:36:16.035Z] C:\build\workspace\root-pullrequests-build\root\tmva\sofie\inc\TMVA/ROperator_LayerNormalization.hxx(96,15): error C2668: 'TMVA::Experimental::SOFIE::RModel::AddIntermediateTensor': ambiguous call to overloaded function [C:\build\workspace\root-pullrequests-build\build\tmva\sofie\ROOTTMVASofie.vcxproj]
• [2024-02-08T19:36:16.036Z] C:\build\workspace\root-pullrequests-build\root\tmva\sofie\inc\TMVA/ROperator_LayerNormalization.hxx(101,15): error C2668: 'TMVA::Experimental::SOFIE::RModel::AddIntermediateTensor': ambiguous call to overloaded function [C:\build\workspace\root-pullrequests-build\build\tmva\sofie\ROOTTMVASofie.vcxproj]

[phsft-bot]

Build failed on ROOT-performance-centos8-multicore/soversion. Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build See console output.

Errors:

• [2024-02-08T21:01:39.996Z] /data/sftnight/workspace/root-pullrequests-build/root/tmva/sofie/inc/TMVA/ROperator_LayerNormalization.hxx:96:65: error: call of overloaded ‘AddIntermediateTensor(std::__cxx11::string&, TMVA::Experimental::SOFIE::ETensorType&, <brace-enclosed initializer list>)’ is ambiguous
• [2024-02-08T21:01:39.996Z] /data/sftnight/workspace/root-pullrequests-build/root/tmva/sofie/inc/TMVA/ROperator_LayerNormalization.hxx:101:70: error: call of overloaded ‘AddIntermediateTensor(std::__cxx11::string&, TMVA::Experimental::SOFIE::ETensorType&, <brace-enclosed initializer list>)’ is ambiguous

[phsft-bot]

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

[phsft-bot]

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

[phsft-bot]

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

[phsft-bot]

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

[phsft-bot]

Build failed on ROOT-ubuntu2204/nortcxxmod. Running on root-ubuntu-2204-1.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Failing tests:

• projectroot.tmva.pymva.test.PyMVA_Torch_Regression
• projectroot.tmva.pymva.test.PyMVA_Torch_Classification
• projectroot.tmva.pymva.test.PyMVA_Torch_Multiclass
• projectroot.tmva.pymva.test.gtest_tmva_pymva_test_TestRModelParserPyTorch

[phsft-bot]

Build failed on ROOT-ubuntu2004/python3. Running on root-ubuntu-2004-1.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Failing tests:

• projectroot.tmva.pymva.test.PyMVA_Torch_Classification
• projectroot.tmva.pymva.test.PyMVA_Torch_Regression
• projectroot.tmva.pymva.test.PyMVA_Torch_Multiclass
• projectroot.tmva.pymva.test.gtest_tmva_pymva_test_TestRModelParserPyTorch

[phsft-bot]

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

[phsft-bot]

Build failed on ROOT-ubuntu2204/nortcxxmod. Running on root-ubuntu-2204-1.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Errors:

• [2024-02-16T17:51:05.902Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/RangeFloat_FromONNX.hxx:23:13: error: ‘other’ was not declared in this scope
• [2024-02-16T17:51:05.902Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/RangeFloat_FromONNX.hxx:23:18: error: template argument 1 is invalid
• [2024-02-16T17:51:05.902Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/RangeFloat_FromONNX.hxx:23:18: error: template argument 2 is invalid
• [2024-02-16T17:51:05.902Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/RangeFloat_FromONNX.hxx:18:7: error: ‘fTensor_Y’ was not declared in this scope
• [2024-02-16T17:51:05.902Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/RangeFloat_FromONNX.hxx:19:7: error: ‘tensor_Y’ was not declared in this scope
• [2024-02-16T17:51:05.902Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/RangeFloat_FromONNX.hxx:27:21: error: ‘fTensor_Y’ was not declared in this scope
• [2024-02-16T17:51:05.902Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/RangeFloat_FromONNX.hxx:29:7: error: ‘tensor_Y’ was not declared in this scope
• [2024-02-16T17:51:05.902Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/RangeFloat_FromONNX.hxx:32:7: error: ‘fTensor_Y’ was not declared in this scope
• [2024-02-16T17:51:05.902Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/RangeFloat_FromONNX.hxx:34:16: error: ‘other’ was not declared in this scope
• [2024-02-16T17:51:05.902Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/RangeFloat_FromONNX.hxx:34:21: error: template argument 1 is invalid

And 16 more

Warnings:

• [2024-02-16T17:51:05.899Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/Linear_16_FromONNX.hxx:420:9: warning: unused variable ‘op_0_transA’ [-Wunused-variable]
• [2024-02-16T17:51:05.899Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/Linear_16_FromONNX.hxx:421:9: warning: unused variable ‘op_0_transB’ [-Wunused-variable]
• [2024-02-16T17:51:05.899Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/Linear_16_FromONNX.hxx:422:8: warning: unused variable ‘op_0_m’ [-Wunused-variable]
• [2024-02-16T17:51:05.899Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/Linear_16_FromONNX.hxx:423:8: warning: unused variable ‘op_0_n’ [-Wunused-variable]
• [2024-02-16T17:51:05.899Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/Linear_16_FromONNX.hxx:424:8: warning: unused variable ‘op_0_k’ [-Wunused-variable]
• [2024-02-16T17:51:05.899Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/Linear_16_FromONNX.hxx:425:10: warning: unused variable ‘op_0_alpha’ [-Wunused-variable]
• [2024-02-16T17:51:05.899Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/Linear_16_FromONNX.hxx:426:10: warning: unused variable ‘op_0_beta’ [-Wunused-variable]
• [2024-02-16T17:51:05.899Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/Linear_16_FromONNX.hxx:427:8: warning: unused variable ‘op_0_lda’ [-Wunused-variable]
• [2024-02-16T17:51:05.899Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/Linear_16_FromONNX.hxx:428:8: warning: unused variable ‘op_0_ldb’ [-Wunused-variable]
• [2024-02-16T17:51:05.899Z] /home/sftnight/build/workspace/root-pullrequests-build/build/tmva/sofie/test/Linear_16_FromONNX.hxx:437:9: warning: unused variable ‘op_2_transA’ [-Wunused-variable]

And 301 more

[phsft-bot]

Build failed on ROOT-ubuntu2004/python3. Running on root-ubuntu-2004-1.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Errors:

• [2024-02-16T20:30:16.039Z] FAILED: tmva/sofie/test/CMakeFiles/TestCustomModelsFromONNX.dir/TestCustomModelsFromONNX.cxx.o
• [2024-02-16T20:30:16.319Z] tmva/sofie/test/RangeFloat_FromONNX.hxx:23:13: error: ‘other’ was not declared in this scope
• [2024-02-16T20:30:16.319Z] tmva/sofie/test/RangeFloat_FromONNX.hxx:23:18: error: template argument 1 is invalid
• [2024-02-16T20:30:16.319Z] tmva/sofie/test/RangeFloat_FromONNX.hxx:23:18: error: template argument 2 is invalid
• [2024-02-16T20:30:16.319Z] tmva/sofie/test/RangeFloat_FromONNX.hxx:18:7: error: ‘fTensor_Y’ was not declared in this scope
• [2024-02-16T20:30:16.319Z] tmva/sofie/test/RangeFloat_FromONNX.hxx:19:7: error: ‘tensor_Y’ was not declared in this scope
• [2024-02-16T20:30:16.319Z] tmva/sofie/test/RangeFloat_FromONNX.hxx:27:21: error: ‘fTensor_Y’ was not declared in this scope
• [2024-02-16T20:30:16.319Z] tmva/sofie/test/RangeFloat_FromONNX.hxx:29:7: error: ‘tensor_Y’ was not declared in this scope
• [2024-02-16T20:30:16.319Z] tmva/sofie/test/RangeFloat_FromONNX.hxx:32:7: error: ‘fTensor_Y’ was not declared in this scope
• [2024-02-16T20:30:16.319Z] tmva/sofie/test/RangeFloat_FromONNX.hxx:34:16: error: ‘other’ was not declared in this scope

And 17 more

Warnings:

• [2024-02-16T20:30:16.318Z] tmva/sofie/test/Linear_16_FromONNX.hxx:420:9: warning: unused variable ‘op_0_transA’ [-Wunused-variable]
• [2024-02-16T20:30:16.318Z] tmva/sofie/test/Linear_16_FromONNX.hxx:421:9: warning: unused variable ‘op_0_transB’ [-Wunused-variable]
• [2024-02-16T20:30:16.318Z] tmva/sofie/test/Linear_16_FromONNX.hxx:422:8: warning: unused variable ‘op_0_m’ [-Wunused-variable]
• [2024-02-16T20:30:16.318Z] tmva/sofie/test/Linear_16_FromONNX.hxx:423:8: warning: unused variable ‘op_0_n’ [-Wunused-variable]
• [2024-02-16T20:30:16.318Z] tmva/sofie/test/Linear_16_FromONNX.hxx:424:8: warning: unused variable ‘op_0_k’ [-Wunused-variable]
• [2024-02-16T20:30:16.318Z] tmva/sofie/test/Linear_16_FromONNX.hxx:425:10: warning: unused variable ‘op_0_alpha’ [-Wunused-variable]
• [2024-02-16T20:30:16.318Z] tmva/sofie/test/Linear_16_FromONNX.hxx:426:10: warning: unused variable ‘op_0_beta’ [-Wunused-variable]
• [2024-02-16T20:30:16.318Z] tmva/sofie/test/Linear_16_FromONNX.hxx:427:8: warning: unused variable ‘op_0_lda’ [-Wunused-variable]
• [2024-02-16T20:30:16.318Z] tmva/sofie/test/Linear_16_FromONNX.hxx:428:8: warning: unused variable ‘op_0_ldb’ [-Wunused-variable]
• [2024-02-16T20:30:16.318Z] tmva/sofie/test/Linear_16_FromONNX.hxx:437:9: warning: unused variable ‘op_2_transA’ [-Wunused-variable]

And 301 more

[phsft-bot]

Starting build on ROOT-performance-centos8-multicore/soversion, ROOT-ubuntu2204/nortcxxmod, ROOT-ubuntu2004/python3, mac12arm/cxx20, windows10/default How to customize builds

[lmoneta]

@dpiparo: I have addressed one of your comment. The other ones (e.g. on the arguments passed by values) I would prefer to address in a separate PR, since it affects the internal API of SOFIE and all the operator code. So, if you don't have further objections, I would like to merge this PR

[phsft-bot]

Build failed on ROOT-ubuntu2204/nortcxxmod. Running on root-ubuntu-2204-3.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Failing tests:

• projectroot.tmva.pymva.test.PyMVA_Torch_Regression
• projectroot.tmva.pymva.test.PyMVA_Torch_Classification
• projectroot.tmva.pymva.test.PyMVA_Torch_Multiclass
• projectroot.tmva.pymva.test.gtest_tmva_pymva_test_TestRModelParserPyTorch

[phsft-bot]

Build failed on ROOT-ubuntu2004/python3. Running on root-ubuntu-2004-1.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.

Failing tests:

• projectroot.tmva.pymva.test.PyMVA_Torch_Regression
• projectroot.tmva.pymva.test.PyMVA_Torch_Classification
• projectroot.tmva.pymva.test.PyMVA_Torch_Multiclass
• projectroot.tmva.pymva.test.gtest_tmva_pymva_test_TestRModelParserPyTorch

[phsft-bot]

Build failed on ROOT-performance-centos8-multicore/soversion. Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build See console output.

Failing tests:

• projectroot.runtutorials.tutorial_rcanvas_rline.cxx