deeprank2 icon indicating copy to clipboard operation
deeprank2 copied to clipboard

Check performance warnings of CNNs

Open DaniBodor opened this issue 3 years ago • 3 comments

test_trainer gives the following warnings. We should investigate and either disable them or make the recommended fixes.

tests/test_trainer.py::TestTrainer::test_grid_regression /home/dbodor/git/DeepRank/deeprank-core/deeprankcore/dataset.py:376: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at /opt/conda/conda-bld/pytorch_1659484809662/work/torch/csrc/utils/tensor_new.cpp:201.)

and

tests/test_trainer.py::TestTrainer::test_grid_regression /home/dbodor/git/DeepRank/deeprank-core/deeprankcore/utils/exporters.py:260: PerformanceWarning: your performance may suffer as PyTables will pickle object types that it cannot map directly to c-types [inferred_type->mixed,key->block1_values] [items->Index(['phase', 'entry'], dtype='object')]

DaniBodor avatar Jan 10 '23 18:01 DaniBodor

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] avatar Feb 27 '23 03:02 github-actions[bot]

As @sonjageorgievska was pointing out, it seems that we use lists of numpy arrays and then we convert them directly to PyTorch tensors. Instead, we should first convert them to numpy arrays of numpy arrays (numpy arrays with one more dimension) and then to tensors.

See also this blog post about performances comparison between numpy arrays and python lists.

gcroci2 avatar Apr 03 '23 14:04 gcroci2

The first performance warning will be fixed by #438

gcroci2 avatar Jun 01 '23 12:06 gcroci2