nengo-dl icon indicating copy to clipboard operation
nengo-dl copied to clipboard

Use TensorFlow 2.3 for remote tests

Open hunse opened this issue 4 years ago • 0 comments

This causes a regression in two benchmark tests:

  • net4: Random network with RectifiedLinear neurons
  • net6: LMU with eager mode off

Two other benchmarks are adjusted slightly to re-center them on the times currently achieved under TensorFlow 2.2.

Here are the full results before this commit:

nengo_dl/tests/test_benchmarks.py::test_performance[net0-False-64-True-1.0-1.15] Execution times (15): 1.092 (min), 1.146 (max), 1.122 (mean), 0.016 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net1-False-64-True-2.25-2.55] Execution times (15): 2.468 (min), 2.507 (max), 2.488 (mean), 0.014 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net2-True-64-True-0.6-0.9] Execution times (15): 0.793 (min), 0.842 (max), 0.812 (mean), 0.015 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net3-True-64-True-0.95-1.15] Execution times (15): 1.034 (min), 1.082 (max), 1.057 (mean), 0.013 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net4-False-None-True-0.5-0.7] Execution times (15): 0.647 (min), 0.656 (max), 0.649 (mean), 0.002 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net5-True-100-True-1.25-1.45] Execution times (15): 1.345 (min), 1.399 (max), 1.367 (mean), 0.015 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net6-True-100-False-1.05-1.25] Execution times (15): 1.180 (min), 1.197 (max), 1.186 (mean), 0.005 (std)

Here are the full results after this commit:

nengo_dl/tests/test_benchmarks.py::test_performance[net0-False-64-True-1.0-1.15] Execution times (15): 1.109 (min), 1.186 (max), 1.134 (mean), 0.020 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net1-False-64-True-2.25-2.55] Execution times (15): 2.488 (min), 2.539 (max), 2.509 (mean), 0.015 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net2-True-64-True-0.6-0.9] Execution times (15): 0.684 (min), 0.728 (max), 0.700 (mean), 0.013 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net3-True-64-True-0.95-1.15] Execution times (15): 1.023 (min), 1.064 (max), 1.040 (mean), 0.011 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net4-False-None-True-0.5-0.7] Execution times (15): 0.700 (min), 0.717 (max), 0.705 (mean), 0.004 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net5-True-100-True-1.25-1.45] Execution times (15): 1.326 (min), 1.357 (max), 1.339 (mean), 0.009 (std) nengo_dl/tests/test_benchmarks.py::test_performance[net6-True-100-False-1.05-1.25] Execution times (15): 1.237 (min), 1.262 (max), 1.245 (mean), 0.006 (std)

hunse avatar Jan 25 '21 17:01 hunse