rbc
rbc copied to clipboard
heavyai
test_externals_cmath.py::test_external_cmath_omnisci tests are slow
As in title. The reason is that each subtest compiles its own function to an independent LLVM module. To speed up the tests, all functions should be compiled into a single LLVM module.
Data points:
- running all rbc tests using CPU-only omniscidb server takes about 12 min.
- running all rbs tests using CUDA-enabled omniscidb server takes about 20 min
for fn in rbc/tests/test_*.py; do echo $fn; (pytest $fn | tail -1) ; done
leads to (using CUDA enabled omniscidb)
rbc/tests/test_ci.py
======================== 3 skipped, 2 warnings in 1.92s ========================
rbc/tests/test_external.py
======================== 5 passed, 2 warnings in 2.53s =========================
rbc/tests/test_externals_cmath.py
=========== 147 passed, 9 skipped, 9 xfailed, 26 warnings in 14.58s ============
rbc/tests/test_externals_libdevice.py
=========== 306 passed, 11 skipped, 2 warnings in 150.51s (0:02:30) ============
rbc/tests/test_libcudart_ctypes.py
============================== 1 skipped in 0.39s ==============================
rbc/tests/test_numpy_rjit.py
============================== 6 passed in 1.49s ===============================
rbc/tests/test_omnisci.py
================== 12 passed, 3 skipped, 3 warnings in 19.56s ==================
rbc/tests/test_omnisci_array.py
================= 21 passed, 12 skipped, 12 warnings in 14.72s =================
rbc/tests/test_omnisci_array_functions.py
======================= 13 passed, 14 warnings in 8.56s ========================
rbc/tests/test_omnisci_array_math.py
================= 107 passed, 119 warnings in 70.66s (0:01:10) =================
rbc/tests/test_omnisci_array_methods.py
================== 13 passed, 1 skipped, 14 warnings in 7.66s ==================
rbc/tests/test_omnisci_array_null.py
======================== 7 passed, 12 warnings in 4.70s ========================
rbc/tests/test_omnisci_array_operators.py
======================= 50 passed, 61 warnings in 26.16s =======================
rbc/tests/test_omnisci_black_scholes.py
======================== 2 passed, 3 warnings in 3.41s =========================
rbc/tests/test_omnisci_column_arguments.py
======================= 54 passed, 55 warnings in 12.43s =======================
rbc/tests/test_omnisci_column_basic.py
======= 76 passed, 16 skipped, 3 xfailed, 1 warning in 124.42s (0:02:04) =======
rbc/tests/test_omnisci_column_default.py
======================= 34 passed, 35 warnings in 5.18s ========================
rbc/tests/test_omnisci_column_list.py
======================= 18 passed, 2 warnings in 11.52s ========================
rbc/tests/test_omnisci_column_null.py
======================== 8 passed, 9 warnings in 8.20s =========================
rbc/tests/test_omnisci_device_selection.py
================ 170 passed, 34 skipped, 205 warnings in 32.98s ================
rbc/tests/test_omnisci_math.py
============ 257 passed, 33 skipped, 1 warning in 169.73s (0:02:49) ============
rbc/tests/test_omnisci_mlpack.py
======================== 2 skipped, 3 warnings in 2.04s ========================
rbc/tests/test_omnisci_testing.py
======================= 10 passed, 30 warnings in 21.53s =======================
rbc/tests/test_omnisci_text.py
======================== 5 passed, 3 warnings in 4.36s =========================
rbc/tests/test_omnisci_text_encoding.py
================= 24 passed, 3 xfailed, 57 warnings in 32.63s ==================
rbc/tests/test_omnisci_treelite.py
======================== 1 skipped, 2 warnings in 2.21s ========================
rbc/tests/test_omnisci_udtf.py
================== 6 passed, 1 skipped, 7 warnings in 24.17s ===================
rbc/tests/test_remotejit.py
====================== 130 passed, 73 warnings in 34.01s =======================
rbc/tests/test_thrift.py
============================== 19 passed in 1.62s ==============================
rbc/tests/test_thrift_multiplexed.py
============================== 18 passed in 1.62s ==============================
rbc/tests/test_typesystem.py
============================== 20 passed in 0.45s ==============================
rbc/tests/test_utils.py
============================== 3 passed in 0.35s ===============================
that gives the following top on slow tests:
rbc/tests/test_omnisci_math.py
============ 257 passed, 33 skipped, 1 warning in 169.73s (0:02:49) ============
rbc/tests/test_externals_libdevice.py
=========== 306 passed, 11 skipped, 2 warnings in 150.51s (0:02:30) ============
rbc/tests/test_omnisci_column_basic.py
======= 76 passed, 16 skipped, 3 xfailed, 1 warning in 124.42s (0:02:04) =======
rbc/tests/test_omnisci_array_math.py
================= 107 passed, 119 warnings in 70.66s (0:01:10) =================
Update on May 2023:
Top slowdowns:
rbc/tests/heavydb/test_array_math.py
================= 107 passed, 225 warnings in 90.87s (0:01:30) =================
rbc/tests/heavydb/test_column_basic.py
===== 76 passed, 18 skipped, 3 xfailed, 227 warnings in 106.57s (0:01:46) ======
rbc/tests/heavydb/test_math.py
========== 129 passed, 161 skipped, 253 warnings in 69.15s (0:01:09) ===========
rbc/tests/heavydb/test_nrt_string.py
============= 9 failed, 50 passed, 10 warnings in 64.83s (0:01:04) =============
$ for fn in rbc/tests/heavydb/test_*.py; do echo $fn; (pytest $fn | tail -1) ; done
rbc/tests/heavydb/test_array_api_unsupported.py
====================== 45 passed, 136 warnings in 12.59s =======================
rbc/tests/heavydb/test_array_functions.py
======================= 50 passed, 91 warnings in 46.56s =======================
rbc/tests/heavydb/test_array_math.py
================= 107 passed, 225 warnings in 90.87s (0:01:30) =================
rbc/tests/heavydb/test_array_methods.py
======================= 25 passed, 9 warnings in 11.35s ========================
rbc/tests/heavydb/test_array_null.py
======================== 7 passed, 5 warnings in 7.19s =========================
rbc/tests/heavydb/test_array_operators.py
======================= 50 passed, 25 warnings in 25.23s =======================
rbc/tests/heavydb/test_array.py
================= 36 passed, 6 skipped, 54 warnings in 23.64s ==================
rbc/tests/heavydb/test_black_scholes.py
======================== 2 passed, 2 warnings in 2.55s =========================
rbc/tests/heavydb/test_caller.py
================== 10 passed, 3 xfailed, 3 warnings in 4.28s ===================
rbc/tests/heavydb/test_column_arguments.py
======================= 54 passed, 40 warnings in 33.65s =======================
rbc/tests/heavydb/test_column_array.py
=================== 8 passed, 2 skipped, 2 warnings in 4.38s ===================
rbc/tests/heavydb/test_column_basic.py
===== 76 passed, 18 skipped, 3 xfailed, 227 warnings in 106.57s (0:01:46) ======
rbc/tests/heavydb/test_column_default.py
======================= 34 passed, 35 warnings in 8.86s ========================
rbc/tests/heavydb/test_column_list.py
======================= 19 passed, 3 warnings in 19.45s ========================
rbc/tests/heavydb/test_column_null.py
======================== 8 passed, 2 warnings in 22.80s ========================
rbc/tests/heavydb/test_column_runtime_alloc.py
======================== 8 passed, 3 warnings in 6.63s =========================
rbc/tests/heavydb/test_constants.py
======================== 4 passed, 2 warnings in 0.99s =========================
rbc/tests/heavydb/test_daytimeinterval.py
======================== 14 skipped, 1 warning in 0.59s ========================
rbc/tests/heavydb/test_device_selection.py
================= 96 passed, 108 skipped, 2 warnings in 22.94s =================
rbc/tests/heavydb/test_external.py
======================== 5 passed, 6 warnings in 1.00s =========================
rbc/tests/heavydb/test_geo_basic.py
============================== 1 passed in 0.20s ===============================
rbc/tests/heavydb/test_geolinestring.py
================== 31 passed, 1 xfailed, 2 warnings in 22.34s ==================
rbc/tests/heavydb/test_geomultipolygon.py
======================== 4 passed, 2 warnings in 1.90s =========================
rbc/tests/heavydb/test_geopoint.py
======================= 12 passed, 2 warnings in 19.07s ========================
rbc/tests/heavydb/test_geopolygon.py
======================= 20 passed, 2 warnings in 11.94s ========================
rbc/tests/heavydb/test_heavydb.py
======================= 17 passed, 17 warnings in 24.49s =======================
rbc/tests/heavydb/test_heavydb_thrift.py
======================== 8 passed, 6 warnings in 1.83s =========================
rbc/tests/heavydb/test_imports.py
============================== 4 passed in 0.20s ===============================
rbc/tests/heavydb/test_math.py
========== 129 passed, 161 skipped, 253 warnings in 69.15s (0:01:09) ===========
rbc/tests/heavydb/test_missing_symbol.py
========================= 1 passed, 1 warning in 0.28s =========================
rbc/tests/heavydb/test_mlpack.py
======================== 2 skipped, 2 warnings in 0.97s ========================
rbc/tests/heavydb/test_multilinestring.py
======================== 4 passed, 2 warnings in 18.11s ========================
rbc/tests/heavydb/test_nrt_list.py
======================= 13 passed, 28 warnings in 10.27s =======================
rbc/tests/heavydb/test_nrt_set.py
================== 16 passed, 1 skipped, 3 warnings in 21.40s ==================
rbc/tests/heavydb/test_nrt_string.py
============= 9 failed, 50 passed, 10 warnings in 64.83s (0:01:04) =============
rbc/tests/heavydb/test_template.py
======================= 11 passed, 6 warnings in 13.61s ========================
rbc/tests/heavydb/test_testing.py
======================= 13 passed, 13 warnings in 25.57s =======================
rbc/tests/heavydb/test_text_encoding_dict.py
================= 72 passed, 4 skipped, 56 warnings in 41.54s ==================
rbc/tests/heavydb/test_text_encoding_none.py
======================= 11 passed, 16 warnings in 10.41s =======================
rbc/tests/heavydb/test_timestamp.py
================= 49 passed, 37 skipped, 10 warnings in 17.27s =================
rbc/tests/heavydb/test_treelite.py
======================== 1 skipped, 1 warning in 1.34s =========================
rbc/tests/heavydb/test_udtf.py
======================= 20 passed, 37 warnings in 46.25s =======================