vectorbt icon indicating copy to clipboard operation
vectorbt copied to clipboard

Unit Tests Failing V0.25.4

Open dougransom opened this issue 1 year ago • 6 comments

After noticing similar behavior on Python 3.11, I reproduced the following on Python 10 when running pytest.

(.venv) PS C:\Users\doug\code\vectorbt\tests> python Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. (.venv) PS C:\Users\doug\code\vectorbt\tests> pip show vectorbt Name: vectorbt Version: 0.25.4 Summary: Python library for backtesting and analyzing trading strategies at scale Home-page: https://github.com/polakowo/vectorbt Author: Oleg Polakow Author-email: [email protected] License: Apache 2.0 with Commons Clause Location: c:\users\doug\code\vectorbt\.venv\lib\site-packages Editable project location: C:\Users\doug\code\vectorbt Requires: dateparser, dill, imageio, ipywidgets, matplotlib, mypy-extensions, numba, numpy, pandas, plotly, pytz, requests, schedule, scikit-learn, scipy, tqdm Required-by: ================================================================= short test summary info ================================================================== FAILED test_base.py::TestColumnGrouper::test_group_by_to_index - AssertionError: Index are different FAILED test_base.py::TestReshapeFns::test_repeat_axis0 - AssertionError: Attributes of Series are different FAILED test_base.py::TestReshapeFns::test_repeat_axis1 - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="a2") are different FAILED test_base.py::TestReshapeFns::test_tile_axis0 - AssertionError: Attributes of Series are different FAILED test_base.py::TestReshapeFns::test_tile_axis1 - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="a2") are different FAILED test_base.py::TestIndexing::test_pandas_indexing - AssertionError: Attributes of Series are different FAILED test_base.py::TestIndexing::test_param_indexing - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="(0.3, 'a6')") are different FAILED test_base.py::TestAccessors::test_wrapper - AssertionError: Attributes of Series are different FAILED test_base.py::TestAccessors::test_apply_func_on_index - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="a3") are different FAILED test_base.py::TestAccessors::test_tile - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="a6") are different FAILED test_base.py::TestAccessors::test_repeat - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="a6") are different FAILED test_base.py::TestAccessors::test_align_to - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="('a7', 'a8')") are different FAILED test_base.py::TestAccessors::test_concat - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="('a', 'a6')") are different FAILED test_base.py::TestAccessors::test_apply_and_concat - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="a") are different FAILED test_base.py::TestAccessors::test_combine - AssertionError: Attributes of Series are different FAILED test_generic.py::TestAccessors::test_groupby_apply_on_matrix - AssertionError: DataFrame.index are different FAILED test_generic.py::TestAccessors::test_funcs[count-<lambda>-nancnt_nb] - AssertionError: Attributes of Series are different FAILED test_generic.py::TestAccessors::test_funcs[sum-<lambda>-nansum_nb] - AssertionError: Attributes of Series are different FAILED test_generic.py::TestAccessors::test_value_counts - AssertionError: DataFrame.index are different FAILED test_generic.py::TestAccessors::test_split - AssertionError: DataFrame.columns are different FAILED test_generic.py::TestAccessors::test_range_split - AssertionError: DataFrame.columns are different FAILED test_generic.py::TestAccessors::test_rolling_split - AssertionError: DataFrame.columns are different FAILED test_generic.py::TestAccessors::test_expanding_split - AssertionError: DataFrame.columns are different FAILED test_indicators.py::TestFactory::test_no_inputs - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="(0, 0)") are different FAILED test_indicators.py::TestFactory::test_input_shape - AssertionError: Attributes of Series are different FAILED test_indicators.py::TestFactory::test_no_inputs_and_params - AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="0") are different FAILED test_indicators.py::TestFactory::test_multiple_params - AssertionError: MultiIndex level [0] are different FAILED test_indicators.py::TestFactory::test_param_settings - AssertionError: MultiIndex level [0] are different FAILED test_indicators.py::TestBasic::test_OBV - AssertionError: Attributes of Series are different FAILED test_portfolio.py::TestFromOrders::test_group_by - AssertionError: Index are different FAILED test_portfolio.py::TestFromOrders::test_cash_sharing - AssertionError: Index are different FAILED test_portfolio.py::TestFromSignals::test_group_by - AssertionError: Index are different FAILED test_portfolio.py::TestFromSignals::test_cash_sharing - AssertionError: Index are different FAILED test_portfolio.py::TestFromRandomSignals::test_from_random_n - AssertionError: Index are different FAILED test_portfolio.py::TestFromOrderFunc::test_group_by[False-False] - AssertionError: Index are different FAILED test_portfolio.py::TestFromOrderFunc::test_group_by[False-True] - AssertionError: Index are different FAILED test_portfolio.py::TestFromOrderFunc::test_group_by[True-False] - AssertionError: Index are different FAILED test_portfolio.py::TestFromOrderFunc::test_group_by[True-True] - AssertionError: Index are different FAILED test_portfolio.py::TestFromOrderFunc::test_cash_sharing[False-False] - AssertionError: Index are different FAILED test_portfolio.py::TestFromOrderFunc::test_cash_sharing[False-True] - AssertionError: Index are different FAILED test_portfolio.py::TestFromOrderFunc::test_cash_sharing[True-False] - AssertionError: Index are different FAILED test_portfolio.py::TestFromOrderFunc::test_cash_sharing[True-True] - AssertionError: Index are different FAILED test_portfolio.py::TestPortfolio::test_stats - AssertionError: Attributes of Series are different FAILED test_records.py::TestMappedArray::test_to_pd - AssertionError: Series.index are different FAILED test_records.py::TestMappedArray::test_count - AssertionError: Attributes of Series are different FAILED test_records.py::TestMappedArray::test_value_counts - AssertionError: DataFrame.index are different FAILED test_signals.py::TestAccessors::test_pos_rank - AssertionError: Attributes of Series are different FAILED test_signals.py::TestAccessors::test_partition_pos_rank - AssertionError: Attributes of Series are different FAILED test_signals.py::TestAccessors::test_total - AssertionError: Attributes of Series are different FAILED test_signals.py::TestAccessors::test_total_partitions - AssertionError: Attributes of Series are different FAILED test_signals.py::TestAccessors::test_logical_funcs[<lambda>-<lambda>0] - AssertionError: DataFrame.columns are different FAILED test_signals.py::TestAccessors::test_logical_funcs[<lambda>-<lambda>1] - AssertionError: DataFrame.columns are different FAILED test_signals.py::TestAccessors::test_logical_funcs[<lambda>-<lambda>2] - AssertionError: DataFrame.columns are different FAILED test_signals.py::TestGenerators::test_RAND - AssertionError: DataFrame.columns are different FAILED test_signals.py::TestGenerators::test_RANDNX - AssertionError: DataFrame.columns are different FAILED test_utils.py::TestChecks::test_assert_dtype - AssertionError: Data type must be <class 'numpy.int32'>, not int64 ================================================= 56 failed, 790 passed, 142 warnings in 620.95s (0:10:20) =================================================

dougransom avatar Jul 06 '23 14:07 dougransom

Same behavior whether I installed vectorbt from pypi or as a local editable install 'pip install -e .'

dougransom avatar Jul 06 '23 14:07 dougransom

I cannot reproduce this behavior with Python 3.10 and Pandas 2.0.1. Only three tests fail, all being minor and requiring some changes to the testing procedure rather than functionality itself.

polakowo avatar Jul 09 '23 11:07 polakowo

Could it be OS Dependent? I am on Windows.

dougransom avatar Jul 11 '23 20:07 dougransom

I inspected a few of the errors, they seem to look like: `tests/test_indicators.py::TestBasic::test_OBV failed: self = <tests.test_indicators.TestBasic object at 0x0000016D6399F280>

def test_OBV(self):
  pd.testing.assert_series_equal(
        vbt.OBV.run(close_ts, volume_ts).obv,
        pd.Series(
            np.array([4, 7, 9, 10, 8, 5, 1]),
            index=close_ts.index,
            name=close_ts.name
        )
    )

E AssertionError: Attributes of Series are different E
E Attribute "dtype" are different E [left]: int64 E [right]: int32

tests\test_indicators.py:3007: AssertionError

`

Unfortunately I am not knowledgeable enough about the workings of VectorBT to attempt to solve. But I am guessing having an integer of an unexpected size is probably harmless.

dougransom avatar Jul 15 '23 18:07 dougransom

The failed tests are indeed harmless, Windows uses 32-bit integers by default, thus the output arrays have the data type np.int32 while they are being tested against the data type np.int64.

polakowo avatar Jul 15 '23 20:07 polakowo

check_dtype=False can be set in the assert. This tests will pass.

` def test_OBV(self): pd.testing.assert_series_equal( vbt.OBV.run(close_ts, volume_ts).obv, pd.Series( np.array([4, 7, 9, 10, 8, 5, 1]), index=close_ts.index, name=close_ts.name ), check_dtype=False

    )

`

Make sense to do that?

dougransom avatar Jul 16 '23 04:07 dougransom