qlib icon indicating copy to clipboard operation
qlib copied to clipboard

It use default param freq='day' for create Account obj in Backtest.

Open ittianyu opened this issue 1 year ago • 2 comments

🐛 Bug Description

I only have 60min freq data. When I try to backtest with 60min. It report can't find day data. So I look into the stack, and found the reason that it use default param freq='day' for create Account obj.

I hope you could add a 'freq' in backtest config, and convert it into Account obj.

To Reproduce

Steps to reproduce the behavior:

user code

freq = '60min'

# backtest and analysis
with R.start(experiment_name="backtest_analysis"):
    recorder = R.get_recorder(recorder_id=rid, experiment_name="train_model")
    model = recorder.load_object("trained_model")

    # prediction
    recorder = R.get_recorder()
    ba_rid = recorder.id
    sr = SignalRecord(model, dataset, recorder)
    sr.generate()

    # backtest & analysis
    par = PortAnaRecord(recorder, port_analysis_config, freq, freq)
    par.generate()

framework code

class PortAnaRecord(ACRecordTemp):

    def _generate(self, **kwargs):


        # custom strategy and get backtest
        portfolio_metric_dict, indicator_dict = normal_backtest(
            executor=self.executor_config, strategy=self.strategy_config, **self.backtest_config
        )
def backtest(
    start_time: Union[pd.Timestamp, str],
    end_time: Union[pd.Timestamp, str],
    strategy: Union[str, dict, object, Path],
    executor: Union[str, dict, object, Path],
    benchmark: str = "SH000300",
    account: Union[float, int, dict] = 1e9,
    exchange_kwargs: dict = {},
    pos_type: str = "Position",
) -> Tuple[PORT_METRIC, INDICATOR_METRIC]:
    trade_strategy, trade_executor = get_strategy_executor(
        start_time,
        end_time,
        strategy,
        executor,
        benchmark,
        account,
        exchange_kwargs,
        pos_type=pos_type,
    )
    return backtest_loop(start_time, end_time, trade_strategy, trade_executor)
def get_strategy_executor(
    start_time: Union[pd.Timestamp, str],
    end_time: Union[pd.Timestamp, str],
    strategy: Union[str, dict, object, Path],
    executor: Union[str, dict, object, Path],
    benchmark: Optional[str] = "SH000300",
    account: Union[float, int, dict] = 1e9,
    exchange_kwargs: dict = {},
    pos_type: str = "Position",
) -> Tuple[BaseStrategy, BaseExecutor]:
    # NOTE:
    # - for avoiding recursive import
    # - typing annotations is not reliable
    from ..strategy.base import BaseStrategy  # pylint: disable=C0415
    from .executor import BaseExecutor  # pylint: disable=C0415

    trade_account = create_account_instance(
        start_time=start_time,
        end_time=end_time,
        benchmark=benchmark,
        account=account,
        pos_type=pos_type,
    )

def create_account_instance(
    start_time: Union[pd.Timestamp, str],
    end_time: Union[pd.Timestamp, str],
    benchmark: Optional[str],
    account: Union[float, int, dict],
    pos_type: str = "Position",
) -> Account:
    return Account(
        init_cash=init_cash,
        position_dict=position_dict,
        pos_type=pos_type,
        benchmark_config=(
            {}
            if benchmark is None
            else {
                "benchmark": benchmark,
                "start_time": start_time,
                "end_time": end_time,
            }
        ),
    )
class Account:
    def __init__(
        self,
        init_cash: float = 1e9,
        position_dict: dict = {},
        freq: str = "day",
        benchmark_config: dict = {},
        pos_type: str = "Position",
        port_metr_enabled: bool = True,
    ) -> None:

There exists param 'freq' in the Account init method. But I can't change it from outer.

Expected Behavior

I hope you could add a 'freq' in backtest config, and convert it into Account obj.

Screenshot

Environment

Note: User could run cd scripts && python collect_info.py all under project directory to get system information and paste them here directly.

  • Qlib version:
  • Python version:
  • OS (Windows, Linux, MacOS):
  • Commit number (optional, please provide it if you are using the dev version):

Additional Notes

ittianyu avatar Aug 31 '24 18:08 ittianyu

Same issue working with 60min (hourly) dataset for intraday hourly strategy. Backtesting continually fails as it tries to resample to 1min and day frequencies without config to override.

What is the correct way to implement hourly trading strategy with hourly (60min) data?

This is for 24/7 crypto trading strategy

I have tried configuring the executor in PortAnaRecord config, but does not seem to effect the resampling

executor:
        class: SimulatorExecutor
        module_path: qlib.backtest.executor
        kwargs:
            time_per_step: 60min
            generate_portfolio_metrics: true

It is unclear if backtesting / portfolio feq should remain "day" or all should be unified to "60min".

The highfreq example shows many nested executors for multiple freq, but it is unclear how they are related to dataset freq, resampling, account, and strategy. nested execution https://github.com/microsoft/qlib/blob/94d138ec230e299355eeff6fe0df7439a6de4ad1/examples/nested_decision_execution/workflow.py

https://github.com/microsoft/qlib/blob/94d138ec230e299355eeff6fe0df7439a6de4ad1/qlib/utils/resam.py#L72-L99

get_higher_eq_freq_feature always called with "day" which attempts to resample down to 1min?

https://github.com/microsoft/qlib/blob/94d138ec230e299355eeff6fe0df7439a6de4ad1/qlib/workflow/record_temp.py#L374-L464

I have tried forcing the other freq in PortAnaRecord

extending the RD-Agent config_baseline.yaml, as baseline test to get it working

port_analysis_config: &port_analysis_config
    strategy:
        class: TopkDropoutStrategy
        module_path: qlib.contrib.strategy
        kwargs:
            signal: <PRED>
            topk: 50
            n_drop: 5
    backtest:
        start_time: 2023-06-01
        end_time: 2025-07-31
        account: 10000
        benchmark: *benchmark
        exchange_kwargs:
            freq: 60min
            trade_unit: 1,
            deal_price: close
            # kraken pro maker (0.25%) and taker (0.4%)
            open_cost: 0.0025
            close_cost: 0.0040
            min_cost: 5
    executor:
        class: SimulatorExecutor
        module_path: qlib.backtest.executor
        kwargs:
            time_per_step: 60min
            generate_portfolio_metrics: true
    risk_analysis_freq: 60min
    indicator_analysis_freq: 60min

...

    record:
        - class: SignalRecord
          module_path: qlib.workflow.record_temp
          kwargs:
            model: <MODEL>
            dataset: <DATASET>
        - class: SigAnaRecord
          module_path: qlib.workflow.record_temp
          kwargs:
            ana_long_short: False
            ann_scaler: 252
        - class: PortAnaRecord
          module_path: qlib.workflow.record_temp
          kwargs:
            config: *port_analysis_config

Similar to:

  • #1122
  • #721

JeremyJonas avatar Sep 11 '25 05:09 JeremyJonas

Still not confident I fully understand account vs strategy vs backtesting vs data freq, but in terms of Account defaulting to day causing resampling issues for backtesting, I have confirmed that forcing the account for backtesting to match the data freq (in my case 60min), the backtesting runs as expected.

https://github.com/microsoft/qlib/blob/94d138ec230e299355eeff6fe0df7439a6de4ad1/qlib/backtest/init.py#L161-L174

Monkey-patched locally to:

    return Account(
        init_cash=init_cash,
        position_dict=position_dict,
        pos_type=pos_type,
        freq="60min", # FORCE ACCOUNT FREQ, BEFORE IS MISSING WHICH DEFAULTS TO DAY
        benchmark_config=(
            {}
            if benchmark is None
            else {
                "benchmark": benchmark,
                "start_time": start_time,
                "end_time": end_time,
            }
        ),
    )

JeremyJonas avatar Sep 11 '25 06:09 JeremyJonas