Runtime error when using Python datetime with tzinfo as History request start and end times

[akosmaroy]

Expected Behavior

Framework should not through a Runtime error

Actual Behavior

Framework throws the following runtime error:

[ERROR] FATAL UNHANDLED EXCEPTION:Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.,   at Python.Runtime.NewReferenceExtensions.Borrow(NewReference& reference) in D:\QuantConnect\MyLean\pythonnet\src\runtime\Native\NewReference.cs:line 165,   at Python.Runtime.Converter.ToPrimitive(BorrowedReference value, Type obType, Object& result, Boolean setError, TypeCode tc, Boolean& usedImplicit) in D:\QuantConnect\MyLean\pythonnet\src\runtime\Converter.cs:line 771,   at Python.Runtime.Converter.ToManagedValue(BorrowedReference value, Type obType, Object& result, Boolean setError, Boolean& usedImplicit) in D:\QuantConnect\MyLean\pythonnet\src\runtime\Converter.cs:line 671,   at Python.Runtime.MethodBinder.Bind(BorrowedReference inst, BorrowedReference args, BorrowedReference kw, MethodBase info) in D:\QuantConnect\MyLean\pythonnet\src\runtime\MethodBinder.cs:line 632,   at Python.Runtime.MethodBinder.Invoke(BorrowedReference inst, BorrowedReference args, BorrowedReference kw, MethodBase info, MethodInfo[] methodinfo) in D:\QuantConnect\MyLean\pythonnet\src\runtime\MethodBinder.cs:line 868,   at Python.Runtime.MethodObject.Invoke(BorrowedReference target, BorrowedReference args, BorrowedReference kw, MethodBase info) in D:\QuantConnect\MyLean\pythonnet\src\runtime\Types\MethodObject.cs:line 73,   at Python.Runtime.MethodBinding.tp_call(BorrowedReference ob, BorrowedReference args, BorrowedReference kw) in D:\QuantConnect\MyLean\pythonnet\src\runtime\Types\MethodBinding.cs:line 243,runLauncher.sh: line 17:     7 Aborted                 (core dumped) dotnet QuantConnect.Lean.Launcher.dll --data-folder /Data --config /QuantConnect/backtesting/airlock/config.json --results-destination-folder /QuantConnect/backtesting/airlock/

Potential Solution

don't add tz info to the datetime objects:

        start_time = datetime(2024, 1, 8, 1, 0)
        end_time = datetime(2024, 1, 8, 8, 0)

But then how to ensure correct timezone?

Reproducing the Problem

Try to run this code:

# region imports
from AlgorithmImports import *
# endregion

from datetime import datetime, time
from datetime import timedelta
import pytz

class BreakoutAlgorithm(QCAlgorithm):
    def Initialize(self):
        self.SetStartDate(2024, 1, 8)  # Set the backtest start date
        self.SetEndDate(2024, 1, 8)  # Set the backtest end date
        self.SetCash(100000)  # Set the initial cash balance
        self.SetBrokerageModel(BrokerageName.InteractiveBrokersBrokerage, AccountType.Margin)

        self.symbol_minute = self.AddEquity("AMD", Resolution.Minute, extendedMarketHours=True,
                                            dataNormalizationMode=DataNormalizationMode.Raw).Symbol
        
        tzinfo=pytz.timezone('America/New_York')
        start_time = datetime(2024, 1, 8, 1, 0, tzinfo=tzinfo)
        end_time = datetime(2024, 1, 8, 8, 0, tzinfo=tzinfo)

        self.Debug(f"start_time: {start_time}, end_time: {end_time}")

        history = self.History([self.symbol_minute], start_time, end_time, Resolution.Minute, True, True)
        if not history.empty:
            self.Debug(f"history length: {len(history.loc[self.symbol_minute])}")
            self.Debug(f"first time: {history.loc[self.symbol_minute].head(1).index}")
            self.Debug(f"last time: {history.loc[self.symbol_minute].tail(1).index}")

    def OnData(self, data: Slice):
        pass

System Information

In the QuantConnect cloud system, using VS Code

Checklist

• [x] I have completely filled out this template
• [x] I have confirmed that this issue exists on the current master branch
• [x] I have confirmed that this is not a duplicate issue by searching issues

• [x] I have provided detailed steps to reproduce the issue

[Martin-Molinero]

Hey @akosmaroy! Start and end time are expected in algorithm TZ, can see more in our documentation https://www.quantconnect.com/docs/v2/writing-algorithms/historical-data/history-requests#02-Key-History-Concepts. Leaving the issue open though because it shouldn't cause an exception and runtime error

[akosmaroy]

agreed, one should still be able to set the expected timezone, as it's unclear what the default timezone is.