Lean.Brokerages.InteractiveBrokers icon indicating copy to clipboard operation
Lean.Brokerages.InteractiveBrokers copied to clipboard

IBKR Data Downloader Exception

Open jaredbroad opened this issue 1 year ago • 9 comments

Expected Behavior

Able to download options data on IBKR

Actual Behavior

Runtime exception looking up IBKR Symbols

20240514 21:08:17.198 TRACE:: InteractiveBrokersBrokerage.LookupSymbols(): Requesting symbol list for AAPL ... 20240514 21:08:17.218 ERROR:: <>c__DisplayClass9_0.b__0(): System.NullReferenceException: Object reference not set to an instance of an object. at QuantConnect.Brokerages.InteractiveBrokers.InteractiveBrokersBrokerage.LookupSymbols(Symbol symbol, Boolean includeExpired, String securityCurrency) at QuantConnect.ToolBox.IBDownloader.IBDataDownloader.GetChainSymbols(Symbol symbol, Boolean includeExpired) at QuantConnect.ToolBox.IBDownloader.IBDataDownloader.Get(DataDownloaderGetParameters dataDownloaderGetParameters)

Potential Solution

Review and address

Reproducing the Problem

Excellent post here: https://www.quantconnect.com/forum/discussion/17071/download-and-backtest-options-data-locally-with-lean-cli-and-interactive-brokers/p1

Checklist

  • [x] I have completely filled out this template
  • [x] I have confirmed that this issue exists on the current master branch
  • [x] I have confirmed that this is not a duplicate issue by searching issues
  • [x] I have provided detailed steps to reproduce the issue

jaredbroad avatar May 15 '24 11:05 jaredbroad

You can use this algo to reproduce it.

BreakoutCallBuySample.cs

using System;
using System.Linq;
using QuantConnect;
using QuantConnect.Util;
using QuantConnect.Algorithm;
using QuantConnect.Indicators;
using QuantConnect.Data;
using QuantConnect.Data.Market;
using QuantConnect.Orders;
using QuantConnect.Securities;

// Source: https://www.quantconnect.com/learning/task/257/Options-Trading
public class Main : QCAlgorithm
{
    private Symbol _equity;
    private OptionContract _call;
    private Maximum _high;

    public override void Initialize()
    {
        SetStartDate(2024, 5, 6);
        SetEndDate(2024, 5, 14);

        SetCash(100000);

        _equity = AddEquity("AAPL", Resolution.Minute).Symbol;
        Securities[_equity].SetDataNormalizationMode(DataNormalizationMode.Raw);
        SetBenchmark(_equity);

        var option = AddOption("AAPL", Resolution.Minute);
        option.SetFilter(-3, 3, TimeSpan.FromDays(20), TimeSpan.FromDays(40));

        _high = MAX(_equity, 21, Resolution.Daily, Field.High);
    }

    public override void OnData(Slice data)
    {
        if (!_high.IsReady)
            return;

        var optionInvested = Portfolio.Keys
            .Where(x => Portfolio[x].Invested && Portfolio[x].Type == SecurityType.Option)
            .ToList();

        if (optionInvested.Any())
        {
            if (Time + TimeSpan.FromDays(4) > optionInvested[0].ID.Date)
            {
                Liquidate(optionInvested[0], "Too close to expiration");
            }
            return;
        }

        if (Securities[_equity].Price >= _high.Current.Value)
        {
            foreach (var optionChain in data.OptionChains)
            {
                var chains = optionChain.Value;
                BuyCall(chains);
            }
        }
    }

    private void BuyCall(OptionChain chains)
    {
        var expiry = chains.OrderBy(x => x.Expiry).Last().Expiry;
        var calls = chains.Where(x => x.Expiry == expiry && x.Right == OptionRight.Call);
        var callContracts = calls.OrderBy(x => Math.Abs(x.Strike - x.UnderlyingLastPrice));

        if (!callContracts.Any())
            return;

        _call = callContracts.First();
        var quantity = Portfolio.TotalPortfolioValue / _call.AskPrice;
        quantity = Math.Floor(0.05m * quantity / 100);
        Buy(_call.Symbol, quantity);
    }

    public override void OnOrderEvent(OrderEvent orderEvent)
    {
        var order = Transactions.GetOrderById(orderEvent.OrderId);
        if (order.Type == OrderType.OptionExercise)
        {
            Liquidate();
        }
    }
}

omidkrad avatar May 15 '24 19:05 omidkrad

Update: at the moment IB does not provide the option symbol chain, so we can't source the symbol list to download, causing the error reported

Martin-Molinero avatar May 16 '24 12:05 Martin-Molinero

Thank you @Martin-Molinero for the update. I think we should show that in the brokerages page that which brokers support requesting option chain, in addition to support for trading options.

Looking at IBKR docs they have an API to provide option chain: https://ibkrcampus.com/ibkr-api-page/twsapi-doc/#option-chain

omidkrad avatar May 16 '24 19:05 omidkrad

I noticed there are two calls to the reqContractDetails method in InteractiveBrokersBrokerage.cs I tried to set breakpoints on them, the process attaches but these calls didn't get hit.

omidkrad avatar May 16 '24 19:05 omidkrad

You can reduce the repro project to the following:

using System;
using QuantConnect;
using QuantConnect.Algorithm;
using QuantConnect.Data;

public class Main : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2024, 5, 13);
        SetEndDate(2024, 5, 15);
        SetCash(100000);

        var equity = AddEquity("SPY", Resolution.Minute).Symbol;
        Securities[equity].SetDataNormalizationMode(DataNormalizationMode.Raw);
        SetBenchmark(equity);

        var option = AddOption("SPY", Resolution.Minute);
        option.SetFilter(-5, 5, TimeSpan.FromDays(20), TimeSpan.FromDays(40));
    }

    public override void OnData(Slice data)
    {
        foreach (var (symbol, optionChain) in data.OptionChains)
        {
            foreach (var option in optionChain)
            {
                Log($"Received {option}");
            }
        }
    }
}

omidkrad avatar May 16 '24 20:05 omidkrad

Related issues for IQFeed and Polygon:

  • IQFeed https://github.com/QuantConnect/Lean.DataSource.IQFeed/issues/13
  • Polygon https://github.com/QuantConnect/Lean.DataSource.Polygon/issues/17

omidkrad avatar Jul 21 '24 19:07 omidkrad

Since a recent update in August I'm getting a different error when running local backtesting:

lean backtest .\MyOptionAlgo --data-provider-historical "Interactive Brokers"

Error:

Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an
invocation.
 ---> System.ArgumentException: Unable to locate any exports matching the requested typeName:
QuantConnect.DownloaderDataProvider.Launcher.Models.BrokerageDataDownloader (Parameter 'typeName')
   at QuantConnect.Util.Composer.GetExportedValueByTypeName[T](String typeName, Boolean forceTypeNameOnExisting) in
/LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Common/Util/Composer.cs:line 317
   at QuantConnect.Lean.Engine.DataFeeds.DownloaderDataProvider..ctor() in
/LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/Engine/DataFeeds/DownloaderDataProvider.cs:line 57

omidkrad avatar Aug 07 '24 21:08 omidkrad

Hey @omidkrad! This issue has been fixed already, please pull the latest lean image 👍

Martin-Molinero avatar Aug 07 '24 21:08 Martin-Molinero

Awesome! Thanks, please go ahead and close the issue.

omidkrad avatar Aug 26 '24 17:08 omidkrad