Performance when computing single indexes on RSI with large datasets

[andyb1979]

Hey there!

First of all, this is a great library and wanted to thank you for creating it. I'm using Trady in an experimental backtesting engine for cryptocurrencies with charting visualisation by SciChart, which when I'm done I may open-source the backtester / visualisation to show how you can create a trading app with Trady.

Now I make heavy use of the Rsi indicator and for backtesting I want the best possible performance. It currently takes 3 minutes to backtest a years-worth of hourly bars and this is too long.

I optimised my code to compute only the latest index on new bar added to the backtest, like this:

        private void OnBarClosed(PriceBarDto p)
        {
            // Compute RSI
            // this.RSI is List<AnalyzableTick<decimal?>> and has been precomputed with historical data
            // at the start of the backtest. 
            // 
            // PrimaryPriceData is List<IOhlcv> with the price data 
            // 
            // Then OnBarClosed I want to compute just the latest index and add it to the list of RSI values
            this.Rsi.Add(new RelativeStrengthIndex(PrimaryPriceData, Args.RsiPeriod).Compute(PrimaryPriceData.Count - 1));

However, when there are about a thousand bars this results in nearly 1 million calls to GenericMovingAverage.ComputeCumulativeValue

I wonder if there's a way to improve the performance of this? For example a 14-period RSI only needs to compute 14-periods of a generic moving average to get the latest values.

Or ... a way to create some kind of a overload of indicator that allows you to Push or update a single IOhlc to the indicator

[andyb1979]

I added an implementation which is about 3x faster than standard Trady RSI. It can be better if I can incrementally update the points

    public class FastRsi<TOutput> : AnalyzableBase<IOhlcv, IOhlcv, decimal?, TOutput>
    {
        private IList<decimal?> _rsi = new List<decimal?>();

        public FastRsi(IEnumerable<IOhlcv> inputs, int period) : base(inputs, i => i)
        {
            _rsi.Add(null);
            int periodMinus1 = period - 1;
            decimal lastGain = 0, lastLoss = 0;
            decimal oneOverPeriod = 1 / period;
            // RMA_Gain=((Gain*(Period-1)) + RMA_Gain[i-1])/Period
            for (int i = 1; i < inputs.Count(); i++)
            {
                decimal change = inputs.ElementAt(i).Close - inputs.ElementAt(i-1).Close;
                decimal gain = change > 0 ? change : 0;
                decimal loss = change < 0 ? -change : 0;
                decimal rmaGain = ((lastGain * periodMinus1) + gain) / period;
                decimal rmaLoss = ((lastLoss * periodMinus1) + loss) / period;
                decimal rs = rmaLoss == 0 ? 100 : rmaGain / rmaLoss;
                decimal rsi = 100 - (100 / (1 + rs));
                _rsi.Add(i < period ? null : (decimal?)rsi);

                lastGain = rmaGain;
                lastLoss = rmaLoss;
            }
        }

        // Implement the compute algorithm, uses mappedInputs, index & backing indicators for computation here
        protected override decimal? ComputeByIndexImpl(IReadOnlyList<IOhlcv> mappedInputs, int index)
        {
            return _rsi[index];
        }
    }

    public class FastRsi : FastRsi<AnalyzableTick<decimal?>>
    {
        public FastRsi(IEnumerable<IOhlcv> inputs, int period) : base(inputs, period)
        {
        }
    }

I've tested this vs. Trady RSI and it gets results after the stabilisation period (about 100 bars to stabilise when Period=14) similar to the built-in Trady RSI.

[andyb1979]

Update. I have an implementation of RSI which is about 1,000 times faster than the Trady built-in RSI, but requires that you update/add a single point via the AddOhlcv() method.

This is therefore not immutable and breaks Trady's immutability which also makes Trady thread safe, but I'm willing to take the risk for the extra speed that this offers. It makes some backtests I'm running which rely on RSI considerably faster.

Here's the code :) (sorry about the reflection hax but these fields are not accessible and probably for good reason)

    public class FastRsi<TOutput> : AnalyzableBase<IOhlcv, IOhlcv, decimal?, TOutput>
    {
        private readonly List<IOhlcv> _inputs;
        public int Period { get; }
        private List<decimal?> _rsi = new List<decimal?>();
        private int _periodMinus1;
        private List<DateTimeOffset> _dateTimes;
        private decimal _lastGain = 0;
        private decimal _lastLoss = 0;

        public FastRsi(IEnumerable<IOhlcv> inputs, int period) : base(inputs, i => i)
        {
            _inputs = inputs.ToList();
            _dateTimes = _inputs.Select(x => x.DateTime).ToList();
            Period = period;
            _rsi.Add(null);
            _periodMinus1 = period - 1;

            // RMA_Gain=((Gain*(Period-1)) + RMA_Gain[i-1])/Period
            for (int i = 1; i < _inputs.Count; i++)
            {
                decimal change = _inputs[i].Close - _inputs[i-1].Close;
                decimal gain = change > 0 ? change : 0;
                decimal loss = change < 0 ? -change : 0;
                decimal rmaGain = ((_lastGain * _periodMinus1) + gain) / period;
                decimal rmaLoss = ((_lastLoss * _periodMinus1) + loss) / period;
                decimal rs = rmaLoss == 0 ? 100 : rmaGain / rmaLoss;
                decimal rsi = 100 - (100 / (1 + rs));
                _rsi.Add(i < period ? null : (decimal?)rsi);

                _lastGain = rmaGain;
                _lastLoss = rmaLoss;
            }
        }

        public FastRsi<TOutput> AddOhlcv(IOhlcv ohlc)
        {
            _inputs.Add(ohlc);
            _dateTimes.Add(ohlc.DateTime);
            IReadOnlyList<IOhlcv> mappedInputs = _inputs;
            IReadOnlyList<DateTimeOffset> mappedDateTimes = _dateTimes;
            
            // Trady base class needs these to be able to compute a single index. 
            // TODO: Modify or create an AnalyzableBase that can be modified. 
            typeof(FastRsi)
                .GetField("_mappedInputs", BindingFlags.Instance | BindingFlags.NonPublic)
                .SetValue(this, mappedInputs);

            typeof(AnalyzableBase<IOhlcv, IOhlcv, decimal?, TOutput>)
                .GetField("_mappedDateTimes", BindingFlags.Instance | BindingFlags.NonPublic)
                .SetValue(this, mappedDateTimes);

            int i = _mappedInputs.Count - 1;
            decimal change = _mappedInputs[i].Close - _mappedInputs[i - 1].Close;
            decimal gain = change > 0 ? change : 0;
            decimal loss = change < 0 ? -change : 0;
            decimal rmaGain = ((_lastGain * _periodMinus1) + gain) / Period;
            decimal rmaLoss = ((_lastLoss * _periodMinus1) + loss) / Period;
            decimal rs = rmaLoss == 0 ? 100 : rmaGain / rmaLoss;
            decimal rsi = 100 - (100 / (1 + rs));
            _rsi.Add(i < Period ? null : (decimal?)rsi);

            _lastGain = rmaGain;
            _lastLoss = rmaLoss;

            return this;
        }

        protected override decimal? ComputeByIndexImpl(IReadOnlyList<IOhlcv> mappedInputs, int index)
        {
            return _rsi[index];
        }
    }

    public class FastRsi : FastRsi<AnalyzableTick<decimal?>>
    {
        public FastRsi(IEnumerable<IOhlcv> inputs, int period) : base(inputs, period)
        {
        }
    }

Usage

// compute the RSI results like normal when you have an input list of IOHLCV
var fastRsi = new FastRsi(inputOhlc, period);
List<decimal?> rsiResults = fastRsi.Compute().ToList();

// If you get a bar closed and have 1 more IOHLCV to compute then just add it like this
rsiResults.Add(fastRsi.AppendOhlc(nextOhlcBar).Compute(count-1));