Comparing TA libraries

[mihakralj]

I wanted to share my experiment this week when I tried to add more validation tests across TA libraries. I wanted to compare four .NET TA libraries - not so much for the same results, but to check the performance of each of them. Here is the setup in my .NET interactive notebook, showing which four libraries I pulled in:

#r "nuget:QuanTAlib"
#r "nuget:Skender.Stock.Indicators"
#r "nuget:Trady.Analysis"
#r "nuget:Trady.Core"
#r "nuget: OoplesFinance.StockIndicators"

using QuanTAlib;
using Skender.Stock.Indicators;
using OoplesFinance.StockIndicators.Models;
using static OoplesFinance.StockIndicators.Calculations;
using Trady.Analysis;
using Trady.Core;

Let's start with the basics - a Simple Moving Average across 10,000 bars. I had to tinker with transforms of data input to make each library happy and supply the source in just perfect format:

RND_Feed bars = new(10000, volatility: 0.2, startvalue: 500);
int p = 5;

// preparing data for each TA library
var skender_bars = bars.Select(q => new Quote{ Date = q.t, Open = (decimal)q.o, High = (decimal)q.h, Low = (decimal)q.l, Close = (decimal)q.c, Volume = (decimal)q.v });
var ooples_bars = new StockData(bars.Select(x => (decimal)x.o), bars.Select(x => (decimal)x.h), bars.Select(x => (decimal)x.l), bars.Select(x => (decimal)x.c), bars.Select(x => (decimal)x.v), bars.Select(x => x.t));
var trady_bars = bars.Select(q => (decimal)q.c );

// calculating indicators
SMA_Series ma = new(bars.Close, period: p);
var skender = skender_bars.GetSma(p).ToList();
var ooples = ooples_bars.CalculateSimpleMovingAverage(length:p).OutputValues["Sma"];
var trady = new Trady.Analysis.Indicator.SimpleMovingAverageByTuple(values: trady_bars, periodCount: p);

Console.Write($"SMA - Simple Moving Average\nFirst:\n");
Console.Write($"QuanTAlib:\t{ma[p+1].v:f7}\n  Skender:\t{skender[p+1].Sma:f7} \n   Trady:\t{trady[p+1]:f7}  \n  Ooples:\t{ooples[p+1]:f7} \n\n");
Console.Write($"Last:\n");
Console.Write($"QuanTAlib:\t{ma.Last().v:f7}\n  Skender:\t{skender.Last().Sma:f7} \n    Trady:\t{trady[ma.Count-1]:f7}\n   Ooples:\t{ooples.Last():f7} \n");

And here is the comparative output, comparing the first valid and last available value:

SMA - Simple Moving Average
First:
QuanTAlib:	407.7740000
  Skender:	407.7740000 
   Trady:	407.7740000  
  Ooples:	407.7740000 

Last:
QuanTAlib:	6009.5320000
  Skender:	6009.5320000 
    Trady:	6009.5320000
   Ooples:	6009.5320000 

So far so good, except when we start comparing more complex indicators - and results are uncomfortably different:

HMA - Hull Moving Average
First:
QuanTAlib:	487.6926667
  Skender:	487.6926667 
   Trady:	487.6926667  
  Ooples:	454.6595000 

Last:
QuanTAlib:	513.1464444
  Skender:	513.1464444 
    Trady:	513.1464444
   Ooples:	528.1782000 

Below is EMA, where only QuanTAlib and Skender are identical, other two libraries are returning very odd results...

EMA - Exponential Moving Average
First:
QuanTAlib:	521.4124444
  Skender:	521.4124444 
   Trady:	506.5420000  
  Ooples:	482.6330000 

Last:
QuanTAlib:	2454.2156816
  Skender:	2454.2156816 
    Trady:	2491.0260000
   Ooples:	2454.2157000 

[DaveSkender]

There’s a very good reason why I don’t source my formulas from other people’s code 😬. I primarily rely on formulas from original authors and don’t apply any “creative improvements” and then do way more manual and automated testing than most.

Though, much of these differences can probably be attributed to compounding or other rounding errors due to coding techniques, data types, or conversions. I’ve done some thinking and experimentation on data precision, so I’ve seen how this can go wrong if not careful.

[mihakralj]

That is exactly why I down-selected validation of my library against three libraries:

• Original TA-LIB (lifted to .NET Core, but the code is still good old C++)
• Kevin's Pandas-TA (the most valid Python TA library that I know)
• yours

I am struggling to create reliable unit tests in C# that would call Kevin's Python library through Python.net - so I was looking for alternative validation library. Trady and Ooples looked decent. Well, not anymore. :-)

[DaveSkender]

Well, I’m happy to have made the cut 😀
If you see anything odd in your comparison journey, let us know! For as much attention to detail and testing we do, it’s still always possible to sneak some faults past the goalie.

[DaveSkender]

@mihakralj, no performance benchmarking yet?

I’m pretty sure I can beat most Python projects and most of the other C# projects. I’m not sure how we’d compare to native C++ projects, but these newer .NET frameworks are much faster overall, might be close.

[mihakralj]

If you want to go down the optimization rabbit hole using SIMD in C#: https://docs.microsoft.com/en-us/dotnet/standard/simd

[DaveSkender]

Yah, for sure incremental will always outperform in certain scenarios when buffering or memory management becomes a drag with larger datasets. I suspect that’s more prevalent on certain indicators or if you’re trying to literally regenerate a whole series over and over again with incrementally more data?

I kinda also figured you’d eventually get to that hybrid model. I’ll be doing the same. It makes sense to start with a meaningful base then just add to it.

[DaveSkender]

SIMD looks nuts 🤪

[DaveSkender]

But not as wild as GPU acceleration

[DaveSkender]

I tend to favor usability more than hyper-optimization purely to eek out small gains in speed, though do respect those 0.1% use cases where that would be nice, like HFT. Anybody doing HFT has the time, money, and incentive.