pandas-ta icon indicating copy to clipboard operation
pandas-ta copied to clipboard

bugfix in smoothing algo

Open mihakralj opened this issue 1 year ago • 13 comments

Initial interpretation of JMA was not calculating the volatility-based smoothing correctly, therefore creating less tight moving average. This bugfix for a main branch corrects most of the coding errors in JMA(). If needed, I can create a separate bugfix for development branch - but that one has rather different structures, math formulas and other elements.

mihakralj avatar Mar 29 '23 16:03 mihakralj

ah geez, pls uncomment line 76 if you want to cut out values within _period. JMA returns perfectly fine results within _period, so there is no need to replace them with NaN. But for consistency with all other indicators, you may want to do it.

mihakralj avatar Mar 29 '23 16:03 mihakralj

This is the most useful indicator. Interestingly, this version of JMA on TradingView is so much more responsive. Any idea why? JMA (update) by capissimo https://www.tradingview.com/script/hahAGARp-4-JMA-update/

alighten-dev avatar May 02 '23 06:05 alighten-dev

You will not like my answer... The best-documented algo for JMA is in this PDF: https://c.mql5.com/forextsd/forum/164/jurik_1.pdf And I most faithfully replicated the multi-stage filtering and smoothing; I cross-tested it with the proprietary .DLL from JurikRes and while not completely equivalent, my algo is calculating results within 2% of original JMA.

JMA calculates results in three stages - the first stage uses adaptive EMA, the second stage uses Kalman filter to dampen the jitter and the third stage uses a unique smoothing using alpha^2 to generate a smooth (kink-free) curve. Each phase generates its own intermediate result - ma1 is generated by phase1, ma2 by phase 2 and jma by final phase 3.

The algo on tradingView is also using three phases, but is discarding lots of dampening complexities that Scott Jurik built into JMA. For example, there is no phase parameter, no dynamic factor and no Jurik bands. Essentially, all smoothing/dampening qualities of JMA are discarded and make the implementation way more jittery and prone to overshooting - as a side-result of unmanaged responsiveness. It suffers the same issues as TEMA does (compared to EMA) - very responsive with unmanaged overshooting.

And while overshooting might be acceptable for a moving average, it is unbearable when you apply it to RSI (Jurik calls it RSX) or to DMI (Jurik calls this DMX)

Scott Jurik wrote a great study of this problem, worth reading: http://jurikres.com/down__/ma_evolv.pdf And another one, explaining the fine balance of JMA: http://jurikres.com/down__/why_jma.pdf

For conclusion: try using my implementation of JMA algo and add phase=100 parameter - the response becomes way more 'lively', but it overshoots more.

mihakralj avatar May 02 '23 15:05 mihakralj

I expected this answer :) I compared your implementation to the TradingView implementation, and TradingView seemed to gloss over some aspects. I'll try updating the Phase. I was under the impression a larger phase created more inertia and, therefore, less responsive. (http://jurikres.com/faq1/faq_ama.htm#phase)

Thank you for your hard work on this indicator.

alighten-dev avatar May 02 '23 16:05 alighten-dev

@twopirllc do we need anything else to accept this PR?

mihakralj avatar May 02 '23 22:05 mihakralj

I was under the impression a larger phase created more inertia and, therefore, less responsive.

You are looking at the wrong explanation of JMA phase. See http://jurikres.com/down__/product_guide_.pdf - page 10:

image

mihakralj avatar May 02 '23 23:05 mihakralj

Hi @mihakralj,

Thanks for providing additional information for JMA! 😎

do we need anything else to accept this PR?

Yeah. Checkout the development branch and make that the PR. 😎 I am not touching main until more work has been put into the development branch. I just wish I have had more free time this past year to do so. 😑

KJ

twopirllc avatar May 03 '23 01:05 twopirllc

... But this is a (serious) bug fix and should be rolled directly into main...

mihakralj avatar May 03 '23 04:05 mihakralj

Thank you for providing this additional information. This clears up a lot.

alighten-dev avatar May 03 '23 16:05 alighten-dev

@mihakralj,

I recall your testing situation with your C# TA cross testing. If it is high priority, then update your local version until I can update the development branch with a num(py/ba) implementation with your corrections.

twopirllc avatar May 04 '23 01:05 twopirllc

Already done, that's where this PR is sourcing from... 😊

mihakralj avatar May 04 '23 01:05 mihakralj

I am just embarrassed about two major mistakes in my python algo interpretation of JMA and want to hide/fix the shame. 😊 The only changes are in my own py file, everything else is untouched.

mihakralj avatar May 04 '23 01:05 mihakralj

@mihakralj

Already done, that's where this PR is sourcing from... 😊

I assumed you had. 😛

I am just embarrassed about two major mistakes in my python algo interpretation of JMA and want to hide/fix the shame. 😊

No need to be ashamed! All my mistakes are public as noted with all the Issues and PRs. 🤦🏼‍♂️

twopirllc avatar May 04 '23 02:05 twopirllc