[ENH] Add adapter for `arch` package.

[ngupta23]

Is your feature request related to a problem? Please describe.

Financial modeling usually deals with modeling the volatility of time series data, ARCH, GARCH, etc. are common models used for this purpose.

Describe the solution you'd like

Add support for https://github.com/bashtage/arch (as an adapter) in sktime. This will be very useful for folks in the financial industry. This is detailed in this video as well for reference: https://www.youtube.com/watch?v=74rDhJexmTg.

Describe alternatives you've considered

None available at this time.

Additional context

• Refer to the video above to see how these models are used.
• Further examples can be found in the repo's example folder: https://github.com/bashtage/arch/tree/main/examples

[fkiraly]

Ah, yes! Now that we have probabilistic predictions, makes a lot of sense! And why not the entire zoo while we're at it!

I think this is a good first issue, our extension template should outline how to get these in.

Although, probably makes sense to implement primarily predict_var, the variance prediction interface, which is not yet merged (but will soon be). https://github.com/alan-turing-institute/sktime/pull/2100

On the other hand, the predict_quantiles interface should be ready to go for an implementation attempt.

Making this a "good first issue", since it seems to be straightforward interfacing.

[Vasudeva-bit]

Following the extension template, I will work on this

[fkiraly]

Excellent, thanks, @Vasudeva-bit

[fkiraly]

@Vasudeva-bit, we've reworked the "estimator implementation" and "dependency handling" guideliens recently. You can refer to these work-in-progress docs until they become available in the next release: https://sktime--2186.org.readthedocs.build/en/2186/developer_guide/add_estimators.html https://sktime--2189.org.readthedocs.build/en/2189/developer_guide/dependencies.html

[Vasudeva-bit]

Thanks @fkiraly, these references are very helpful!

[rohan-gt]

Any update on this enhancement?

[fkiraly]

hmm, @rohan-gt, history is, I think:

• @Vasudeva-bit started this here https://github.com/alan-turing-institute/sktime/pull/2282 but then abandoned it
• @lbventura recently added a lot of proba interfaces and estimator interfaces, so I asked him to look at the abandoned PR whether it is easy to wrap up
• the jury is still out on that - what do you think, @lbventura? what still needs doing? is this easy/hard?

[lbventura]

I still have not looked at this closely because I have an exam Saturday. I will take a closer look to it during the weekend. Can comment that the task is probably easy if the arch package already has a method to compute probabilistic forecasts, in which case we would only have to adapt it to our purposes (as I did for statsmodels). If it is not implemented, we would need to do so from first principles: again, statsmodels would be a good place to see how this is built.

[lbventura]

I still have not looked at this closely because I have an exam Saturday. I will take a closer look to it during the weekend. Can comment that the task is probably easy if the arch package already has a method to compute probabilistic forecasts, in which case we would only have to adapt it to our purposes (as I did for statsmodels). If it is not implemented, we would need to do so from first principles: again, statsmodels would be a good place to see how this is built.

After taking a look into arch, it looks like one would need to implement the method from first principles.

[fkiraly]

ouch - is that really so? Sorry for asking, you may well be right, but that would make the arch package a bit strange.

Isn't arch only interesting if you can also get the heteroscedastic noise/variance terms out of it?

[gerardkr]

Has there been any progress on adding arch?

[fkiraly]

@Vasudeva-bit, I saw you closed your PR - any reason for this? It looks like we were close.

FYI @gerardkr, the statsmodels versions have been interfaced.

[Vasudeva-bit]

I need some help to test the code on my system. I don't find any resources online. Kindly help me with some references to test the code to pass CI/CD tests. I got stuck with that. @fkiraly

[fkiraly]

ohhh, I see! You should have said something! (or if you did and no one reacted, feel free to always ping us on discord, the dev-chat channel! link here: https://discord.com/invite/54ACzaFsn7)

We have recently reworked our documentation on the topic of describing how you set up your local testing setup, hope this helps: https://www.sktime.net/en/latest/developer_guide/continuous_integration.html

If you are unsure and/or get stuck don't worry: we have regular meet-ups on Fridays (see schedule on discord), it is common that new contributors show up and we help them get setup via screenshare.

[Vasudeva-bit]

I will go through and try to setup tests. If I succeed, I shall try to test the arch model, otherwise will join next meeting. Thanks @fkiraly

[Vasudeva-bit]

I see an arch.py has already been implemented, interface of arch from statsmodel python package. However my version was using arch python package. Shall I continue with the work, if yes? Let me know to how to name the file. Both of these interfaces are univariate. There is a python package mgarch for multivariate forecasting.

I would like to know to add any new dependency/package, is that just adding the dependency in requirements.txt and reloading project or directly pip install into my conda env. @fkiraly

[fkiraly]

However my version was using arch python package. Shall I continue with the work, if yes?

Of course! The philosophy of sktime is that we let the user decide which package or backend they want to use. Have a look at the various ARIMA models already in sktime, this would be very similar.

(we have ARIMAs from statsmodels, from pmdarima, and from statsforecast)

Both of these interfaces are univariate. There is a python package mgarch for multivariate forecasting.

That might also be nice!

I would like to know to add any new dependency/package, is that just adding the dependency in requirements.txt and reloading project or

Have a look at the "adding estimator" developer guide: https://www.sktime.net/en/latest/developer_guide/add_estimators.html

You would have to add the soft dependencies in pyproject.toml. If soft deps are widely used, you can add them to forecasting and all_extras; otherwise just to all_extras. In either case, to all_extras_pandas2 as well, if they are pandas 2 compatible.

[Vasudeva-bit]

I was able to fit the estimator and forecast manually. By this, I mean, I checked whether it can be used by importing from sktime.forecasting (not ci/cd tests). I was also able to use check_estimator to test the estimator. Some of tests failed.

I think these errors are mostly due to forecasting index returned by this estimator not matching with expected forecasting index for various ys (i.e., ys with different index types).

There's is an ambiguity between sktime.forecasting.arch and arch python package while importing. When I include arch_model interface of arch package inside sktime.forecasting.arch (already existing estimator, arch interface from statsmodel), it's causing import cycle dependency. I thought of renaming arch.py in sktime.forecasting to uarch.py (meaning univariate) to resolve this but implementing this would require changes in documentation, and other things. Shall I rename? @fkiraly

[fkiraly]

Shall I rename?

There already is an arch module, I would recommend replacing this by a folder with an init that exports the currently existing statsforecast based models, and the new arch package based ones.