Forecasting Menu [Work in Progress]

[martinb-ai]

Description

• [x] Forecasting menu option
• [x] Forecasting Controller/model/view
• [x] Show dataset
• [x] Plot dataset
• [x] Combine dataset
• [x] Delete dataset column
• [x] Rename dataset column
• [x] Load dataset
• [x] Probabilistic Exponential Smoothing model
• [x] Theta model
• [x] Probabilistic Recurrent Neural Network (RNN, LSTM, GRU)
• [x] Block Recurrent Neural Network (RNN, LSTM, GRU) (feat. Past covariates)
• [x] Neural Bayesian Estimation (feat. Past covariates)
• [x] Temporal Convolutional Neural Network (feat. Past covariates)
• [x] Dataset cleaning
• [x] Dataset descriptive statistics
• [x] Feature engineering menu option
• [x] EMA
• [x] Stochastic Oscillator
• [x] RSI
• [x] ROC
• [x] Momentum
• [x] % Delta change
• [x] Average true range
• [x] simple moving average short/long window
• [x] price signal
• [x] Refresh loaded dataset on menu
• [x] Regression model (@colin99d )
• [x] Transformer model (@colin99d - not temporal)
• [x] Temporal Fusion transformer (performs weirdly)
• [x] Display user compute device (CPU vs. GPU)
• [x] Display user torch and darts version
• [x] Port over Arima (non torch) (@colin99d ) (Hide for now)
• [x] Port over knn (non torch) (@colin99d )
• [x] Port over MonteCarlo (non torch) (@colin99d)
• [x] Linear Regression (@colin99d )
• [x] Plot seasonality of data (@colin99d this technically "works" but graph looks terrible, reached out to @piiq )
  • Going to use our own version of their graphs for now, in the future might work with them to add functionality.
  • Also, submitted a PR to darts: https://github.com/unit8co/darts/pull/1056
• [x] Plot residuals from best_model (flag to set before training) (plot_residuals_analysis()) (@colin99d) --> Theta model does not work - make theta just print out that its not available. Also, theme needs a few tweaks. (@colin99d )
• [x] Plot dataset feature correlation coefficients
• [x] Pytest - Just make it run and successfully train (@colin99d )
• [x] Base class/ refactoring (@colin99d + @martinb-bb ) - 69.420% done for now...
• [x] Get rid of this printout: [2022-06-16 14:43:03,449] WARNING | darts.models | Support for AutoARIMA, BATS and TBATS is not available.To enable it, install u8darts[pmdarima] or u8darts[all]. [2022-06-16 14:43:03,450] WARNING | darts.models | Support for Facebook Prophet is not available. To enable it, install "darts", "u8darts[prophet]" or "u8darts[all]" (with pip);or "u8darts-all" (with conda). (fixed with PR https://github.com/unit8co/darts/pull/1010)
• [x] Remove external_axis from the docstrings or use the argument(@colin99d)
• [x] Glossary (@colin99d) (dict)
• [x] Confirm features work with Polygon and cryptodata (@colin99d)
  • For this PR we support daily, weekly, and monthly
  • https://github.com/unit8co/darts/issues/1057
• [x] Check if ram requirements match data size
• [x] plot only historical forecast (not entire series) (@colin99d)
• [x] Develop API (@colin99d)
• [x] Remove pred menus from the terminal (@colin99d )
• [x] Remove Tensorflow requirements (@colin99d )
• [x] Allow to bring any data from another menu to forecasting directly stocks/load AAPL/forecast/USE MODEL - works for both stocks and crypto menu. - to add economy soon or in V2
• [x] grab exported data from EXPORT folder as well (@colin99d)
• [x] standarize all models/views for ease of use in API
• [x] load mutliple stocks from stocks menu
• [x] Give combine warnings if it needs to use index
• [x] Add --all-covariates flag
• [x] Naive Bayes (for setting baseline)
• [x] If someone doesn't specify combine columns combine all columns
• [x] Rename menu to forecast
• [x] Make the disclaimer when we boot up terminal
• [x] Handle no darts gracefully
• [x] Fix hard-coded export path (this looks like a Darren specific issue)
• [x] Do not format non dollar amounts as dollar amounts
• [ ] ML documentation about networks
• [x] Standardize names in columns (default string for ticker)
• [x] Have a better way of handling target column not working

Other Bugs Fixed:

• Crypto export command does nothing
• Fixes #2128
• Fixes #2085
• Fixes #1088
• Fixes #1778
• Fixes #346

Checklist:

• [x] Update our Hugo documentation following these guidelines.
• [x] Update our tests following these guidelines.
• [x] Make sure you are following our CONTRIBUTING guidelines.
• [x] If a feature was added make sure to add it to the corresponding scripts file.
• [x] One letter parameters for common args
• [x] Handle regular days included with Business days when building timeseries
• [x] make disclaimer for loading menu
• [x] remove default csvs (@colin99d)
• [x] Notebooks with usage examples

Known issues

• [x] rnn residuals with aapl.csv does not work - fixed: https://github.com/unit8co/darts/pull/1066 to be added in 0.21.0
• [x] reset command (@colin99d )
• Make reset work when loading from another menu (im making this out of scope for now, if anyone legitimately wants this feature we can add in V2)
• [x] positive series with MAPE bug (https://github.com/unit8co/darts/issues/1063) - fixed
• [x] linregr and regr prints a lot of FutureWarning: pandas.Int64Index is deprecated warnings
• [x] Plot shows numbers instead of dates for X axis
• [x] Big dataframes print in an ugly fashion

V2 Features:

• Bring in Darts Datasets for further user experimentation
• Plot data in different ways (bar chart, box and whisker, 3D???)
• Static covariates
• stochastic forecasts capturing model uncertainty using Monte Carlo Dropout with deep learning models
• Load previously trained models for inference
• Train OpenBB NN model for off-the-shelf forecasting ( Long term goal))
• past covariates with different delta T
• Create a dashboard
• encoding string variables in columns
• hyperparameter tuning with Raytune: https://docs.ray.io/en/latest/tune/examples/tune-pytorch-lightning.html
• Do a better job at merging dataframes with different times (Polygon data can be 4 hours off)
• Refactor the views to use all the same code
• More descriptive feature engineering names (running the same feat eng on different columns replaces old values)

Others

• [x] I have performed a self-review of my own code.
• [x] I have commented my code, particularly in hard-to-understand areas.
• [x] My code passes all the checks pylint, flake8, black, ... To speed up development you should run pre-commit install.
• [x] New and existing unit tests pass locally with my changes. You can test this locally using pytest tests/....

To use in Jupyter: from openbb_terminal.api import forecast as fc from openbb_terminal import api

import matplotlib.pyplot as plt import matplotlib_inline.backend_inline

%matplotlib inline matplotlib_inline.backend_inline.set_matplotlib_formats("svg") api.theme.applyMPLstyle()

[DidierRLopes]

Someone has been avoiding sleep 👀

[jmaslek]

This is looking awesome. Small point I'm noticing. For controllers/argparse choices, y'all are doing --some_thing.

We should stick to doing --some-thing

[martinb-ai]

Gotcha, will update!

[deeleeramone]

https://github.com/OpenBB-finance/OpenBBTerminal/pull/2208 - FYI

[JerBouma]

@martinb-bb Could you write an "Introduction to" page for the Forecasting menu? See as an example the following: https://openbb-finance.github.io/OpenBBTerminal/terminal/stocks/ or https://openbb-finance.github.io/OpenBBTerminal/terminal/economy/.

This is crucial so users can get the hang of how the menu works. We have functionality within the terminal (about) that links the user automatically to the page:

[martinb-ai]

Yes, working on it!

Update- Intro page is done.