How can ARIMA do one step predict with new test data?

[reminia]

I trained a ARIMA model with two weeks data.

Now I want to use the model to do one step predict with my new test data.

But I dont know how to feed my test data to the predict function.

http_march_train = http_ratio['2017-03-01': '2017-03-14'] # collect every 15minutes
start = '2017-03-15 00:00'
end = '2017-03-15 23:45'
http_march_test = http_ratio[start: end]  

arima = smt.ARIMA(http_march_train, (3, 0, 3))
model = arima.fit()

# this is a dynamic forecast of the 15th day
model.predict(start = start, end = end)

# How can I feed the http_march_test data to predict to do an in sample one-step new forecast?

[ChadFulton]

If you are using Statsmodels 0.8, then my answer here https://github.com/statsmodels/statsmodels/issues/2788#issuecomment-175209448 shows how to do this using the SARIMAX model. As far as I know this use case isn't supported by our ARIMA.

[reminia]

@ChadFulton As your comment, two model needed to predict based on new data.

But what if I want to use a fixed model for a long time, and to predict with

new data supplied, so I have to train a new model every new forward predict?

[ChadFulton]

It's not clear to me what you want to do. If you have two datasets and you want to fit using dataset A and then use the estimated parameters to perform prediction on dataset B, then you would do:

modelA = sm.tsa.SARIMAX(dataset_A, order=(3, 0, 3))
resA = modelA.fit()

modelB = sm.tsa.SARIMAX(dataset_B, order=(3, 0, 3))
resB = modelB.smooth(resA.params)

[ChadFulton]

And then e.g.

resB.forecast()

[reminia]

@ChadFulton
Thanks very much. This is very helpful.

If dataset_A is followed by dataset_B , then:

modelB = sm.tsa.SARIMAX(dataset_A + dataset_B, order=(3, 0, 3))

right?

[Nomarsky]

Hi

@ChadFulton I am not sure if I understood the answer to @reminia. I have a train and test data time series so test part is continuing the train dataset. If you have a SARIMA model like SARIMAX(2, 0, 0)x(4, 1, 1, 7) to predict in the test dataset, for the first predictions in the test dataset, you would need data that is containned in the train dataset because of the seasonality part, so you can't apply correctly the model, is this correct?

Thanks

[ChadFulton]

So I think you're saying that you want to train the model (i.e. estimate the parameters of the model) on a training sample, and then you want to do something with the test dataset. But I can't tell what it is you want to do with the test dataset, can you elaborate on that?

[rafaello5555]

@ChadFulton Hello,

I spliced data into train and test . I applied Arima to train set and now i want to forecasts of AMZN stock volumes for the test data series.

How can i do that?

[ChadFulton]

Please take a look at this documentation https://www.statsmodels.org/dev/examples/notebooks/generated/statespace_forecasting.html, especially the "Cross-validation" section.

[tempcollab]

@ChadFulton the above solution doesn't work for the ARIMA model:

modelA = sm.tsa.ARIMA(dataset_A, order=(3, 0, 3))
resA = modelA.fit()

modelB = sm.tsa.ARIMA(dataset_B, order=(3, 0, 3))
resB = modelB.smooth(resA.params)
resB.forecast()

AttributeError: 'ARIMA' object has no attribute 'forecast'

[ChadFulton]

@tempcollab please double-check your code. Based on the code you have given, resB would be an instance of ARIMAResultsWrapper , not an instance of ARIMA.

It is true that instances of the ARIMA class (such as modelB) do not have a forecast method, but when you use smooth or fit you get an instance of the results class (should be both resA and resB), which does have a forecast method.

[ZJsheep]

@ChadFulton the above solution doesn't work for the ARIMA model:

modelA = sm.tsa.ARIMA(dataset_A, order=(3, 0, 3))
resA = modelA.fit()

modelB = sm.tsa.ARIMA(dataset_B, order=(3, 0, 3))
resB = modelB.smooth(resA.params)
resB.forecast()

AttributeError: 'ARIMA' object has no attribute 'forecast'

You can use get_forecast instead. Here is an example.

tmp_model = ARIMA(X_test, order=(p, d, q))
fit_res = tmp_model.smooth(params)
y = fit_res.get_forecast(steps=pred_len).predicted_mean

where params is obtained from

model = ARIMA(X_train, order=(p, d, q))
fit_res = self.model.fit()
params = fit_res.params

[ZJsheep]

Actually it seems that currently there is a apply method just does that.