nonconformist
nonconformist copied to clipboard
donlnz
Getting a range for multi-step prediction with LSTM
Hi, thanks for the work you done in this field!
I am trying to do a multi-step prediction with timeseries via a LSTM NN. To better present the results i looked for a prediction intervall to use with LSTM, I found conformal predictions and eventually this repo.
I have a few questions: My target values (y) are a list of arrays with size 6 (number of steps ahead I'm trying to predict) and my feature vectors are arrays of 12 (steps read before the prediction). When applying the error function (absolute difference) to the timeseries I am currently flattening the y and the predictions made by the model (similar to this thread: https://gist.github.com/donlnz/aa8791fbb5e05d77a48db8e0e88376a2). I still get an error though when trying this. Would this remove the ability to give a worse prediction intervall for timesteps that are futher in the future? Since the different timesteps are no longer distinguishable from eachother?
Also I get an error when trying to calibrate the icp that I don't understand:
from nonconformist.cp import IcpRegressor
from nonconformist.base import RegressorAdapter
from nonconformist.nc import RegressorNc
X, X_valid, y, y_valid = train_test_split(X, y, test_size=0.2)
X_train, X_cal, y_train, y_cal = train_test_split(X, y, test_size=0.2, shuffle=True)
y_cal = np.array(y_cal).ravel()
# define model
def make_model(n_steps, n_features, target_size):
model = Sequential()
model.add(LSTM(20, activation='relu', input_shape=(n_steps, n_features), return_sequences = False))
model.add(Dense(target_size))
model.compile(optimizer='RMSprop', loss= RMSE)
return model
class MyRegressorAdapter(RegressorAdapter):
def init(self, model, fit_params = None):
super(MyRegressorAdapter, self).init(model, fit_params)
def fit(self, x, y):
self.model.fit(x,y, epochs = 1)
return
def predict(self, x):
pred = self.model.predict(x).ravel()
print(pred.shape) ##Returns (8352,)
return pred
model = make_model(n_steps, n_features, target_size)
model1 = MyRegressorAdapter(model)
nc = RegressorNc(model1)
icp = IcpRegressor(nc)
icp.fit(X_train,y_train)
print(y_cal.ravel().shape) ##Returns (8352,)
icp.calibrate(X_cal, y_cal)
predict = icp.predict(X_valid, significance= 0.1)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-22-cd15fa2f8cd4> in <module>
1 print(y_cal.ravel().shape)
----> 2 icp.calibrate(X_cal, y_cal)
3 predict = icp.predict(X_valid, significance= 0.1)
c:\users\arvid.bjorklund\appdata\local\programs\python\python37\lib\site-packages\nonconformist\icp.py in calibrate(self, x, y, increment)
102 else:
103 self.categories = np.array([0])
--> 104 cal_scores = self.nc_function.score(self.cal_x, self.cal_y)
105 self.cal_scores = {0: np.sort(cal_scores)[::-1]}
106
c:\users\arvid.bjorklund\appdata\local\programs\python\python37\lib\site-packages\nonconformist\nc.py in score(self, x, y)
370 norm = np.ones(n_test)
371
--> 372 return self.err_func.apply(prediction, y) / norm
373
374
ValueError: operands could not be broadcast together with shapes (8352,) (1392,)
1392 is the number true values in one timestep (8352/6=1392) but clearly both the y value and pred values are 8352?
