Combine fireTS library with neupy library for NARX network based on Levenberg Marquardt

[TobiasEl]

Hi. I want to create a NARX (Nonlinear Autoregressive with exogenous variables) model based on LM (Levenberg Marquardt) method.

Since this two method are not implemented in keras, I search for the library fireTs (for NARX) and neupy (for LM).

I'm using the sample code for both libraries:

fireTS (NARX): fireTS neupy (LM): Neupy

and I combine them:

from fireTS.models import NARX
from sklearn.ensemble import RandomForestRegressor
import numpy as np
from neupy import algorithms
from neupy.layers import *

x = np.array([[1, 2], [3, 4]])
y = np.array([[1], [0]])

#y = np.ravel(y) <-just to avoid a warning (the error is the same without comment)

network = Input(2) >> Sigmoid(3) >> Sigmoid(1)
optimizer = algorithms.LevenbergMarquardt(network)

mdl1 = NARX(
    optimizer, #I change random forest for LevenbergMarquardt
    auto_order=2,
    exog_order=[2, 2],
    exog_delay=[1, 1])

mdl1.fit(x, y)

ypred1 = mdl1.predict(x, y)

ypred1

But I'm having this error in .fit method:

ZeroDivisionError Traceback (most recent call last)

in () 18 exog_delay=[1, 1]) 19 ---> 20 mdl1.fit(x, y) 21 22 ypred1 = mdl1.predict(x, y)

5 frames

/usr/local/lib/python3.7/dist-packages/neupy/utils/iters.py in count_minibatches(inputs, batch_size) 22 23 def count_minibatches(inputs, batch_size): ---> 24 return int(math.ceil(count_samples(inputs) / batch_size)) 25 26

ZeroDivisionError: division by zero

Any solution?

[itdxer]

Thank you for reporting the issue. I looks like a bug, at least Levenberg-Marquardt algorithm doesn't work with mini-batches.

[itdxer]

as a temporary solution you can either try to run

mdl1.fit(x, y, batch_size=None)

or modify events attribute before you do the training

from neupy.algorithms.base import Events

optimizer = algorithms.LevenbergMarquardt(network)
optimizer.events = Events(network=optimizer, signals=[])

[TobiasEl]

Hi. Thanks for answer the question. I have tried both options, but neither works. The error remains the same: division by zero

[TobiasEl]

I also write to fireTS library: https://github.com/jxx123/fireTS/issues/13 It something about LM batch_size. I put it to 1: mdl1.fit(x, y, batch_size=1) And it pass the fit method. But the same error happens in predict ypred1 = mdl1.predict(x, y):

/usr/local/lib/python3.7/dist-packages/fireTS/models.py in predict(self, X, y, step) 98 99 for k in range(step): --> 100 yhat = self._predictNA(features) 101 if k == step - 1: 102 break

/usr/local/lib/python3.7/dist-packages/fireTS/core.py in _predictNA(self, Xdata) 131 mask = np.isnan(Xdata).any(axis=1) 132 X2pred = Xdata[~mask] --> 133 ypred[~mask] = self.base_estimator.predict(X2pred) 134 return ypred 135

/usr/local/lib/python3.7/dist-packages/neupy/algorithms/gd/base.py in predict(self, *X, **kwargs) 274 raise TypeError("Unknown arguments: {}".format(kwargs)) 275 --> 276 return self.network.predict(*self.format_input(X), **predict_kwargs) 277 278 def train(self, X_train, y_train, X_test=None, y_test=None,

/usr/local/lib/python3.7/dist-packages/neupy/layers/graph.py in predict(self, *inputs, **kwargs) 528 inputs=inputs, 529 batch_size=batch_size, --> 530 show_progressbar=verbose, 531 ) 532 return np.concatenate(outputs, axis=0)

/usr/local/lib/python3.7/dist-packages/neupy/utils/iters.py in apply_batches(function, inputs, batch_size, show_progressbar, show_output, average_outputs) 170 batch_size = n_samples if batch_size is None else batch_size 171 --> 172 n_batches = count_minibatches(inputs, batch_size) 173 bar = progressbar.NullBar() 174

/usr/local/lib/python3.7/dist-packages/neupy/utils/iters.py in count_minibatches(inputs, batch_size) 22 23 def count_minibatches(inputs, batch_size): ---> 24 return int(math.ceil(count_samples(inputs) / batch_size)) 25 26

ZeroDivisionError: division by zero

[itdxer]

I have tried both options, but neither works. The error remains the same: division by zero

I'm quite surprised that the second option didn't work. Do you get the same exception as before? I mean even if the same function fails I wonder if it fails in exactly the same place or if it happens elsewhere

[TobiasEl]

This is the code:

from fireTS.models import NARX
import numpy as np
from neupy import algorithms
from neupy.layers import *
from neupy.algorithms.base import Events

# Random training data
x = np.array([[1, 2], [3, 4]])
y = np.array([[1], [0]])

#y = np.ravel(y)

network = Input(2) >> Sigmoid(3) >> Sigmoid(1)

optimizer = algorithms.LevenbergMarquardt(network)
optimizer.events = Events(network=optimizer, signals=[])

mdl1 = NARX(
    optimizer,
    auto_order=2,
    exog_order=[2, 2],
    exog_delay=[1, 1])

mdl1.fit(x, y, batch_size=None)

ypred1 = mdl1.predict(x, y)

ypred1

And here the error:

---

ZeroDivisionError Traceback (most recent call last)

in () 22 exog_delay=[1, 1]) 23 ---> 24 mdl1.fit(x, y, batch_size=None) 25 26 ypred1 = mdl1.predict(x, y)

5 frames

/usr/local/lib/python3.7/dist-packages/neupy/utils/iters.py in count_minibatches(inputs, batch_size) 22 23 def count_minibatches(inputs, batch_size): ---> 24 return int(math.ceil(count_samples(inputs) / batch_size)) 25 26

ZeroDivisionError: division by zero