Output of a network to a single input vector after being trained in batches

[bblais]

Not quite an issue, but I didn't see an example of this. I train a network in batches, say batch size of 10 like below. Then I want the output of each layer to a single input pattern. Not sure of the recommended way of doing this. My various attempts have failed. I copy one below, for a simple case.

I made a model like:

model_dict={
    'input':9,               # number of inputs
    'hidden':[(5,'relu'),],
    'output':(9,'linear'),  # number of moves
    'cost':'mse',
}
_model = Network(batch=10, input_shape=(1,1,model_dict['input']))

hidden=model_dict.get('hidden',[])
for n,typ in hidden:
    _model.add(Connected_layer(outputs=n, activation=typ))

n,typ=model_dict['output']
_model.add(Connected_layer(outputs=n, activation=typ))
_model.add(Cost_layer(cost_type=model_dict['cost']))

_model.compile(optimizer=Adam(lr=0.1))

_model.summary()

layer       filters  size              input                output
   0 input                    10 x   1 x   1 x   9   ->    10 x   1 x   1 x   9
   1 connected                10 x   1 x   1 x   9   ->    10 x   5
   2 connected                10 x   1 x   1 x   5   ->    10 x   9
   3 cost                     10 x   1 x   1 x   9   ->    10 x   1 x   1 x   9

This works to train a batch of 10 on dummy input vectors and target values:

X=np.zeros((10,9))
y=np.ones((10,9))

# Reshape the data according to a 4D tensor
num_samples_X, size_X = X.shape
num_samples_y, size_y = y.shape

assert num_samples_X==num_samples_y

X = X.reshape(num_samples_X, 1, 1, size_X)
y = y.reshape(num_samples_y, 1, 1, size_y)

_model.fit(X=X, y=y, max_iter=10,verbose=False)

Now I want to get the model predictions for a single vector (and also get access to the outputs of each layer, but I think I know how to do that part):

Xp=np.random.randn(1,9)
size_X=9
Xp = Xp.reshape(1, 1, 1, size_X)  # reshape to a single 4d tensor

_model.predict(Xp)

yields the error:

ValueError                                Traceback (most recent call last)
Cell In[52], line 5
      2 size_X=9
      3 Xp = Xp.reshape(1, 1, 1, size_X)
----> 5 _model.predict(Xp)

File ~/anaconda3/lib/python3.11/site-packages/NumPyNet-1.0.0-py3.11.egg/NumPyNet/network.py:656, in Network.predict(self, X, truth, verbose)
    653 num_data = len(X)
    654 _truth = None
--> 656 batches = np.array_split(range(num_data), indices_or_sections=num_data // self.batch)
    658 begin = now()
    659 start = begin

File <__array_function__ internals>:200, in array_split(*args, **kwargs)

File ~/anaconda3/lib/python3.11/site-packages/numpy/lib/shape_base.py:778, in array_split(ary, indices_or_sections, axis)
    776 Nsections = int(indices_or_sections)
    777 if Nsections <= 0:
--> 778     raise ValueError('number sections must be larger than 0.') from None
    779 Neach_section, extras = divmod(Ntotal, Nsections)
    780 section_sizes = ([0] +
    781                  extras * [Neach_section+1] +
    782                  (Nsections-extras) * [Neach_section])

ValueError: number sections must be larger than 0.

[Mat092]

Hi @bblais

Sorry for the late response, commit 777db4f should have fixed this issue. The problem was in how we handled the prediction in batches. Now if number_of_data < batch, the network just predict the whole dataset.

If you have any other problem, please let me know! Mattia