Error altering batch_size

[ajthomas1949]

• Orion version: Slightly modified Tulog Colab demo
• Python version:
• Operating System: WIndows

Description

Running the TadGAN pipeline, I set the batch_size to 32 rather than the default 64. I got the following error in orion.fit() /usr/local/lib/python3.7/dist-packages/tensorflow_core/python/client/session.py in call(self, *args, **kwargs) 1470 ret = tf_session.TF_SessionRunCallable(self._session._session, 1471 self._handle, args, -> 1472 run_metadata_ptr) 1473 if run_metadata: 1474 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

InvalidArgumentError: Incompatible shapes: [32,100,1] vs. [64,1,1] [[{{node random_weighted_average_1/mul}}]]

Setting batch_size to 64 runs fine.

from orion import Orion
hyperparameters = {
    "mlprimitives.custom.timeseries_preprocessing.time_segments_aggregate#1": {
        "interval": 1
    },
    "orion.primitives.tadgan.TadGAN#1": {
        'epochs': 5,
        'batch_size': 32
    }
}

orion = Orion(
    pipeline='tadgan',
    hyperparameters=hyperparameters
)
print(df_train.shape)
print(df.shape)
orion.fit(df_train)
anomalies = orion.detect(df)

Can you please explain why this is causing an error? Thank you!

[shavidan123]

Thank you for making us aware of this issue. Within the TadGAN primitive is a class called RandomWeightedAverage that implements a Keras method (random_uniform) that (in this case) always uses an input of 64,1,1 to return a tensor of uniform random distribution. There is clearly some dimensional incompatibility of the lower batch size with this specific function. I think the method itself is working fine but it possibly causes some matrix multiplication errors down the line when the dimensions are changed. I will have look deeper to see if it's unique to the data you're using or applies to different datasets. Once again, thank you for raising this issue, and I'll get back to you when I have an update.

[sarahmish]

Thank you @ajthomas1949!

That is definitely not the expected behavior. I opened an issue to resolve this problem, you can follow it here.

[sarahmish]

PR #315 was just merged. The fix should be available in the next release!