API + UK-DALE

[klemenjak]

Hi,

I tried to use the dataset UK-DALE in some experiments. Could it be that there are some problems with that particular dataset?

This is the error message I get for DAE:

Using TensorFlow backend.
Started training for  DAE
Joint training for  DAE
............... Loading Data for training ...................
Loading data for  UK-DALE  dataset
Loading building ...  1
Dropping missing values
Train Jointly
(4812480, 1) (4812480, 1) MultiIndex([('power', 'active')],
           names=['physical_quantity', 'type']) MultiIndex([('power', 'active')],
           names=['physical_quantity', 'type'])
Doing Preprocessing
Traceback (most recent call last):
  File "ukstudies.py", line 275, in <module>
    api_results = API(experiments[experiment_name])
  File "/home/users/chklemen/anaconda3/envs/mirum/lib/python3.6/site-packages/nilmtk/api.py", line 59, in __init__
    self.experiment(params)
  File "/home/users/chklemen/anaconda3/envs/mirum/lib/python3.6/site-packages/nilmtk/api.py", line 104, in experiment
    self.train_jointly(clf,d)            
  File "/home/users/chklemen/anaconda3/envs/mirum/lib/python3.6/site-packages/nilmtk/api.py", line 257, in train_jointly
    clf.partial_fit(self.train_mains,self.train_submeters)
  File "/home/users/chklemen/anaconda3/envs/mirum/lib/python3.6/site-packages/nilmtk_contrib/dae.py", line 61, in partial_fit
    app_df = pd.concat(app_df,axis=0).values
  File "/home/users/chklemen/anaconda3/envs/mirum/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 258, in concat
    return op.get_result()
  File "/home/users/chklemen/anaconda3/envs/mirum/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 473, in get_result
    mgrs_indexers, self.new_axes, concat_axis=self.axis, copy=self.copy
  File "/home/users/chklemen/anaconda3/envs/mirum/lib/python3.6/site-packages/pandas/core/internals/managers.py", line 2044, in concatenate_block_managers
    values = values.copy()
MemoryError: Unable to allocate array with shape (99, 4812391) and data type float64
Closing remaining open files:/home/users/chklemen/ukdale.h5...done

I use the latest public version of UK-DALE.

Thanks!