das icon indicating copy to clipboard operation
das copied to clipboard

AttributeError thrown in das.train() issuing from numpy - partial saving of results.h5?

Open avakiai opened this issue 9 months ago • 0 comments

Hi Jan,

I encountered an error at the end of training that seems to cause the code to break while saving the results_dict to the results.h5 file. My results file then only has the fit_hist data, and not the other data that it seems it should be saving from the evaluation of the test split.

This doesn't seem to be so critical* since the model has already been saved and, if I've understood correctly, some of the data headed for this file is already in the params file or can be computed by passing the test data to the predict/evaluate_segments functions - right?

*EDIT: In fact, this is critical if doing manual tuning by looping through different parameters, as the code then breaks after the training with first set of parameters are run.

Traceback (most recent call last):
  File "/mnt/cs/projects/BWFAFdeactivNpx/code/3_das/1_train_tune.py", line 27, in <module>
    model, params, fit_hist = das.train.train(data_dir=dataset_dir, 
  File "/gs/home/kiaia/.conda/envs/das_forge/lib/python3.10/site-packages/das/train.py", line 528, in train
    fl.save(save_filename, results_dict)
  File "/gs/home/kiaia/.conda/envs/das_forge/lib/python3.10/site-packages/flammkuchen/hdf5io.py", line 638, in save
    _save_level(h5file, group, value, name=key,
  File "/gs/home/kiaia/.conda/envs/das_forge/lib/python3.10/site-packages/flammkuchen/hdf5io.py", line 249, in _save_level
    _save_ndarray(handler, group, name, level, filters=filters)
  File "/gs/home/kiaia/.conda/envs/das_forge/lib/python3.10/site-packages/flammkuchen/hdf5io.py", line 124, in _save_ndarray
    elif x.dtype == np.object:
  File "/gs/home/kiaia/.conda/envs/das_forge/lib/python3.10/site-packages/numpy/__init__.py", line 324, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'object'.
`np.object` was a deprecated alias for the builtin `object`. To avoid this error in existing code, use `object` by itself. Doing this will not modify any behavior and is safe. 
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'object_'?

My env:

INFO:das.cli:  Linux-4.18.0-193.28.1.el8_2.x86_64-x86_64-with-glibc2.28
INFO:das.cli:  DAS v0.32.4
INFO:das.cli:     GUI is available.
INFO:das.cli:     xarray-behave v0.35.7
INFO:das.cli:     pyqtgraph v0.13.7
INFO:das.cli:     PySide6 v6.7.2
INFO:das.cli:     Qt v6.7.2
INFO:das.cli:
INFO:das.cli:  tensorflow v2.15.0
INFO:das.cli:  keras v2.15.0
INFO:das.cli:     GPU is not available.
INFO:das.cli:
INFO:das.cli:  python v3.10.15 | packaged by conda-forge | (main, Oct 16 2024, 01:24:24) [GCC 13.3.0]
INFO:das.cli:  pandas v2.2.3
INFO:das.cli:  numpy v1.26.4
INFO:das.cli:  h5py v3.12.1
INFO:das.cli:  scipy v1.14.1
INFO:das.cli:  scikit-learn v1.5.2
INFO:das.cli:  xarray v2024.10.0

Thanks!

avakiai avatar Apr 18 '25 15:04 avakiai