ut train error

[adnanhd]

Hi mathias,

I am still working on the reproduction of the u-sleep model using these commands :). I started everything from scratch, cloned the repository, and created a conda environment u-sleep. Thereafter, first I downloaded and prepared the dataset, then, I started training the model.

Here I get the following error, there are also log files that I shared with you below this error window.

/home/aharun/miniconda3/envs/u-sleep/lib/python3.10/site-packages/tensorflow_addons/utils/ensure_tf_install.py:53: UserWarning: Tensorflow Addons supports using Python ops for all Tensorflow versions above or equal to 2.6.0 and strictly below 2.9.0 (nightly versions are not supported).
 The versions of TensorFlow you are currently using is 2.9.0 and is not supported.
Some things might work, some things might not.
If you were to encounter a bug, do not file an issue.
If you want to make sure you're using a tested and supported configuration, either change the TensorFlow version or the TensorFlow Addons's version.
You can find the compatibility matrix in TensorFlow Addon's readme:
https://github.com/tensorflow/addons
  warnings.warn(
Parent directory of the 'utime' package is not a git repository or Git is not installed. Git information will not be added to this hyperparameter file.
Traceback (most recent call last):
  File "/home/aharun/miniconda3/envs/u-sleep/bin/ut", line 8, in <module>
    sys.exit(entry_func())
  File "/home/aharun/miniconda3/envs/u-sleep/lib/python3.10/site-packages/utime/bin/ut.py", line 103, in entry_func
    mod.entry_func(script_args + help_agrs)
  File "/home/aharun/miniconda3/envs/u-sleep/lib/python3.10/site-packages/utime/bin/train.py", line 296, in entry_func
    run(args=args)
  File "/home/aharun/miniconda3/envs/u-sleep/lib/python3.10/site-packages/utime/bin/train.py", line 239, in run
    train_seq, val_seq = get_generators(train_datasets_queues,
  File "/home/aharun/miniconda3/envs/u-sleep/lib/python3.10/site-packages/utime/utils/scriptutils/train.py", line 178, in get_generators
    train_seqs = [get_batch_sequence(dataset_queue=d,
  File "/home/aharun/miniconda3/envs/u-sleep/lib/python3.10/site-packages/utime/utils/scriptutils/train.py", line 178, in <listcomp>
    train_seqs = [get_batch_sequence(dataset_queue=d,
  File "/home/aharun/miniconda3/envs/u-sleep/lib/python3.10/site-packages/utime/sequences/utils.py", line 85, in get_batch_sequence
    data_per_epoch, n_channels = infer_dpe_and_chans(dataset_queue)
  File "/home/aharun/miniconda3/envs/u-sleep/lib/python3.10/site-packages/utime/sequences/utils.py", line 59, in infer_dpe_and_chans
    with dataset_queue.get_random_study() as sleep_study:
AttributeError: 'list' object has no attribute 'get_random_study'

Here is the content of a log file under /logs/training_log, which is the only log file under /logs directory

INFO | 2022/05/17 07:39:41 | train:run:196 | Args dump: {'num_gpus': 1, 'force_gpus': '', 'continue_training': False, 'initialize_from': None, 'overwrite': True, 'log_file': 'training_log', 'datasets': None, 'just': None, 'no_val': False, 'max_val_studies_per_dataset': 20, 'max_train_samples_per_epoch': 500000.0, 'n_epochs': None, 'channels': None, 'train_queue_type': 'eager', 'val_queue_type': 'lazy', 'max_loaded_per_dataset': 40, 'num_access_before_reload': 32, 'preprocessed': False, 'final_weights_file_name': 'model_weights.h5', 'train_on_val': False}
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/TRAIN] SleepStudyDataset(identifier: dataset_1/TRAIN, N pairs: 91, N loaded: 0)
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/VAL] SleepStudyDataset(identifier: dataset_1/VAL, N pairs: 32, N loaded: 0)
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/TRAIN] Setting select channels: ['EEG Fpz-Cz']
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/VAL] Setting select channels: ['EEG Fpz-Cz']
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/TRAIN] Setting sample rate of 128 Hz
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/VAL] Setting sample rate of 128 Hz
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/TRAIN] Setting 'drop_class' strip function with parameters {'class_int': 5}...
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/VAL] Setting 'drop_class' strip function with parameters {'class_int': 5}...
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/TRAIN] Setting 'zero_out_noisy_epochs' quality control function with parameters {'max_times_global_iqr': 20}...
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/VAL] Setting 'zero_out_noisy_epochs' quality control function with parameters {'max_times_global_iqr': 20}...
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/TRAIN] Setting 'RobustScaler' scaler...
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/VAL] Setting 'RobustScaler' scaler...
INFO | 2022/05/17 07:39:41 | utils:get_data_queues:33 | Using data queue type: EagerQueue
INFO | 2022/05/17 07:39:41 | abc_sleep_study_dataset:log:59 | [Dataset: dataset_1/TRAIN] Loading 91/91 SleepStudy objects...
INFO | 2022/05/17 07:41:17 | utils:get_data_queues:33 | Using data queue type: LazyQueue

Is there anything I can do to fix this bug? thanks in advance -adnan

[perslev]

Hi adnan

Sorry for my late reply! Could you please try and update to the latest version of utime (version 1.0.9) and see if you still experience this issue?

Cheers Mathias