themattinthehatt
AttributeError: 'NoneType' object has no attribute 'dumps'
Got the following error when running my own data. Running the training in the data folder from the repo works fine. I went through my data to see if there is anything amiss, but couldnt find an issue.
TRAIN CONFIG:
learning_rate: 0.0001
sequence_length: 500
batch_size: 2
12-reg:9
min_epochs: 10
max_epochs: 500
val_check_interval: 1
enable_early_stop: True early_stop_history: 5
save_last_model: False train_frac: 1
trial_splits: 9;1;8;8
rng_seed_train: 0
plot_train_curves: True device: cuda gpus_vis: 0
tt_n_cpu_trials: 100
tt_n_cpu_workers: 5
Exception ignored in: ‹function Pool.__del__ at 0x7fd03d01bd00>
Traceback (most recent call last):
File "/gs/home/abdelhaym/.conda/envs_ppc/daart/lib/python3.10/multiprocessing/pool.py", line 271, in __del-File "/gs/home/abdelhaym/.conda/envs_ppc/daart/lib/python3.10/multiprocessing/queues.py", line 371, in put
AttributeError: 'NoneType' object has no attribute 'dumps'
After playing around for a while, I figured it might be due to the short input files. I have many shorter videos for manual labelling (maximum 1 minute). The moment I use only longer files (more then 10 minutes long at 30fps), the model trains (but ends with the same error after training. Not sure if anything else is supposed to happen).
Here are some example files:
sub-463_strain-B6_2024-12-14T12_00_05_18677.csv
sub-463_strain-B6_2024-12-14T12_00_05_18677.csv
@hummuscience so sorry I missed this issue! Were you able to get this working? I'm pretty sure this has to do with the sequence length being 500, which is potentially longer than your individual videos. If you shorted sequence length does the issue go away?
also of note, this repo is quite out of date now, and the plan is to do a major upgrade at the beginning of the fall to match a lot of the lightning pose patterns - any suggestions or feature requests appreciated 🙏
Yeah, I went ahead and only ran longer videos.
I was wondering how much input is needed to get good models. I am currently going with DeepEthogram but hope to be able to switch some of the behavior classifications to something that is easier to train. Which is why I tried daart out
Really depends on the species/behaviors/etc., but I would say minimum a few thousand labels per behavior class. You can also check out this paper we published recently (using the daart repo) that might be interesting to you: https://arxiv.org/pdf/2407.16727
TL;DR the TCN model is a good one, but still needs lots of labeled data
@hummuscience just wanted to alert you that we're archiving this repo soon and moving over to our new Lightning Action repo, in case you're sitll interested in this: https://github.com/paninski-lab/lightning-action/