open-sesame icon indicating copy to clipboard operation
open-sesame copied to clipboard

problem with python 3.6

Open asaivasavi opened this issue 5 years ago • 2 comments

Dimensions of lookup parameter /_0 lookup up from file ({100,400574}) do not match parameters to be populated ({100,410050})

asaivasavi avatar Feb 26 '19 05:02 asaivasavi

Here's a working version of the code in python3.6 with GPU compute option: https://github.com/free-soellingeraj/open-sesame

I resolved the above issue by training longer, especially in the latter 2 models.

free-soellingeraj avatar Mar 14 '20 10:03 free-soellingeraj

Could it be that the targetid is getting stuck during learning for the implementation given in the 3.6 fork? How many epochs till one can get improvements in avg. loss value/f1 for targetid? In the former version val_f1 was getting much higher.

Old vs New for TargetId Training: epoch = 2 loss = 6.148638 train_f1 = 0.0000 val_f1 = 0.7283 best_val_f1 = 0.7764: 76%|███████▌ | 2631/3479 [02:42<00:17, 49.20it/s] epoch = 2 loss = 6.153349 train_f1 = 0.0000 val_f1 = 0.7283 best_val_f1 = 0.7764: 76%|███████▌ | 2631/3479 [02:42<00:17, 49.20it/s]

[dev epoch=20] loss = 9.383501 p = 0.4452 (1069.0/2401.0) r = 0.4501 (1069.0/2375.0) f1 = 0.4477 [lr=0.01 clips=0 updates=100] epoch = 20.1600 loss = 9.137601 train f1 = 0.4556 [lr=0.01 clips=0 updates=100] epoch = 20.1700 loss = 9.140439 train f1 = 0.4548 [lr=0.01 clips=0 updates=100] epoch = 20.1800 loss = 9.125574 train f1 = 0.4761

Could it be an issue when the last line if commented out it part?:

        f_i_cpu = to_device(f_i, 'CPU')
        logloss = log_softmax(f_i_cpu, valid_frames)
        # logloss = to_device(logloss, 'GPU:0')

chrisoutwright avatar Mar 18 '21 02:03 chrisoutwright