Juhan Bae comments

Results 8 comments of


                                            Juhan Bae

Improve Numerical Stability of Function calc_loss

I thought cross_entropy just combines log_softmax and nll_loss.

Error when includes any torchvision transforms in the pipeline | Cannot determine Numba type of <class 'ffcv.transforms.module.ModuleWrapper'>

Were you able to solve the issue? I am facing a same problem.

Error when includes any torchvision transforms in the pipeline | Cannot determine Numba type of <class 'ffcv.transforms.module.ModuleWrapper'>

Great, that worked for me too. Thank you!

Validation loss goes extremely large or NaN

Hello, thank you for your interest in the code. Could you share which PyTorch version you are using?

Validation loss goes extremely large or NaN

I believe that to reproduce the results, it might be better to use PyTorch 1.5.1. Depending on small changes (e.g., initialization), you might get different results. If you want to...

LM training data attribution question for individual sentences

Sorry for the late reply! I am unsure what the issue might be here - if you are still stuck, please feel free to leave a full code. I can...

CUDA out of memory

Sorry for delay in my responses! 1. Unfortunately, we don’t have block-diagonal approximation implemented. If you are running into memory issues, you could increase ‘_module_partition’ so that we iteratively fit...

CUDA out of memory

1. If you would like to implement block-diagonal approximation, the easiest way would be to keep track of a list of covariances (e.g., https://github.com/pomonam/kronfluence/blob/main/kronfluence/module/tracked_module.py#L131), and modify the code accordingly to...