nngeometry

nngeometry copied to clipboard

Is there 'regression' option in FIM_MonteCarlo?

1

Hi, Thank you for your great work. That is amazing! Is there 'regression' option in FIM_MonteCarlo?

zoezhou1999

Implementing BatchNorm for KFAC

7

Hello, I am trying to use BatchNormalization in my network trained on CIFAR. The network has about 50,000 parameters and I want to use the KFAC representation in order to...

a-cowlagi

How to compute FIM with nn.DataParallel(model)?

2

Hey Thomas – Thank you for creating this terrific package! I am wondering what should be the correct way to compute FIM when we use multiple GPUs with `nn.DataParallel()` to...

ZIYU-DEEP

Extracting Eigenvalues of Fisher using KFAC Representation

1

I am trying to get the eigenspectrum of the Fisher of my neural network using the `compute_eigendecomposition()` and `get_eigendecomposition()` methods in the KFAC implementation, but I am having trouble intepreting...

a-cowlagi

Error: `I do not know what to do with layer Embedding(50304, 512)`

4

First of all great library, I've always been looking for some ways to get jacobians and fisher information matrices for my PyTorch models. While the library is fine with my...

CarloNicolini

Make differentiating inputs optional

2

Hello, I am calculating the EK-FAC representation on the linear layers of a model with an embedding layer. The inputs are integers which are lookup values and therefore should not...

Xuzzo

tfjgeorge