ConcreteDropout
ConcreteDropout copied to clipboard
yaringal
Code for Concrete Dropout as presented in https://arxiv.org/abs/1705.07832
Hey, I'm trying out concrete dropout with bigger nets (namely DenseNet121 and ResNet18) and for that tried to port the Keras implementation for spatial concrete dropout to PyTorch. Since it...
https://github.com/yaringal/ConcreteDropout/issues/3#issuecomment-352718724 I can't find why weights are upscaled by 1/1-p after concrete dropout in the paper. Can anyone tell me why?
First of all, great work, In your thesis, the "Dropout as a Bayesian Approximation..." and "Concrete Dropout" article, @yaringal, you seem to apply the Dropout distribution only to the weights...
Dear Yarin, fascinating research, which I am now trying to use in my own. I believe there is a small error in the 'fit_model' function of the pytorch version concerning...
Hi, To calculate the value of weight_regularizer & dropout_regularizer the dataset_size N is used. What if we don't know the dataset size in advance like in case of Lifelong Reinforcement...
I am trying to port the implementation of concrete dropout in keras in https://github.com/yaringal/ConcreteDropout/blob/master/concrete-dropout-keras.ipynb to tensorflow 2. This is mostly straightforward, as tf 2 has most of the keras API...
How could we adapt this for a classification problem using crossentropy?: ``` def heteroscedastic_mse(true, pred): mean = pred[:, :D] log_var = pred[:, D:] precision = K.exp(-log_var) return K.sum(precision * (true-mean)**2....
In the paper, entropy of a Bernoulli random variable is `H(p) := -p * log(p) - (1-p) * log(1-p)` But in the code, dropout_regularizer was computed by `dropout_regularizer = self.p...
I read your paper **Concrete Dropout**. I find an inconsistency of your code and paper. The regularizer of kernel matrix should be proportional to **1-p**. (Eq.(3) of your paper) But...
Hi, since I have updated keras from 2.0.4 to 2.2.0, I am getting line 76, in build weight = self.layer.kernel AttributeError: 'LSTM' object has no attribute 'kernel' when trying to...
