shape error with gumbel softmax

[haibao-yu]

Hi, latifisalar, thanks for your great work.

When I run python supernet_main_file.py --train_or_sample train under Pytorch0.4.1, here is running error: ""assert len(shape) == 2" in line 22 in "supernet_functions/model_supernet.py"", and self.thetas is 1-dim tensor. What's wrong for self.thetas? Thanks~

[AnnaAraslanova]

@walk2out hi! It's strange because we have the same PyTorch version. Can you share a Dockerfile? I will look next week

[haibao-yu]

Thanks, and I have dealt with this problem by replacing the gumbel softmax with following code u = Variable((torch.zeros_like(log_alpha)).uniform_()) softmax = torch.nn.Softmax(-1) soft_mask_variables = softmax((log_alpha + (-((-(u.log())).log())).cuda()) / self._temp)

However, there is another problem: the output size of the last stages is [1000, 352, 1, 1] which cause the size error, and I think maybe the super_net is not available, is there any problem? Is the dataset ImageNet instead of Cifar10?

Thanks~

[haibao-yu]

Thanks, and I have dealt with this problem by replacing the gumbel softmax with following code u = Variable((torch.zeros_like(log_alpha)).uniform_()) softmax = torch.nn.Softmax(-1) soft_mask_variables = softmax((log_alpha + (-((-(u.log())).log())).cuda()) / self._temp)

However, there is another problem: the output size of the last stages is [1000, 352, 1, 1] which cause the size error, and I think maybe the super_net is not available, is there any problem? Is the dataset ImageNet instead of Cifar10?

Thanks~

I have found that the super_net is for ImageNet instead of Cifar10, and I have modified the code, all is ok, thanks~

[bloatybo]

Hi @walk2out I met the same problem as you my env is also under Pytorch0.4.1. But your solution doesn't make sense for me. It involves some unknown variables like log_alpha and temp: u = Variable((torch.zeros_like(log_alpha)).uniform()) softmax = torch.nn.Softmax(-1) soft_mask_variables = softmax((log_alpha + (-((-(u.log())).log())).cuda()) / self._temp) could you please give more details on how you solve the issue Thanks! Also @AnnaAraslanova is it the correct way as stated by walk2out to solve the issue? Thanks!

[haibao-yu]

Hi @walk2out I met the same problem as you my env is also under Pytorch0.4.1. But your solution doesn't make sense for me. It involves some unknown variables like log_alpha and temp: u = Variable((torch.zeros_like(log_alpha)).uniform()) softmax = torch.nn.Softmax(-1) soft_mask_variables = softmax((log_alpha + (-((-(u.log())).log())).cuda()) / self._temp) could you please give more details on how you solve the issue Thanks! Also @AnnaAraslanova is it the correct way as stated by walk2out to solve the issue? Thanks!

I wrote wrong code, correct code is: u = Variable((torch.zeros_like(self.thetas)).uniform_()) softmax = torch.nn.Softmax(-1) soft_mask_variables = softmax((self.thetas + (-((-(u.log())).log())).cuda()) / temperature)

Thanks

[ShipXu]

A simple way to implement it is to use the 'private' function：

soft_mask_variables = nn.functional._gumbel_softmax_sample(self.thetas, temperature)

Actually, you have implemented the 'gumbel_softmax_sample' function. Good For You! @walk2out