Michael Rudolph comments

Repositories
Issues
Comments

Results 2 comments of


                                            Michael Rudolph

Randomness found during training on GPU

#554 solved a similar issue for me. Essentially, sorting by coordinates before passing a tensor through each layer produced deterministic outputs (for my tests)

Non Reproducible Outputs on GPU when using MinkowskiConvolution and stride > 1

Thanks @WilliamHBW, this solved my problem! A "deterministic" MinkowskiConvolution could be implemented as follows ` class SortedMinkowskiConvolution(ME.MinkowskiConvolution): def forward(self, input): # Sort the coordinates weights = torch.tensor([1e12, 1e8, 1e4, 1],...