temperature_scaling
temperature_scaling copied to clipboard
gpleiss
A simple way to calibrate your neural network.
My network is trained according to a binary classification approach, so the model outputs as a single logit value which I then convert into probability by applying the sigmoid function....
ECE (%)
From the phrase "most datasets and models experience some degree of miscalibration, with ECE typically between 4 to 10%" in the paper, I get what it means but I still...
Hi, I ran this with a very simple 10 layer CNN model I trained on MNIST using pytorch lightning. ```python orig_model = pl_module.model val_loader = trainer.datamodule.val_dataloader() scaled_model = ModelWithTemperature(orig_model) scaled_model.set_temperature(val_loader)...
In the paper, it is stated that the temperature T has to be a positive number. In the code, however, although the temperature is initialized with a positive number (i.e.,...
Hey Geoff, I know this is 5 year old research code, but I'm a bit confused about something. In the accompanying paper, it seems like the output of temperature scaling...
I want to adopt temperature_scaling on my own dataset, but I found self.temperature changed just a little from 1.5 to1.495 with lr=0.01 and max_iter=100. But it can continue to decrease...
In your code ``` # Expand temperature to match the size of logits temperature = self.temperature.unsqueeze(1).expand(logits.size(0), logits.size(1)) return logits / temperature ``` Why not just doing something like ``` return...
Great work! From the softmax setting, it seems the trick is typically applied on multi-class classification. Do you think the trick can be extended to multi-label classification? If so, what...
Hello Sir. I tested your code (temperature scaling) in my code. my code is based on inception_v3. But I met some error. ```python # Code # This function probably should...
The current code optimize temperature parameter with respect to nll_criterion, is it more reasonable to optimize with respect to ece_criterion in order to obtain a well-calibrated model? Thanks!
