Simon Jégou

Hello, would it be possible to re-open this issue ?

What if you apply a threshold to the dot products directly (ie. the logits) ? Or use temperature into your softmax to smooth predictions ? Or add combinations such as...

Did you apply L2 normalization before the dot products ? This should be done as cosine loss was used for training (so in fact it is normalized) I proposed to...

Maybe if you have a dataset, calibrate logits (class wise) with a logistic regression ? Le ven. 23 déc. 2022, 15:39, justlike-prog ***@***.***> a écrit : > Yes, normalisation is...

I am not sure what you mean for multi-label calibration. If all your classes are well calibrated, a single threshold will "work" for all the classes by definition of calibration...

Also if you have a big enough dataset, I would recommend to train the logistic regression from the layer before the projection (i.e. replace proj @ text_encoder(tiger) by the weights...

@justlike-prog I would also advice sklearn : extract the scores with your torch model, then calibrate them (e.g. Platts scaling or isotonic regression). If you use logistic regression you can...

Yes exactly ! The quality of the calibration will depend on the number of samples as usual Le mer. 4 janv. 2023, 08:26, justlike-prog ***@***.***> a écrit : > Alright,...

@marcotcr @slundberg up on this issue, especially for `substrin` :) Following [this issue](https://github.com/guidance-ai/guidance/issues/439) I think it would make sense to use beam search for `substrings`, but it would imply some...

Hi @Puzer, thank you for this great repo ! Do you plan to publish the work you mentioned in this thread soon ? @kohatkk here is a code to finetune...