Jigyasu Krishnan

Hello, I would like to work on this issue. This would be my first time contributing to open-source so I would be grateful for some help.

@NathanHB is this still relevant? I would be happy to work on this and add it to lighteval

@fkiraly @benHeid pinging to request a review

I am planning to work on adding the box uncertainty set and have a few design questions. Question: Should we create new estimators for every uncertainty set, or implement it...

Sounds good! I will start with the new parameter approach and make the default value for `shape`=`Ellipsoidal`, so that it does not break things downstream!

> This kind of information may be particularly helpful in retrieval and estimator selection for the user I agree. And even from an estimator's developer perspective, it will be helpful...

> There is some taxonomy in the API reference that we could start from. @fkiraly I did not get this part, can you please explain this a bit? We can...

> so, info:model_type as a tag name? Sounds good! I suppose we have decided to have multiple categories (e.g., DL, FM) for some models, that means the tag is a...

One way that I can think of is that we can modify the base class to accept an extra parameter `quantization=true` and when it is set to true then require...

```py from transformers import BitsAndBytesConfig class SomeForecaster: def __init__(self, model_name, quantization_config=None): self.model_name = model_name self.quantization_config = quantization_config def load_model(self): if self.quantization_config: quantization_config = BitsAndBytesConfig(**self.quantization_config) self.model = AutoModelForCausalLM.from_pretrained( self.model_name, quantization_config=quantization_config )...