Philip May

Add example for deepspeed config with cpu offloading.

4

### ⚠️ Please check that this feature request hasn't been suggested before. - [X] I searched previous [Ideas in Discussions](https://github.com/OpenAccess-AI-Collective/axolotl/discussions/categories/ideas) didn't find any similar feature requests. - [X] I searched...

[FR] Use normal dict instead of `**model_kwargs` in `_from_pretrained`

6

The `_from_pretrained` function of the `modeling` module has an argument called `**model_kwargs`. See here: https://github.com/huggingface/setfit/blob/fa1021d2355f0cb3a2c85732ee7ffe44b0cef0d1/src/setfit/modeling.py#L337 IMO this should be changed. The problem I see is that it "collects" all keyword...

This is an experiment - work still in progress The idea is: if we have more (or equal) classes than batch size we are able to create batches of sentence...

If we have more (or equal number of) classes than batch size it should be possible to use `MultipleNegativesRankingLoss`. This might improve performance. I would like to get comments about...

[FR] add Dropout to Dense SetFitHead

2

Hi, maybe a dropout layer in the Dense SetFitHead would be nice to have. Implementation could be like in the Transformers `BertForSequenceClassification` head: https://github.com/huggingface/transformers/blob/d447c460b16626c656e4d7a9425f648fe69517b3/src/transformers/models/bert/modeling_bert.py#L1506-L1517 What do you think? @blakechi @lewtun

Hi, could you please add an option to provide a password here? https://github.com/atx/prometheus-tor_exporter/blob/8e382ca63da65c8c43a089c9450c21281784d89b/prometheus-tor-exporter.py#L22 Many thanks Philip

Tokens split by space in English text

1

Hi, it seems like the text of the English sentences is split by space. Like here: ``` [...] Preserve , known as Palos Verdes Peninsula of California . ``` While...

- function to ask if model is the best - get info about study and trial - to be able to use that to save the model