Alexey Mametyev

Provide logits or logprobs in the API

48

Feature request: How can i get logits (probabilites of each next token), during generation, just like I can do it in Open AI API (logprobs)? This feature will be helpfull...

Problem with installation

4

I'v tried to install unsloth to my server using pip, but pip can't found required version: ``` %pip install "unsloth[cu121_ampere_torch211] @ git+https://github.com/unslothai/unsloth.git" Defaulting to user installation because normal site-packages is...

Turn off gravity for rigid body

3

I want to turn off gravity for one scene, but methods from pymunk docs does not works. Firstly I tryied to change scene space gravity paramether in construct method: ```...

[FEATURE REQUEST] Add Support for Qwen1.5-MoE Architecture in DeepSpeed-MII

1

# Qwen1.5-MoE Support With the increasing attention on mixture-of-experts (MoE) models, especially following the advancements heralded by Mixtral, I propose considering the integration of the Qwen1.5-MoE architecture, particularly its A2.7B...

Can this be used for Jambo inference

1

Can I use this solution for inference https://huggingface.co/ai21labs/Jamba-v0.1/discussions with offloading mamba moe layers? Jambo it SOTA open source long context model and its support would be very useful for this...

Run without quantization

9

QuantConfig is mandatory of make model function ```python model = build_model( device=device, quant_config=quant_config, offload_config=offload_config, state_path=state_path, ) ``` Can I run mixtral with layer offloading, but WITHOUT quntization using this library?

problem with one letter world

1

## Example of code ``` cl = Client("en", "fr") for tr in cl.get_translations("I"): print(tr) ``` ## Expected behavior prints Moi ## Real behavior don't print anything

Problem with deepspeed finetuning

1

I've tried to train Mistral-7b-v0.1 on multiple GPU-s using deepspeed. I started with example from ReadMe - ```python from xllm import Config from xllm.datasets import GeneralDataset from xllm.cli import cli_run_train...

Problem with LLama training with LoRA

3

### 🐛 Describe the bug I've tried to train LLama model using the reward model. I created such a config ```python config = TRLConfig( train=TrainConfig( seq_length=4096, epochs=100, total_steps=10000, batch_size=4, checkpoint_interval=100,...

``` 2023-09-12 17:25:29.450 | DEBUG | __main__:make_completion:34 - Chat/Completions Nb Retries : 0 2023-09-12 17:25:31.479 | DEBUG | __main__:make_completion:43 - Status Code : 200 2023-09-12 17:25:44.356 | DEBUG | __main__:make_completion:34...