Ismael Delgado issues

Repositories
Issues
Comments

Results 1 issues of


                                            Ismael Delgado

Server error: transport error

### System Info We are deploying the model meta-llama/Meta-Llama-3.1-70B-Instruct with FP8 quantization and everything works perfectly for hours until the server crashes with this error: 2024-10-01T07:43:22.055987Z ERROR batch{batch_size=1}:prefill:prefill{id=290 size=1}:prefill{id=290 size=1}:...