lorax icon indicating copy to clipboard operation
lorax copied to clipboard

Published 20 hours ago • verified publisher icon predibase

Potential race condition

Open xingjianan opened this issue 8 months ago • 1 comments

System Info

We are using lorax 0.12.1 and we notice that sometimes different requests of different adapters could affect each other.

We noticed that sometimes one request's input could pollute into another request(that request could be sent to a total different adapter)'s output with a special token N/ . Not sure if you can pour some insights into this issue.

Our base model is mistralai/Mistral-7B-Instruct-v0.2

Information

  • [ ] Docker
  • [ ] The CLI directly

Tasks

  • [ ] An officially supported command
  • [ ] My own modifications

Reproduction

Still trying to reproduce

Expected behavior

Requests should be isolated

xingjianan avatar Mar 27 '25 21:03 xingjianan