text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

OpenAI Client format + chat template for a single call

Open vitalyshalumov opened this issue 1 year ago • 1 comments

System Info

latest docker

Information

  • [X] Docker
  • [ ] The CLI directly

Tasks

  • [ ] An officially supported command
  • [ ] My own modifications

Reproduction

Hello, Can you tell me please how to implement the following functionality combined:

  1. I'm interested in OpenAI Client format: prompt= [ {"role": "system", "content": "You are a sassy, wise-cracking robot as imagined by Hollywood circa 1986."}, {"role": "user", "content": "Hey, can you tell me any fun things to do in New York?"} ]

  2. I want to make sure that chat template of the served odel is aplied

  3. I dont want a chat - I want that each call with a prompt statrt from a clear history to aoid token overflow. Thank you!

Expected behavior

An answer for each prompt indepedent of the previos anser, but with OpenAI client API

vitalyshalumov avatar Oct 14 '24 08:10 vitalyshalumov