Jie Fu

Can you provide more details about your machine? like system configuration, python version, python environment. @hsalehgh

what is the version of `annlite` you use now?

I used the code in Coca Colab so it should be `2.18.0`

Yes, that works for single batch, but probably not for batch_size > 1 since each question may have different length. Also, the output somehow concatenate the prompt and answer: Is...

conclusion for now: We can pass `past_key_values` to `generate()`, however, it's not possible to return the updated `past_key_values` from `generate()`. We can modify the `utils` in huggingface package to enable...

What's the output for `r = client.encode(['she smiled, with pain'])` and `r = client.encode(['she smiled, with pain', 'what is pain?'])`? I am wondering why they had different behaviors. @kaushikb11

Hi, what's your output of `jina -vf`? Can you set `JINA_LOG_LEVEL=debug` and start the server again?

We are actually working on this model, it will be available on [Inference-API](https://cloud.jina.ai/user/inference)

The flow running behinds the demo is deleted, let me redeploy it

I haven't started it yet. Will do it before next Monday.