Issue with response completion

[maotoledo]

I've processed a document of a book chapter. But It gives me always same limited length response.

[jasonzhouxf]

me too

[kkatrina28]

Same issue here, how to finish the sentence?

[DjangoMaker]

try to adjust the chunk_size=1000 and chunk_overlap=200, test with 1500 and 250 respectively and see if that makes any difference.

[mohitkapoor1230]

any update on this issue please? When I increased the chunk_size=1000 and chunk_overlap=200 then error comes from Inference API: "Input validation error: inputs must have less than 1024 tokens. Given: 1208" How to fix this response completion issue? Please assist @kkatrina28 @DjangoMaker @jasonzhouxf

[SoniaGrh]

try this : llm = HuggingFaceHub(repo_id="tiiuae/falcon-7b-instruct", task='text-generation', model_kwargs={'max_new_tokens': 200})