streaming llm problem

Open seanxuu opened this issue 2 years ago • 0 comments

python run_streaming_llama_longalpaca.py \
	----enable_streaming \
	--test_filepath outputs_stream.json \
	--use_flash_attn True \
	--recent_size 32768

My transformers' version is 4.35.0

when I use the code above, I got the result:

Jan 18 '24 07:01 seanxuu