kejian

Thank you. I'm taking a look and will ask follow-up questions.

As I understood the Colab QA demo (using TextStreamer) should also work in a standard HF pipeline right? I'm trying to do inference on a .jsonl dataset. According to your...

btw Can I ask what's a good sampling strategy if you've tried some? I'm doing: model = AutoModelForCausalLM.from_pretrained( MODEL_PATH, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True, mem_attention_grouping=(1, 2048), use_cache=True, ) output = model.generate( input_ids=input_ids,...

I fixed it by: 1. Only manually copy (ctrl+c or cmd+c) the URL text in the email. Don't do "copy link" 2. if 403 still occur, request a link again....