zoubaihan

我在运行时，报如下错误： Traceback (most recent call last): File "a2c-train.py", line 348, in main() File "a2c-train.py", line 330, in main xent_trainer.train(opt.start_epoch, opt.start_reinforce - 1, start_time) File "F:\code_summarization_public-master\lib\train\Trainer.py", line 30, in train train_loss...

How to specify which GPU the model inference on?

1

Hello, I have 4 GPUs. And when I set `tensor_parallel_size` as 2, when running the service, it would takes CUDA:0 and CUDA:1. My question is, if I want start two...

I want use the function prefix_allowed_tokens_fn, where of basaran's source code shall I modify?

1

Hello, we all know that in huggingface transformers' origin `model.generate()` method, we can set the function paremeter`prefix_allowed_tokens_fn` to restrict the generate rule. I want to use this function in basaran...

How to set my own parameters in model.generate() in basaran?

2

Hello, I want use my customize parameters when model.generate(), like this: ``` model.generate(input_ids, max_new_tokens=max_new_tokens, do_sample=True, max_length=max_length, temperature=temperature, top_p=top_p, repetition_penalty=repetition_penalty) ``` but if I use basaran, the code is like this:...

The generator doesn't work if set num_beams not equals 1

3

if I set num_beams=3 or other number that not equals 1, this tools will not work, and the generator can not output as stream, how to solve this?