distilabel icon indicating copy to clipboard operation
distilabel copied to clipboard

[FEATURE] Replace `extra_sampling_params` for normal arguments in `vLLM`

Open gabrielmbmb opened this issue 6 months ago • 0 comments

Is your feature request related to a problem? Please describe. Currently, we don't have all the attribute from vllm SamplingParams in the vLLM.generate method. What we do instead is to allow passing extra parameters using the extra_sampling_params dict.

Describe the solution you'd like Remove extra_sampling_params dict and instead create an argument for all the missing attributes from SamplingParams.

gabrielmbmb avatar Aug 12 '24 16:08 gabrielmbmb