Aakash kaushik

@zoq, @rcurtin and @shrit tagging you all so we can continue our discussion for documentation here and make a plan to proceed.

@shrit seems like a good plan but i am not much aware about the documentation to write a proper guide on what to avoid and what not to and might...

Hey @shubham1206agra I think this issue has been handled but @rcurtin might be able to provide more info on it.

hi, is this PR being worked on? Pretty excited for this to merge, ready to extend any help if needed.

@mrPsycox `—dtype` is supported in vllm, please take a look at the engine args on vllm docs

> This PR breaks the support for Phi-2, which uses `layer_norm_epsilon` instead of `layer_norm_eps`. does using `layer_norm_eps` for phi-1.5 make sense and for all other cases of phi if in...

> Hi @Aakash-kaushik, can we do something like > > ```python > eps = getattr(config, "layer_norm_eps", None) > if eps is None: > eps = getattr(config, "layer_norm_epsilon", 1e-6) > ```...

> Was this resolved? I am also getting the same error when running this: > > ``` > python -m vllm.entrypoints.openai.api_server --model TheBloke/phi-2-GPTQ --quantization gptq --trust-remote-code > ``` #2428 was...

> @Aakash-kaushik Thank you for writing this, I think we can add some points about tutorials too. > It is quite straightforward to write examples. Most of the above points...

> I think we should leave space for especially interesting or practical examples, even if they aren't especially simple. Yes that is correct I will add that point too.