What is the plan to support beam search

[feifeibear]

I have noticed that LightLLM currently seems to only support decoding through sampling. Additional decoding methods such as BeamSearch and GreedySearch are not yet supported. I would like to know about the development plans in this regard.

[hiworldwzj]

We have an internal version that supports beam search, but we feel like beam search doesn't add much value to diversity generation, so we have been keeping it out to maintain simplicity in the framework. Is there a rigid use case where beam search is a must-have?

[feifeibear]

I believe that beam search is a must-have feature for the LLM serving framework.

[hiworldwzj]

@feifeibear ok, we will support this feature soon.

[feifeibear]

@feifeibear ok, we will support this feature soon.

Thanks! let me if you need help.