fix(reranker): reproduce ignoring top_n

[mkhludnev]

Description

This PR fixes #6700

Notes for Reviewers

Signed commits

• [v] Yes, I signed my commits.

[netlify[bot]]

✅ Deploy Preview for localai ready!

Name	Link
🔨 Latest commit	f638a2c04074ca997fb3d722794f9cb6e11485fe
🔍 Latest deploy log	https://app.netlify.com/projects/localai/deploys/690c5cac56835b0009f40194
😎 Deploy Preview	https://deploy-preview-7025--localai.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[mkhludnev]

presumably it might be fixed for llama_cpp rerank as well, if @mudler consider it necessary.

[mudler]

Hey, thanks for looking at this!

presumably it might be fixed for llama_cpp rerank as well, if @mudler consider it necessary.

I think for llama.cpp we should be covered already, as top_k/top_p settings are already propagated for the chat (and rerankers follows along the same reasoning):

https://github.com/mudler/LocalAI/blob/257310231782ca5b0bcaa0877d5f7465b6d244cf/backend/cpp/llama-cpp/grpc-server.cpp#L104

Unless you noticed something off!

[mkhludnev]

I just realized that the most users don't pass top_n at all. So, this PR cause a trouble for them. Here's the fix #7199