migrate Weaviate logger from logrus to more performant logger library

[moogacs]

Describe your feature request

logging is crucial in weaviate and of course used everywhere in the code base. However weaviate uses Logrus as underlying logging library.

with this benchmark we can see that Logrus performance is not the best as comparison to Zerolog or slog, so with this enhancement the cost of logging shall be minimum.

source

Requirements :

• Preserve Log Levels: Ensure that all existing log levels (e.g., DEBUG, INFO, WARN, ERROR) remain unchanged and functional.
• Maintain Log Styles: Keep the current formatting and visual style of logs, including indentation, color coding, and any other stylistic elements.
• Optimize Performance: Implement the solution using the most performance-efficient approach, minimizing any overhead while maintaining full functionality. the option of using slog incase zerlog won't be fully compatible with Weaviate needs

Code of Conduct

• [X] I have read and agree to the Weaviate's Contributor Guide and Code of Conduct

[gunishmatta]

@moogacs I would like to work on this.

[moogacs]

Hey @gunishmatta , sure thing go for it. could you let us know before starting the implementation which one will go with Zerolog or slog,

[gunishmatta]

Hi @moogacs I would like to go with Zerolog considering performance and features like Zero-allocation logging. Let me know if it works.

[moogacs]

That sounds good. Could you please open a POC so we can review the changes first and evaluate them before moving forward? This way, we can make sure we’re on the right track and avoid any unnecessary effort.

[gunishmatta]

Hi @moogacs you can expect a POC by this weekend.

[stale[bot]]

Thank you for your contribution to Weaviate. This issue has not received any activity in a while and has therefore been marked as stale. Stale issues will eventually be autoclosed. This does not mean that we are ruling out to work on this issue, but it most likely has not been prioritized high enough in the last months. If you believe that this issue should remain open, please leave a short reply. This lets us know that the issue is not abandoned and acts as a reminder for our team to consider prioritizing this again. Please also consider if you can make a contribution to help with the solution of this issue. If you are willing to contribute, but don't know where to start, please leave a quick message and we'll try to help you. Thank you, The Weaviate Team

[tpanza]

Any news on this? Is this still on the roadmap?

Also, for a self-hosted instance of Weaviate, would configuration options of zerolog be exposed to Weaviate administrators?

[stale[bot]]

Thank you for your contribution to Weaviate. This issue has not received any activity in a while and has therefore been marked as stale. Stale issues will eventually be autoclosed. This does not mean that we are ruling out to work on this issue, but it most likely has not been prioritized high enough in the last months. If you believe that this issue should remain open, please leave a short reply. This lets us know that the issue is not abandoned and acts as a reminder for our team to consider prioritizing this again. Please also consider if you can make a contribution to help with the solution of this issue. If you are willing to contribute, but don't know where to start, please leave a quick message and we'll try to help you. Thank you, The Weaviate Team