Boost AI Development: The Power of LLM Observability

[wjayesh]

• Title: Boost AI Development: The Power of LLM Observability
• Speaker: Jayesh Sharma
• City: Bengaluru
• Country: India
• Language: English
• Type: Presentation 30-45 mins
• Level: Standard
• Tags: [llm, ai, demo, zenml, mlflow, weights&biases]

Description

LLM applications are taking over the world. To ensure that you ship your ideas effectively, it's essential to move fast, break things and iterate over your solutions quickly. In this talk, we’ll explore the power of observability and how it can enable you to get insights into your LLM workflows that enhance both product quality and developer experience. We’ll cover:

• Versioning Pipelines and Models: Learn how to track changes in your pipelines, models, and codebase. With proper versioning, you can confidently reproduce results and troubleshoot issues.
• Artifact Tracking: Explore and visualize the artifacts produced by your LLM pipelines. Whether it’s model checkpoints, embeddings, or intermediate outputs, observability ensures you know what’s being generated.
• Effective Logging: Interpret logs from your pipeline steps, spread across different runs on various infrastructure in a central place.
• Performance insights: Learn how well your LLM is performing, monitor critical metrics and compare different versions of your applications to decide what makes it to production or what versions need work.

We will achieve the above using open source tools like ZenML, which is a framework that empowers you to build production-ready pipelines from Day 0 and comes with versioning, tracking and reproducibility built into it. It integrates with other tools like MLflow Tracking, Weights & Biases, Comet, etc. that give you even more control over your metrics and let you monitor your apps in more advanced use cases.

Speaker Bio Jayesh (@wjayesh) loves playing around with new technology and currently builds and manages the infrastructure for machine learning applications at ZenML. He plans his days to maintain a balance between coding, sports and music, and will never refuse a chance to go hiking in nature!

Assign to the Community team

Africa, Europe, Middle East: @adriennedevries and @naoiserush Asia, Australia, and Pacific: @ashishtiwari1993 North America: @oliviapetrie12 and @ullyta South America: @gabi-elastic