[Question] LangChain Callbacks?

[austinmw]

Hi, does this library support LangChain callbacks? I'd like to use Chainlit, which requires callbacks, but not sure how. This is what I'm attempting:

from langchain import PromptTemplate, LLMChain
from langchain.chat_models import ChatOpenAI
import chainlit as cl
from langchain.chains import RetrievalQA
from langchain.document_loaders import TextLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Chroma
from nemoguardrails import LLMRails, RailsConfig


@cl.on_chat_start
def main():
    # Instantiate the chain for that user session
    loader = TextLoader("./paul_graham_essay.txt")
    documents = loader.load()
    text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
    texts = text_splitter.split_documents(documents)
    embeddings = OpenAIEmbeddings()
    docsearch = Chroma.from_documents(texts, embeddings)
    llm = ChatOpenAI(model='gpt-4')
    qa_chain = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=docsearch.as_retriever(), verbose=True)

    config = RailsConfig.from_path("./config/example")
    app = LLMRails(config)
    app.register_action(qa_chain, name="qa_chain")
    
    # Store the chain in the user session
    cl.user_session.set("app", app)


@cl.on_message
async def main(message: cl.Message):
    # Retrieve the chain from the user session
    app = cl.user_session.get("app")  # type: qa_chain
    
    history = [
        {"role": "user", "content": message.content}
    ]
    
    # Call the chain asynchronously
    res = await app.generate_async(message.content, callbacks=[cl.AsyncLangchainCallbackHandler()])
    
    # Do any post processing here
    
    # "res" is a Dict. For this chain, we get the response by reading the "text" key.
    # This varies from chain to chain, you should check which key to read.
    await cl.Message(content=res["content"]).send()

[austinmw]

I was able to put the callback in RetrievalQA init to get it to run, but of course Chainlit can’t see what NeMo is doing and display it in the UI

[drazvan]

Hey @austinmw! Yes, NeMo Guardrails uses callbacks (this his how the verbose logging is done). However, they're not directly exposes. If you want to test quickly, you can do a monkey patching and add your callbacks to the logging_callbacks here: https://github.com/NVIDIA/NeMo-Guardrails/blob/main/nemoguardrails/logging/callbacks.py .

[aniruddha-adhikary]

Same here, I can't seem to get chainlit callbacks to work!