Pedro Azevedo

This example doesn't work for the inference, even the 3B model takes 30+ minutes for inference on a A10 GPU. I tried logging the model with pyfunc, langchang and huggingface...

As described in the demo: https://www.dbdemos.ai/demo-notebooks.html?demoName=llm-dolly-chatbot ````def publish_model_to_mlflow(): # Build our langchain pipeline langchain_model = build_qa_chain() with mlflow.start_run() as run: # Save model to MLFlow # Note that this only...

I tried this but still get an error while publishing the model @srowen " TypeError: MLflow langchain flavor only supports logging langchain.chains.llm.LLMChain instances, found " I'm not sure where else...

If you can make the tensorRT engine I don't see why not. Can the TAO models be converted to Darknet weights?