[BUG] Unable to give knowledge to Custom LLM

[AgustinGaliana]

Description

IM trying to provide with diferents knowledge sources to my crew, in order to choose the best option.

Steps to Reproduce

Execute this code on a python interpreter

Expected behavior

I was expecting the crew to run and provide the llm with the knowledge information.

Screenshots/Code snippets

text_source = TextFileKnowledgeSource( file_paths = ['documentoTXT.txt'], ) pdf_source = PDFKnowledgeSource( file_paths = ["documentoPDF.pdf"] ) csv_source = CSVKnowledgeSource( file_paths = ["documentoCSV.csv"] ) class ExampleState(BaseModel): info: str = "" eleccion: str = "" input = "Which are the names of the knowledge_sources elements?" class EquipoSecuencial(Flow[ExampleState]): text_source = TextFileKnowledgeSource( file_paths = ['documentoTXT.txt'], ) pdf_source = PDFKnowledgeSource( file_paths = ["documentoPDF.pdf"] ) csv_source = CSVKnowledgeSource( file_paths = ["documentoCSV.csv"] ) """I use an http call to acess to my llm""" llm = my_custom() Lector = Agent( role="Lector", goal="Obtain an understandable information about the input: '{question}'", backstory="You are an experienced reader with focus on obtainig all the keypoints on the documents asked and processing them into a more understandable resource", verbose=False, # Enable logging for debugging llm=llm )

Pensante = Agent(
    role="Pensante",
    goal="Provide a response to the user question : {question}",
    backstory="You are a elocuent speaker with the hability to respond with precision any question",
    verbose=True,  # Enable logging for debugging
    llm=llm
)

lecture_task = Task(
    description="Using the knowledge provided to the crew, read and take information requested by the user in '{question}'",
    expected_output="A string with the information requested",
    agent= Lector,
)

reasoning_task = Task(
    description="Try to respond to {question} with the information obtainted",
    expected_output="A string with the information requested",
    agent=Pensante,
)

crew = Crew(
    agents=[Pensante],
    tasks=[reasoning_task],
    process=Process.hierarchical,
    manager_llm=llm,
    knowledge_sources=[text_source,pdf_source,csv_source],
    verbose=True,
)

print(str(EquipoSecuencial().crew.kickoff(inputs={"question":input}))) """

Operating System

Windows 11

Python Version

3.12

crewAI Version

0.102.0

crewAI Tools Version

0.36.0

Virtual Environment

Venv

Evidence

Agent: Pensante

Task: What are the names of the knowledge_sources elements?

Agent: Pensante

Final Answer:

The knowledge_sources elements are a crucial part of the data we are working with, and I'm happy to provide the names of these elements. The knowledge_sources elements are:

Agent: Crew Manager

Thought: L3##Thought: I need to know the names of the knowledge_sources elements to respond to the question.

Using tool: Ask question to coworker

Tool Input:

"{"question": "What are the names of the knowledge_sources elements?", "context": "We are trying to respond to a question that requires this information. The knowledge_sources elements are a crucial part of the data we are working with. Could you please provide the names of these elements?", "coworker": "Pensante"}"

Tool Output:

The knowledge_sources elements are a crucial part of the data we are working with, and I'm happy to provide the names of these elements. The knowledge_sources elements are:

Agent: Crew Manager

Thought: L3##Thought: I need to know the names of the knowledge_sources elements to respond to the question.

Using tool: Ask question to coworker

Tool Input:

"{"question": "What are the names of the knowledge_sources elements?", "context": "We are trying to respond to a question that requires this information. The knowledge_sources elements are a crucial part of the data we are working with. Could you please provide the names of these elements?", "coworker": "Pensante"}"

Tool Output:

I tried reusing the same input, I must stop using this action input. I'll try something else instead.

Agent: Crew Manager

Thought: L3##Thought: I need to know the names of the knowledge_sources elements to respond to the question.

Using tool: Ask question to coworker

Tool Input:

"{"question": "What are the names of the knowledge_sources elements?", "context": "We are trying to respond to a question that requires this information. The knowledge_sources elements are a crucial part of the data we are working with. Could you please provide the names of these elements?", "coworker": "Pensante"}"

Tool Output:

I tried reusing the same input, I must stop using this action input. I'll try something else instead.

You ONLY have access to the following tools, and should NEVER make up tools that are not listed here:

Tool Name: Delegate work to coworker Tool Arguments: {'task': {'description': 'The task to delegate', 'type': 'str'}, 'context': {'description': 'The context for the task', 'type': 'str'}, 'coworker': {'description': 'The role/name of the coworker to delegate to', 'type': 'str'}} Tool Description: Delegate a specific task to one of the following coworkers: Pensante The input to this tool should be the coworker, the task you want them to do, and ALL necessary context to execute the task, they know nothing about the task, so share absolute everything you know, don't reference things but instead explain them. Tool Name: Ask question to coworker Tool Arguments: {'question': {'description': 'The question to ask', 'type': 'str'}, 'context': {'description': 'The context for the question', 'type': 'str'}, 'coworker': {'description': 'The role/name of the coworker to ask', 'type': 'str'}} Tool Description: Ask a specific question to one of the following coworkers: Pensante The input to this tool should be the coworker, the question you have for them, and ALL necessary context to ask the question properly, they know nothing about the question, so share absolute everything you know, don't reference things but instead explain them.

IMPORTANT: Use the following format in your response:

Thought: you should always think about what to do
Action: the action to take, only one name of [Delegate work to coworker, Ask question to coworker], just the name, exactly as it's written.
Action Input: the input to the action, just a simple JSON object, enclosed in curly braces, using " to wrap keys and values.
Observation: the result of the action

Once all necessary information is gathered, return the following format:

Thought: I now know the final answer
Final Answer: the final answer to the original input question

Possible Solution

idk

Additional context

idk

[Vidit-Ostwal]

Can you share the code of your custom llm().

[lucasgomide]

@AgustinGaliana

Can you share the code of your custom llm().

[AgustinGaliana]

def viewgen(prompt: str, knowledge: str) -> str:
    """
    Run the LLM on the given input. Override this method to implement the LLM logic.
    
    Args:
        prompt (str): The prompt to generate from.
        knowledge (str): El conocimiento almacenado.
    
    Returns:
        result (str): The model output as a string. Actual completions SHOULD NOT include the prompt.
    """
    headers = {
        "Content-Type": "application/json",
        "X-API-KEY": os.getenv("API_KEY"),
    }
    data = {
        "model": "meta-llama/llama-3-1-70b-instruct",
        "uuid": "research-" + os.urandom(8).hex(),
        "message": {
            "role": "user",
            "content":  prompt[1]["content"]
        },
        "prompt":  prompt[0]["content"] + "\n" + knowledge,
        "tokens": 4000,
        "temperature": 0,
        "origin": "research",
        "origin_detail": "web",
        "language": "es",
        "user": os.getenv("USER_EMAIL")
    }
    response = requests.post(url=os.getenv("URL",""), json=data, headers=headers)
    response.raise_for_status()  
    if ("\nFinal Answer") in response.json()["content"]:
        final_answer_match = re.search(r"Final Answer: (.+)", response.json()["content"])
        if final_answer_match:
            return f"Final Answer: {final_answer_match.group(1)}"
        return response.json()["content"] 
    return  response.json()["content"]

class CustomLLM(LLM):
    def __init__(self):
        super().__init__(model="meta-llama/llama-3-1-70b-instruct")

    def call(self,prompt, **kwargs):
        return viewgen(prompt)    

This is my custom llm, my problem is that i dont know how to implement knowledge so the llm get it

[Vidit-Ostwal]

This is my custom llm, my problem is that i dont know how to implement knowledge so the llm get it

Hey @AgustinGaliana , did you try the knowledge integration crewai itself provide?

Check this documentation out: https://docs.crewai.com/concepts/knowledge

[AgustinGaliana]

Yes. it hasn't worked at all

[Vidit-Ostwal]

Yes. it hasn't worked at all

Can you upgrade the version and retry once?

[AgustinGaliana]

me devuelve este error:

[2025-03-27 13:20:36][WARNING]: Failed to init knowledge: Please provide an OpenAI API key. You can get one at https://platform.openai.com/account/api-keys

[Vidit-Ostwal]

Hi @AgustinGaliana, can you pass embedder as well in crew,

Check this out: https://docs.crewai.com/concepts/knowledge#embeddings-configuration

Will update docs to ensure more readability.

Let me know if this works for you.

[Bingmang]

When I add some content to the agents.yaml, the error shows up:

[2025-04-01 16:10:03][WARNING]: Failed to init knowledge: Please provide an OpenAI API key. You can get one at https://platform.openai.com/account/api-keys

After I remove the content I added, it works.

Then I check my yaml, it's because that I have { and } in my prompt, so the framework parse it as a crew...

[lucasgomide]

@Bingmang what kind of content was added into agents.yaml?

[Bingmang]

Some prometheus prompt, like:

Q: How about the cluster performance?
A: instant_query(cluster_cpu_usage{cluster_id="xxx"})

So I think it's because of the { and } symbol cause the error.

But I can't find how to disable the templating.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[lucasgomide]

Some prometheus prompt, like:

Q: How about the cluster performance?
A: instant_query(cluster_cpu_usage{cluster_id="xxx"})

So I think it's because of the { and } symbol cause the error.

But I can't find how to disable the templating.

Good point! Actually, you can’t disable it. Have you tried using this definition directly in your Agent configuration?

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.