Ollama uses localhost as api_base instead of given api_base

[Praj-17]

Hi, I'm coming back to this issue again. I have my ollama server on the URL as present in the original code. However, crawl4ai still uses the local llama that has been downloaded on the machine running the original code instead of running llama from the given API base.

However, this code from the Litellm docs, works perfectly.

from litellm import completion

response = completion(
    model="ollama/llama2", 
    messages=[{ "content": "respond in 20 words. who are you?","role": "user"}], 
    api_base="http://localhost:11434" # replacing this local host to my server endpoint
)
print(response)

I tried passing the api_base parameter in the AsyncWebCrawler but it doesn't help.

Any suggestions @unclecode ?

Originally posted by @Praj-17 in https://github.com/unclecode/crawl4ai/issues/166#issuecomment-2422561340

[Praj-17]

Hi @unclecode any updates on this? I am really counting on you

[unclecode]

Actually, I think there has been a kind of misunderstanding in our documentation. I'm so sorry for that. The point is that you should pass the url_base instead of the api_base. But anyway, in the coming version, 0.3.72, which I'm going to release hopefully by tonight or tomorrow, the code that you shared above will work. Right now, if you change the api_base to the url_base, it will work. Sorry for the inconvenience.

Here I share a cool example of how we can create a knowledge graph from any article. You can see it below.

async def main():
    class Entity(BaseModel):
        name: str
        description: str
        
    class Relationship(BaseModel):
        entity1: Entity
        entity2: Entity
        description: str
        relation_type: str

    class KnowledgeGraph(BaseModel):
        entities: List[Entity]
        relationships: List[Relationship]

    extraction_strategy = LLMExtractionStrategy(
            provider='ollama/qwen2.5:7b',
            # api_base="http://localhost:11434",
            # In current version should be url_base="http://localhost:11435"
            api_base="http://localhost:11435", 
            api_token=os.getenv('OPENAI_API_KEY'),
            schema=KnowledgeGraph.model_json_schema(),
            extraction_type="schema",
            instruction="""Extract entities and relationships from the given text."""
    )
    async with AsyncWebCrawler() as crawler:
        url = "https://paulgraham.com/love.html"
        result = await crawler.arun(
            url=url,
            bypass_cache=True,
            extraction_strategy=extraction_strategy,
            # magic=True
        )
        # print(result.extracted_content)
        with open(os.path.join(__data__, "kb_test.json"), "w") as f:
            f.write(result.extracted_content)

    print("Done")

[Praj-17]

Hey thanks, @unclecode can you also please take care that the newer version complies in both ways so that either we send api_base or url_base it understands it? Instead of replacing url_base with api_base can you add it as an alias somehow, so that the existing codes won't stop working, would be of much help,

thanks and regards. Prajwal. Keep up the good work 🙌🙌

[unclecode]

@Praj-17 sure, noted. You can use either of them.