How to use correctly Gitingest with Ollama?

[k33g]

Hello,

I have a few questions about using Gitingest (I'll also post them as an issue on GitHub)

So, I have a side project: to learn Go, I'm developing a library to use Ollama's REST API - and I'm making lots of simple examples (this time to learn different techniques with LLMs).

I tested Gitingest with ChatGPT 4, Claude 3.5, Mistral and I always get very good results.

So I wanted to do the same but with Ollama and local models. I tried with and without RAG with models ranging from 1.5b to 32b parameters (my machine can't handle more). I also tried with deepseek-r1:32b.

And nothing works, I don't get accurate results. I'm doing my tests on this project https://github.com/extism/go-sdk keeping only the go files and the README - the project isn't too big.

I always ask the same question: "Explain me how to call a function of a wasm module in golang with the extism-go sdk" and every time all the models make the same mistake (among others, but this one allows me to see if the model answers correctly): normally the "Call" function returns 3 values but for the models it only returns 2.

I can't accept the fact that it's not possible with "LLMs". Until now I was more or less able to make them do what I wanted. But maybe I'm wrong and in this use case it's not possible.

Nevertheless, I was wondering if you had any advice on how to construct the prompts and system instructions. 🙏

[ChakradharG]

Ollama sets the context length of all models to 2048 by default, even if the model can support higher context length. Try manually increasing it and try again.

[codefromthecrypt]

and soon by ENV 🤞 https://github.com/ollama/ollama/pull/8938

[k33g]

@ChakradharG @codefromthecrypt, increasing the context length with "small" LLMs is not enough. I need to provide a shorter context with the most accurate information. But my search for similarities wasn't optimal. @cyclotruc advised me to look at the signatures of the functions, methods, ... (and to extract and use them as metadata of the chunks - if I understood well) and 🙏 thanks again for this

I experimented with some things with this technique, and my first tests seemed to be conclusive.

[cyclotruc]

@k33g Thank you for raising this interesting issue.

Based on various feedback, I believe this is indeed a limitation of self-hosted models. When testing an ingest on your example repository, the total size was around 42k tokens, which should be reasonable.

I have a potential solution to improve response accuracy: implementing an option to generate ingests containing only function signatures (function definitions with variables and return values). This would drastically reduce ingest size, which appears to correlate with hallucination rates in models.

While I don't yet have tooling to measure ingest performance across different models, developing such benchmarking capabilities is on the roadmap. Having the ability to benchmark different code base ingestion strategies would be valuable for the community. This feature will take time to integrate into Gitingest, but I believe it could significantly improve accuracy for queries about API usage and architectural understanding, especially when combined with key elements like import statements.

[cyclotruc]

Unless you have more questions or things to add to this Issue I'm going to close it for clarity! Thanks again @k33g, btw I made recently https://github.com/cyclotruc/gitcontainer and I whish to update it very soon with build/registry hosting capabilities!