Continue ignores contextLength when using openai provider with local server

[ShaunaGordon]

Before submitting your bug report

• [x] I've tried using the "Ask AI" feature on the Continue docs site to see if the docs have an answer
• [x] I believe this is a bug. I'll try to join the Continue Discord for questions
• [x] I'm not able to find an open issue that reports the same bug
• [x] I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Arch
- Continue version:
- IDE version: 1.2.8
- Model: gpt-oss:latest, aliased at gpt-4o:latest, per #7035
- config:
  
---
name: gpt-oss 20b
version: 1.1.0
schema: v1
models:
  - name: gpt-oss
    provider: openai
    model: gpt-4o
    apiBase: "http://localhost:11434/v1"
    apiKey: "ollama"
    capabilities:
      - tool_use
    supportsTools: true
    roles:
      - chat
      - apply
      - edit
    defaultCompletionOptions:
      contextLength: 16768
      temperature: 0.1

Description

When using Ollama to mimic OpenAI, Continue doesn't honor the contextLength setting and instead just uses the default. Switching to the in-built ollama provider honors the context length setting.

To reproduce

1. Run ollama and download gpt-oss
2. ollama cp gpt-oss gpt-4o to alias
3. Add model config to an assistant
4. Query it
5. ollama ps

Log output

[bdougie]

@uinstinct do you think you could take a look at this? For triaging?

[uinstinct]

Sure, will look into it today

[uinstinct]

So the contextLength parameter is only passed down in ollama providers. For other providers, it is used internally for pruning/budgeting.

[HoussemMEG]

I encountered the same problem. The fix is on the ollama side instead of continue, you can verify that the right context lenght has been sent inside continue : continue console (after enabling it) -> options (underneath prompt) you will see something like this : "contextLength": 32000,.

To enable ollama and make it accept more context lenght you can do :

Temporary solution

1. Stop Ollama:

sudo systemctl stop ollama

2. Start Ollama with a larger context length:

OLLAMA_CONTEXT_LENGTH=128000 ollama serve

Permanent solution

1. Edit the Ollama service file: sudo systemctl edit --full ollama.service

2. add under the first Environement variable the ollama context length env var: Environment="OLLAMA_CONTEXT_LENGTH=128000"

3. reload the deamon and restart ollama: sudo systemctl daemon-reload sudo systemctl restart ollama

4. check if ollama is running systemctl status ollama

5. check the env variable if it has been set correctly sudo systemctl show ollama | grep OLLAMA_CONTEXT_LENGTH