langchaingo icon indicating copy to clipboard operation
langchaingo copied to clipboard

openai: use ollama in test

Open codefromthecrypt opened this issue 1 year ago • 5 comments

Ollama has some support for openai emulation and could be a nice way to support more tests without requiring an API key or a public endpoint. This might be a nice way to extend coverage of the openai, which could also run relevant versions of the tests also in the ollama llm package. What do you think?

e.g. Some test configuration could work when OPENAI_API_KEY is not set, but OLLAMA_TEST_MODEL is.

	llm, err := openai.New(
		openai.WithBaseURL("http://localhost:11434/v1"),
		openai.WithModel(ollamaModel),
		openai.WithToken("unused"),
		// openai.WithHTTPClient(httputil.DebugHTTPClient),
	)

codefromthecrypt avatar Aug 19 '24 01:08 codefromthecrypt

for test, I think that use testcontainer with ollama, but I need try it.

devalexandre avatar Sep 10 '24 17:09 devalexandre

yep you can use testcontainers or use a system managed one. If you are using small models the perf hit of running via containers may not be a big deal.

codefromthecrypt avatar Sep 11 '24 02:09 codefromthecrypt

@devalexandre there is an Ollama module for Go: https://golang.testcontainers.org/modules/ollama/ 😉

mdelapenya avatar Oct 31 '24 13:10 mdelapenya

fyi if you are doing multiple tests or ones where the completions aren't quick, could be simpler to use system managed. here's an example of that in practice and it works https://github.com/block/goose/blob/49a30b4d22329e10262128d08aab03964f910d9f/.github/workflows/ci.yaml#L52-L100

codefromthecrypt avatar Nov 19 '24 07:11 codefromthecrypt

I'm going to submit a PR for this, will link to this issue once ready

mdelapenya avatar Nov 28 '24 13:11 mdelapenya