prompttools icon indicating copy to clipboard operation
prompttools copied to clipboard

Published 20 hours ago • verified publisher icon hegelai

Ollama Integration

Open steventkrawczyk opened this issue 1 year ago • 3 comments

🚀 The feature

We want to integrate with Ollama: https://github.com/jmorganca/ollama

Goal is to build an experiment, like the LlamaCpp example: https://github.com/hegelai/prompttools/blob/main/prompttools/experiment/experiments/llama_cpp_experiment.py

Motivation, pitch

Ollama is a new way to run local models. It would be good to support this so developers can compare Ollama to LlamaCpp and other models/frameworks.

Alternatives

No response

Additional context

No response

steventkrawczyk avatar Jul 31 '23 17:07 steventkrawczyk

@steventkrawczyk Can I take care of this issue?

osamanatouf2 avatar Aug 01 '23 19:08 osamanatouf2

Sure! We had someone else interested in it but if you can raise a PR we'll check it out and get it merged 🚀

steventkrawczyk avatar Aug 01 '23 19:08 steventkrawczyk

@osamanatouf2 For more context, we need a python API to the Ollama server to run experiments. You can look at other experiments as examples.

steventkrawczyk avatar Aug 01 '23 19:08 steventkrawczyk