LLPhant icon indicating copy to clipboard operation
LLPhant copied to clipboard
verified publisher icon theodo-group

Use tool with ollama

Open devbysb opened this issue 1 year ago • 10 comments

Hi,

Is it possible to use tools with ollama ?

I have found this article https://ollama.com/blog/tool-support

I have tried with OllamaChat() but it's not implemented.

I also tried with OpenAiChat() but I can specify the url of ollama server (to test with openAi compatibility)

Thank you

devbysb avatar Aug 01 '24 13:08 devbysb

Yes we should update the class so we can use tools with Ollama

MaximeThoonsen avatar Aug 01 '24 22:08 MaximeThoonsen

I have with this problem to use Ollama

"message": "cURL error 7: Failed to connect to localhost port 11434 after 0 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for http://localhost:11434/api/embeddings", "exception": "GuzzleHttp\Exception\ConnectException", "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php", "line": 275, "trace": [ { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php", "line": 205, "function": "createRejection", "class": "GuzzleHttp\Handler\CurlFactory", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php", "line": 157, "function": "finishError", "class": "GuzzleHttp\Handler\CurlFactory", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/CurlHandler.php", "line": 47, "function": "finish", "class": "GuzzleHttp\Handler\CurlFactory", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php", "line": 28, "function": "__invoke", "class": "GuzzleHttp\Handler\CurlHandler", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php", "line": 48, "function": "GuzzleHttp\Handler\{closure}", "class": "GuzzleHttp\Handler\Proxy", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php", "line": 64, "function": "GuzzleHttp\Handler\{closure}", "class": "GuzzleHttp\Handler\Proxy", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Middleware.php", "line": 31, "function": "__invoke", "class": "GuzzleHttp\PrepareBodyMiddleware", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/RedirectMiddleware.php", "line": 71, "function": "GuzzleHttp\{closure}", "class": "GuzzleHttp\Middleware", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Middleware.php", "line": 66, "function": "__invoke", "class": "GuzzleHttp\RedirectMiddleware", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/HandlerStack.php", "line": 75, "function": "GuzzleHttp\{closure}", "class": "GuzzleHttp\Middleware", "type": "::" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Client.php", "line": 333, "function": "__invoke", "class": "GuzzleHttp\HandlerStack", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Client.php", "line": 169, "function": "transfer", "class": "GuzzleHttp\Client", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/Client.php", "line": 189, "function": "requestAsync", "class": "GuzzleHttp\Client", "type": "->" }, { "file": "/var/www/vendor/guzzlehttp/guzzle/src/ClientTrait.php", "line": 95, "function": "request", "class": "GuzzleHttp\Client", "type": "->" }, { "file": "/var/www/vendor/theodo-group/llphant/src/Embeddings/EmbeddingGenerator/Ollama/OllamaEmbeddingGenerator.php", "line": 38, "function": "post", "class": "GuzzleHttp\Client", "type": "->" }, { "file": "/var/www/vendor/theodo-group/llphant/src/Query/SemanticSearch/QuestionAnswering.php", "line": 78, "function": "embedText", "class": "LLPhant\Embeddings\EmbeddingGenerator\Ollama\OllamaEmbeddingGenerator", "type": "->" }, { "file": "/var/www/vendor/theodo-group/llphant/src/Query/SemanticSearch/QuestionAnswering.php", "line": 28, "function": "searchDocumentAndCreateSystemMessage", "class": "LLPhant\Query\SemanticSearch\QuestionAnswering", "type": "->" }, { "file": "/var/www/app/Http/Controllers/RagController.php", "line": 38, "function": "answerQuestion", "class": "LLPhant\Query\SemanticSearch\QuestionAnswering", "type": "->" }, { "file": "/var/www/app/Http/Controllers/ChatBotController.php", "line": 12, "function": "initRag", "class": "App\Http\Controllers\RagController", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php", "line": 46, "function": "pergunta", "class": "App\Http\Controllers\ChatBotController", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Route.php", "line": 260, "function": "dispatch", "class": "Illuminate\Routing\ControllerDispatcher", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Route.php", "line": 206, "function": "runController", "class": "Illuminate\Routing\Route", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 808, "function": "run", "class": "Illuminate\Routing\Route", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 144, "function": "Illuminate\Routing\{closure}", "class": "Illuminate\Routing\Router", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php", "line": 50, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Routing\Middleware\SubstituteBindings", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php", "line": 88, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\VerifyCsrfToken", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php", "line": 49, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\View\Middleware\ShareErrorsFromSession", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php", "line": 121, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php", "line": 64, "function": "handleStatefulRequest", "class": "Illuminate\Session\Middleware\StartSession", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Session\Middleware\StartSession", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php", "line": 37, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Cookie\Middleware\AddQueuedCookiesToResponse", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php", "line": 75, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Cookie\Middleware\EncryptCookies", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 119, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 807, "function": "then", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 786, "function": "runRouteWithinStack", "class": "Illuminate\Routing\Router", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 750, "function": "runRoute", "class": "Illuminate\Routing\Router", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Routing/Router.php", "line": 739, "function": "dispatchToRoute", "class": "Illuminate\Routing\Router", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php", "line": 201, "function": "dispatch", "class": "Illuminate\Routing\Router", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 144, "function": "Illuminate\Foundation\Http\{closure}", "class": "Illuminate\Foundation\Http\Kernel", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php", "line": 21, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ConvertEmptyStringsToNull.php", "line": 31, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\TransformsRequest", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php", "line": 21, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TrimStrings.php", "line": 51, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\TransformsRequest", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\TrimStrings", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Http/Middleware/ValidatePostSize.php", "line": 27, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Http\Middleware\ValidatePostSize", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php", "line": 110, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Foundation\Http\Middleware\PreventRequestsDuringMaintenance", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Http/Middleware/HandleCors.php", "line": 49, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Http\Middleware\HandleCors", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Http/Middleware/TrustProxies.php", "line": 57, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 183, "function": "handle", "class": "Illuminate\Http\Middleware\TrustProxies", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php", "line": 119, "function": "Illuminate\Pipeline\{closure}", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php", "line": 176, "function": "then", "class": "Illuminate\Pipeline\Pipeline", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php", "line": 145, "function": "sendRequestThroughRouter", "class": "Illuminate\Foundation\Http\Kernel", "type": "->" }, { "file": "/var/www/vendor/laravel/framework/src/Illuminate/Foundation/Application.php", "line": 1188, "function": "handle", "class": "Illuminate\Foundation\Http\Kernel", "type": "->" }, { "file": "/var/www/public/index.php", "line": 17, "function": "handleRequest", "class": "Illuminate\Foundation\Application", "type": "->" } ] }

Gudetomi avatar Aug 05 '24 13:08 Gudetomi

Hi @Gudetomi , first of all you need to install Ollama locally https://ollama.com/ (you can also run Ollama via docker, but I think that it's easier and faster to run a local installation of it).

After installation, you can verify that Ollama is correctly installed by running:

ollama

You should see the various options of the command.

Then you need to download a model, for example:

ollama pull llama2

Once the model is downloaded, you can run it with the following command:

ollama run llama2

Then you can run your PHP application with LLPhant:

$config = new OllamaConfig();
$config->model = 'llama2';
$chat = new OllamaChat($config);

f-lombardo avatar Aug 06 '24 09:08 f-lombardo

Hi @f-lombardo , I ran Ollama via docker, trying to use llama3.1 but this error occurs.

Gudetomi avatar Aug 06 '24 13:08 Gudetomi

@f-lombardo Would there be any possibility of using GroqClud?

Gudetomi avatar Aug 06 '24 14:08 Gudetomi

@f-lombardo Would there be any possibility of using GroqClud?

From a technical point of view, I don't see problems. Try to create an issue for that, so we can verify how many people are interested in it and we can understand if anyone wants to create an implementation.

f-lombardo avatar Aug 06 '24 17:08 f-lombardo

Hi @f-lombardo , I ran Ollama via docker, trying to use llama3.1 but this error occurs.

You could try these steps:

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

docker exec -it ollama ollama run llama3.1

f-lombardo avatar Aug 06 '24 17:08 f-lombardo

Hi @Gudetomi ,

Depending on wheter you are running your PHP application and your Ollama server, there could be various settings for the url properties of the OllamaConfig:

Ollama PHP url
local local http://localhost:11434/api/
local docker http://host.docker.internal:11434/api/
docker local http://localhost:11434/api/
docker docker http://ollama:11434/api/

In the above schema I supposed that:

  1. If Ollama runs in a docker container, it exposes the 11434 port
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
  1. If both ollama and PHP are running inside docker containers, they are started with docker compose and the Ollama service is called ollama

f-lombardo avatar Aug 07 '24 08:08 f-lombardo

Depending on wheter you are running your PHP application and your Ollama server, there could be various settings for the url properties of the OllamaConfig:

Hi @Gudetomi , did this suggestion fix your problem?

f-lombardo avatar Aug 15 '24 09:08 f-lombardo

@f-lombardo I was limited in GPU memory to use Ollama, as it is for academic work I will use Mistral as it uses the model online. Thank you for your attention!

Gudetomi avatar Aug 15 '24 11:08 Gudetomi