Add Ollama Support

[P3GLEG]

Add Support for Ollama

Hey team,

First off, I want to say I’m a big fan of this repo and can't wait to see more of it. I’ve added support for Ollama, which simplifies prototyping on macOS for us GPU/API poor people. Since the OpenAI spec is marked as experimental and subject to change, I opted for a straightforward HTTP client implementation that aligns with your existing parameters.

Overview of Changes

• Refactored openai_utils.py into a new module: tinytroupe.clients.
• Moved default parameters and config objects into a new tinytroupe/clients/config.py.
• Moved client registry into tinytroupe/clients/__init__.py.

Usage Instructions

To get it running, execute the following in your terminal:

ollama pull Gemma-2-Ataraxy-v2-9B-2:latest 
ollama serve

Change your config.ini file to reflect

API_TYPE=ollama
MODEL=Gemma-2-Ataraxy-v2-9B-2:latest #Best on leaderboard today https://eqbench.com/creative_writing.html
MAX_TOKENS=8192 

Set the API key to

export OPENAI_API_KEY="ollama"

The implementation works well overall, but I’ve noticed occasional JSON parsing failures when retrieving cognitive_state. For example, the issue commonly occurs here:

@repeat_on_error(retries=5, exceptions=[KeyError])
        def aux_act_once():
            role, content = self._produce_message()

            self.episodic_memory.store({'role': role, 'content': content, 'simulation_timestamp': self.iso_datetime()})
            cognitive_state = content["cognitive_state"]

I think maybe we could add guidanceor lm-format-enforcer to help reduce it so you're also not wasting money on API calls to keep retrying due to badly formatted json. We may even be able to get the prompt size down. Let me know if that sounds interesting to you.

Also, in the config.ini you specified MAX_TOKENS=4000, is that related to the agents or content length accepted by the model? The docs are showing it can be 16384

Cheers, Paul

[P3GLEG]

@microsoft-github-policy-service agree

[aminekhelif]

@P3GLEG either ollama, azure or openai clients require the set_api_cache method, which is not implemented in the Ollama client

[paulosalem]

Oh, this is very interesting, thank you! I'll need to take some time to review it though, as these are substantial changes, so please give us some more time.

[OriginalGoku]

This is a great addition to the project. I was about to write something very similar for myself. Would really appreciate if @paulosalem could give us a time line to merge this. Looking forward to play around with this on my local computer

[OriginalGoku]

@P3GLEG Ollama now support Pydantic Json generators so we don't need to integrate this with guidance or any format enforcer. I'd be glad to assist and implement the necessary changes to define the Pydantic class to define the general output structure of the models.

[kjozsa]

@P3GLEG I admittedly did not spent much time with your implementation, but wouldn't it make sense to use https://github.com/andrewyng/aisuite for unifying the LLM calls?

[4F2E4A2E]

Thank you for this PR. I was unable to successfully run the simulation using Ollama with either the fork from this PR or the one from [1]. Here's the code I used:

1: https://github.com/OminousIndustries/TinyTroupeOllama.

import json
import sys
sys.path.append('..')

import tinytroupe
from tinytroupe.agent import TinyPerson
from tinytroupe.environment import TinyWorld, TinySocialNetwork
from tinytroupe.examples import *

# Initializing agents
lisa = create_lisa_the_data_scientist()
oscar = create_oscar_the_architect()

# Creating the world and making everyone accessible
world = TinyWorld("Chat Room", [lisa, oscar])
world.make_everyone_accessible()

# Running the simulation
world.run(steps=4)

# using python 3.12
git clone --depth 1 https://github.com/P3GLEG/TinyTroupe.git
cd TinyTroupe && pip install .

python TinyTroupe/test.py

!!!!
DISCLAIMER: TinyTroupe relies on Artificial Intelligence (AI) models to generate content. 
The AI models are not perfect and may produce inappropriate or inacurate results. 
For any serious or consequential use, please review the generated content before using it.
!!!!

Looking for default config on: /app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/config.ini
Failed to find custom config on: /app/acme/acme-ai-tiny-troupe/config.ini
Will use only default values. IF THINGS FAIL, TRY CUSTOMIZING MODEL, API TYPE, etc.

=================================
Current TinyTroupe configuration 
=================================
[OpenAI]
api_type = ollama
base_url = http://host.docker.internal:11434/v1
model = gemma-2-Ifable-9B
max_tokens = 8192
temperature = 0.3
freq_penalty = 0.0
presence_penalty = 0.0
timeout = 60
max_attempts = 5
waiting_time = 1
exponential_backoff_factor = 5
embedding_model = text-embedding-3-small
cache_api_calls = False
cache_file_name = openai_api_cache.pickle
max_content_display_length = 1024

[Simulation]
rai_harmful_content_prevention = True
rai_copyright_infringement_prevention = True

[Logging]
loglevel = ERROR

─────────────────────────────────────────────────────────────────────────────────────────── Chat Room step 1 of 4 ───────────────────────────────────────────────────────────────────────────────────────────
Lisa --> Lisa: [THOUGHT] 
          > I will now act a bit, and then issue DONE.
Lisa --> Lisa: [THOUGHT] 
          > I will now act a bit, and then issue DONE.
Lisa --> Lisa: [THOUGHT] 
          > I will now act a bit, and then issue DONE.
Lisa --> Lisa: [THOUGHT] 
          > I will now act a bit, and then issue DONE.
Lisa --> Lisa: [THOUGHT] 
          > I will now act a bit, and then issue DONE.
Traceback (most recent call last):
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/test.py", line 23, in <module>
    world.run(steps=4)
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/control.py", line 542, in wrapper
    result = transaction.execute()
             ^^^^^^^^^^^^^^^^^^^^^
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/control.py", line 427, in execute
    output = self.function(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/environment.py", line 126, in run
    agents_actions = self._step(timedelta_per_step=timedelta_per_step)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/control.py", line 542, in wrapper
    result = transaction.execute()
             ^^^^^^^^^^^^^^^^^^^^^
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/control.py", line 427, in execute
    output = self.function(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/environment.py", line 86, in _step
    actions = agent.act(return_actions=True)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/control.py", line 542, in wrapper
    result = transaction.execute()
             ^^^^^^^^^^^^^^^^^^^^^
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/control.py", line 427, in execute
    output = self.function(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/agent.py", line 483, in act
    aux_act_once()
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/utils.py", line 118, in wrapper
    raise e
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/utils.py", line 114, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/app/acme/acme-ai-tiny-troupe/TinyTroupe/tinytroupe/agent.py", line 436, in aux_act_once
    cognitive_state = content["cognitive_state"]
                      ~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'cognitive_state'