griptape
griptape copied to clipboard
Add a Tool to display/play media
- [x ] I have read and agree to the contributing guidelines.
Is your feature request related to a problem? Please describe. When working with Image Generation Drivers (and eventually video, audio, and any other media), there's no built in way to display that media.
Describe the solution you'd like I would like a tool that I can add to a Griptape Structure that will display the media. For example, I could have an agent generate an image, and then ask it to display the image and it would open it in an appropriate application on my computer. If it's an audio file, then it could play the audio file for me.
Describe alternatives you've considered Because we're using Python, I have created my own tool to do this for displaying images, but it'd be better if it were a part of Griptape so others could do it, and if it were extended to handle all media types.
Additional context An example of a ViewImage tool
from schema import Schema, Literal
from attr import define
import os, subprocess, sys
from griptape.artifacts import TextArtifact, ErrorArtifact
from griptape.utils.decorators import activity
from griptape.tools import BaseTool
def open_image(image_path):
if sys.platform == "win32":
os.startfile(image_path)
elif sys.platform == "darwin": # macOS
subprocess.run(["open", image_path])
else: # linux variants
subprocess.run(["xdg-open", image_path])
@define
class ViewImage(BaseTool):
@activity(
config={
"description": "View the image.",
"schema": Schema(
{
Literal(
"filename", description="The filename of the image to view."
): str,
}
),
}
)
def view_image(self, params: dict) -> TextArtifact | ErrorArtifact:
try:
filename = params["values"]["filename"]
agent_response = ""
# Make sure the path exists
agent_response = os.path.exists(filename)
if agent_response:
open_image(filename)
return TextArtifact(agent_response)
except Exception as e:
return ErrorArtifact(f"Error retrieving projects: {e}")