Action Server does not handle output size

[kariharju]

Issue by mkorpela Sunday Jan 14, 2024 at 11:53 GMT Originally opened as https://github.com/robocorp/robocorp/issues/152

---

What happened?

• Current Status *

Current Action Server @action has limitless return value size. OpenAI GPT and Large Language Models in general work with limited size tokenized texts. OpenAI GPT allowed return value size is around 3500 ASCII characters. After that it will just error on Action use.

@action
def kaboom():
    return "A"*10000

This means that actions that return too big outputs will always fail from the callers (clients) point of view.

• How should it work *

I propose that Action Server internally truncates the outputs. The exact max size number could be configured in the Web UI.

Reference implementation:

def truncate_output_with_beginning_clue(output: str, max_chars: int = 2000) -> str:
    beginning_clue = "[Cut] "  # A very short clue at the beginning to indicate possible truncation

    if len(output) > max_chars:
        truncated_output = output[:max_chars - len(beginning_clue)]
        chars_missed = len(output) - len(truncated_output)
        truncated_message = f"[+{chars_missed}]"
        return beginning_clue + truncated_output + truncated_message
    else:
        return output

System Info

• 0.0.13 * Python 3.11.5 * OSX *