1-to-many transitions

[zilto]

Currently

Below is a valid Burr graph definition to "get a user input, and select the most appropriate action out of 3"

graph = (
  GraphBuilder()
  .with_actions(
    process_user_input,
    decide_next_action,
    generate_text,
    generate_image,
    ask_for_details,
  )
  .with_transitions(
      ("process_user_input", "decide_next_action"),
      ("decide_next_action", "generate_text"),
      ("decide_next_action", "generate_image", expr("mode==generate_image")),
      ("decide_next_action", "ask_for_details", expr("mode==ask_user_for_details")),
  )
  .build()
)

Notes:

• Needs a "decide node" that writes a value to state (i.e., mode).
• The "decide node" is the result of Condition objects requiring to return bool
• Has one transition per graph edge
• Requires at least 2 out of 3 explicit condition
• Conditions are evaluated sequentially

Desired solution

The main benefit of the above is that everything is explicit. But it can also be less ergonomic, add complexity to defining transitions, and be inefficient when computing transitions is expensive.

Consider the following API

ApplicationBuilder()
  .with_actions(
    process_user_input,
    generate_text,
    generate_image,
    ask_user_for_details,
  )
  .with_transitions(
    ("process_user_input", "decide_next_action"),
    (
      "decide_next_action",
      ["generate_text", "generate_image", "ask_user_for_details"],
      OneToManyCondition(...)   # TODO
    ),
  )
  .with_entrypoint("process_user_input")
  .build()
)

Note:

• No longer requires a "decide node"; this is moved to a OneToManyCondition object. When "resolved", it should return the index or the name of the next action.
• New transition syntax allowing 1 -> n. This allows to resolve multiple binary conditions at once
• arguably easier to read and manage code changes. Removing the sequential condition checking simplifies debugging
• less transitions to define

Use case

The popular use case I have in mind is "use an LLM to decide the next node". Given the Graph building process, it would be possible to dynamically create a model of "available next actions". Here's a sketch using instructor, which has better guarantees regarding structured LLM outputs (for OpenAI at least)

ApplicationBuilder()
  .with_actions(
    process_user_input,
    generate_text,
    generate_image,
    ask_user_for_details,
  )
  .with_transitions(
    ("process_user_input", "decide_next_action"),
    (
      "decide_next_action",
      ["generate_text", "generate_image", "ask_user_for_details"],
      LLMDecider()
    ),
  )
  .with_entrypoint("process_user_input")
  .build()
)

def create_decision_model(tos: list[FunctionBasedAction]):
    next_action_names = []
    description = ""
    for to in tos:
        if not to.__doc__:
            raise ValueError(f"LLMDecider: {to.name} needs to have a non-empty docstring.")
        
        next_action_names.append(to.name)
        description += f"{to.name}\n{to.__doc__}\n\n"

    return create_model(
        "next_action",
        next_action=(
            Literal[tuple(next_action_names)],
            Field(description="AVAILABLE ACTIONS\n\n"+next_action_descriptions)
        )
    )
 
 def _llm_resolver(state: State, llm_client, response_model) -> str:
    user_query = state["user_query"]

    next_action = llm_client.chat.completions.create(
        model="gpt-4o.mini",
        response_model=response_model,
        messages=[
            {"role": "system", "content": "You are an automated agent and you need to decide the next action to take to fulfill the user query."},
            {"role": "user", "content": user_query}
        ]
    )

    return next_action
 
 # apply something along those lines
 condition = Condition(keys=["user_query"], resolver=partial(_llm_resolver, llm_client=...,  response_model=create_decision_model)