Support callbacks/tool use in JSON generation

[wjn0]

Is your feature request related to a problem? Please describe. When generating a complex schema, sometimes I want to interleave generation, templating, and tool use. One way to support this would be via callbacks attached to a particular JSONpath.

Describe the solution you'd like

class Book(BaseModel):
    title: str
    isbn: str

class Author(BaseModel):
    name: str
    books: List[Book]

def lookup_isbn(generation_state):
  isbn = _lookup_isbn_by_author_and_title(author=generation_state.current_object.parent.name, title=generation_state.current_object.title)
  return isbn

lm += prompt
callbacks = {r"author.books[\d].isbn": lookup_isbn}
lm += guidance.json(schema=schema, callbacks=callbacks)

Describe alternatives you've considered Another way would be to support temporarily interrupting generation somehow, but I think this reduces down to something very similar to the callback approach. A less clean approach (IMO) would be inheritance-based, pushing the current JSON library methods under a class and allowing the user to inherit from them, e.g. overriding _gen_json_object and hooking in that way.

Additional context See also discussion on #876 and #889 (some kind of lazy approach is likely necessary here)