breadboard
breadboard copied to clipboard
Implement a simple ELT provider
As mentioned in #1037, Breadboard wants to have an extract/load/transform (ELT) provider that allows managing things other than text: PDFs, docs, images, videos, audio.
Here's a couple of scenarios:
- I want to upload a few PDF files as part of the "persona" prompt in the node. I expect that Breadboard will keep it there.
- I uploaded an audio file when running the board. I want to run again. Can I just reuse the file? Do I have to upload it again?
Let's figure out how to build the simplest-possible ELT provider that relies on IDB. Here's what we need:
- The ability to upload, store, and choose various files as part of the
llm-contenteditor - A way to store multimodal content in configuration
- Being able to pass this data efficiently as part of the
llm-contentdata that's passed across the wire - (later?) The ability to manage these files in one place, so that I could zap some files
Let me ponder the multimodal content in config first
There's something here about saving an llm-content as one item. For instance, imagine I have a board that represents a single turn of a chat. The board takes llm-content as input and llm-content as output (aka "worker shape"). It would be exceedingly cool if I could save the output and then, when running the board again, pick the saved llm-content from some store of saved things. This would allow me to debug worker-shaped boards very easily and even simulate multiple turns
omg, I just realized something. With FSAPI, we can save a .bb directory on disk and store settings and ELT bits there.
For now:
- let's just stuff base64 bits directly into configuration values in BGL, and see where it goes. No change needed here.
- add a simplest possible "Run Again" capability: remember the inputs entered so that they could be used again in the next run (instead of entering them again from scratch)
wait... I can just reuse saved runs
Okay, with InspectableRun.inputs(), I think we have what we need. This is still an important issue, but we can decouple it from the Easy Mode MVP.