[Feature]: Freeze/lock/cache outputs for arbitrary nodes?

[aaronsantiago]

trafficstars

Feature Request

Especially when parsing large documents that may not be stable enough for the caching in the LLM requests to stick, it would be nice to save the output from arbitrary nodes or subgraphs to save costs during development and testing.

For example--pinging a web API might return slightly different results causing the entire chain to be processed even if the inputs don't change.

I would imagine this working either:

• TouchDesigner style where you click the lock button and the output is set to the current state for all further requests
• The way the chat node currently works where if the input does not change, the output also does not change.

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[abrenneke]

I think that's a great idea! Thanks!

[mindplay-dk]

This should probably go hand-in-hand with a "seed" option?

An option to seed the LLM, that is - in order for caching to make sense, you want to make sure the LLM-based nodes are always returning the same result.

I haven't use the test suite feature yet (Trivet) but it would probably useful in that context, too - but regardless of whether you use tests, being able to globally freeze/cache all the LLM-based nodes would make it a lot easier to design and develop things.

For example, if you were to change the prompt to an LLM, that would invalidate the cache, and the renewed output in turn might invalidate the cache of other blocks - which would also speed up development in general, since you're now only waiting for things that actually have a reason to run again. 🙂