chatcraft.org
chatcraft.org copied to clipboard
tarasglek
Secrets support for tools
I would like to use chatcraft as a convenient UI to call APIs. Those APIs have secrets. Need something like cloudflare env vars for secrets.
I imagine the flow to be like this:
- I write myself a system prompt and a gist to convert pdfs to markdown using https://mathpix.com/docs/ocr/creating-an-api-key
2)the gist will declare dependency on process.env.MATHPIX_OCR_API_KEY
3)When we issue the function call, chatcraft will issue error "please enter key MATHPIX_OCR_API_KEY in options dialog", pop up options.
4)We have a UI in options that lets one enter keys, keys are only saved in localstorage
- We can now safely share chatcraft chats with tool-using secret-requiring system prompts
@tarasglek, what if we invoke functions with a second arg:
/**
* data: object args from LLM
* chatcraft: object that app passes to the function with extra features/data
*/
export default function(data, chatcraft) {
const { value } = data;
const apiKey = chatcraft.env.get('SOME_API_KEY');
// use apiKey with value...
};
A function can optionally use the chatcraft object to get extra features. For now, exposing .env might be all we do, but later we could also put other things on there (e.g., API access to message history, websocket or other communication channel, pre-built convenience functions, ability to call LLM directly using stored API key, etc).
If we make .env.get() a function, we can use the call to pop-up some custom UI for obtaining it, or pull it from storage if we already have it.
This way we don't have to extend the metadata for a function (i.e., we don't have to explicitly declare the secrets we need).
Dump all this onto the global? I don't love that idea, but I suppose it's no less secure than anything else we are planning here.
Thinking about this more, let's not pollute the global: if we namespace this stuff, we won't risk collisions with other libraries we pull in within the app or function.
I suggest: window.ChatCraft so you can use ChatCraft.env for example.
interesting extension of the sops concept. you could totally store the private key as a password in your password manager, use that to unlock some random yamls with passwords hosted somewhere
@rjwignar I know you have some interest in working on low level code, and this might be something you'd find interesting.
The problem here is that we'd like to be able to allow users to store secrets (e.g., API Keys) securely in the app (i.e., in the browser's storage), and then allow the code to use them when needed. For example, imagine that I want to write a function for the LLM to use which needs to use some service that requires an API Key.
You've already had some experience with sops, which lets us encrypt and store secrets in a git repo. We're now wondering about using the same mechanism to do it in the browser.
Making this possible would require running sops in the browser, which means using WASM. The sops code is written in Go and Rust, both of which can compile to WASM. We could explore whether this is possible (it might not be).
Alternatively, we could use (age)[https://github.com/FiloSottile/age] via WASM on its own, or even just the web crypto APIs.
You don't have to take this on, but FYI, it's one of the things we're thinking about.
I read a little bit on WASM during the weekend after you brought this to my attention. I think this is something I'd like to work on.
So far, I've followed a couple YouTube tutorials and managed to use Emscripten to compile a simple C program into WASM, but that's a program written entirely in C. If the sops source code is divided between Go and Rust code I wonder if there's a way to compile it to WASM.
I took a look at the repo's Language stats and ~5% of the repo code is Rust:
That 5% is all in one Rust file: which appears to be a Unit Test library.
If that's the case, then we should be able to compile the Go code into WASM (assuming all the actual source code is in Go).
