microsoft
provide option for more ways to connect to LLM
Currently it doesn't support Amazon Bedrock or SAP AI core for example. Since this is open source, maybe adding more ways to connect to LLM would be helpful to larger community.
Screenshot from Cline below:
or it's meant to be spec that can be used in conjunction with Cline? The option to configure API key directly within VS Code extension is kinda confusing if it's meant to be just spec.
How do you plan to use POML with Cline? From what I see, Cline is another vscode extension providing code assistance. Do you want to send prompts built with POML to cline or whatever?
yes, I was thinking of using POML as a prompt library and then using Cline to execute the prompts. Then I saw that POML extension has its own LLM configuration option, which doesn't include the providers I want.
Looking for more guidance on what the future of POML is: are we saying that POML will be:
- Standalone extension with its own LLM config, in which case adding Amazon Bedrock and SAP AI Core would be helpful
- Meant to be a framework and can be used as prompt library for other extensions such as Cline
The LLMs integrations in POML vscode extension are called "testing the prompt" for a reason. They can used for debugging when you developing a prompt, not for daily interaction with one prompt, not for production use.
So who would develop a prompt? In my opinion, there are two kinds of users. One is LLM application (or agent) developers. They manage and test tons of prompts. That I believe, is the main targetted audience of POML. Check with this new example here: https://microsoft.github.io/poml/stable/tutorial/expense_part1/
The other kind is regular users. From what I observe, the regular use pattern of these users are ephemeral, one-time-use prompts; as well as multi-turn conversations. Such cases are not suitable for current POML vscode user experience (we focus on refining long-use single one prompt) -- and that's why we are expanding to outside vscode and to non-coding for prompt orchestration, though that's a whole new story.
To answer your question, I think POML is more like (2), but not exactly. Whether POML is the correct language to "program" your prompt depends on the nature of your prompt, for how long do you plan to use it, how complicated it is, does it contain rich context, how difficult it is too debug. Language is just a language after all. You need a scenario to tell whether the language is right for you, and the toolkit can only be useful when you use it right.