poml icon indicating copy to clipboard operation
poml copied to clipboard

GItHub (a Microsoft subsidiary) already has a format for this, but in YAML (and more accessible than a self-proclaimed SGML format)

Open Diablo-D3 opened this issue 4 months ago • 8 comments

https://docs.github.com/en/github-models/use-github-models/storing-prompts-in-github-repositories

Have you considered making tooling around this instead? This already is supported by Github and Microsoft.

Diablo-D3 avatar Aug 10 '25 12:08 Diablo-D3

I hope so. But the project needs a lot more efforts and love to accomplish this goal.

ultmaster avatar Aug 10 '25 12:08 ultmaster

There's also https://github.com/microsoft/prompty, https://github.com/microsoft/prompt-engine, https://github.com/microsoft/promptflow, ... None of these support POML, and POML isn't referenced at all in microsoft agent libraries like semantic kernel or autogen, or industry standard prompt optimization frameworks like DSPy.

There's also competitors like https://github.com/BoundaryML/baml.

I'm not trying to poo-poo POML, I'm just struggling to see what value it provides that other tools don't, and the lack of integration within microsoft's own product family is concerning

ahgraber avatar Aug 10 '25 15:08 ahgraber

I think the first batch of frameworks I'm going to integrate with are observability frameworks like mlflow, weave and agentops.

Microsoft is a very very large company. And I don't see those "internal" ones are any closer to me than the external ones. But thanks for the reminder anyway.

ultmaster avatar Aug 10 '25 16:08 ultmaster

I'm just struggling to see what value it provides that other tools don't,

I don't understand what can be accomplished by commenting instead of contributing. If there's an element of contribution and it's found that there's lesser value, then makes sense.

For the effort of ultmaster, I express my gratitude.

pramodhbn avatar Aug 17 '25 13:08 pramodhbn

I don't understand what can be accomplished by commenting instead of contributing

Aren't you also just commenting? 🤔

But I think you make a fair point that comments should be actionable, even if they're critical.

I believe the concern from both @Diablo-D3 and myself is that the landscape for prompt management and templating is quite dense with options, and I don't see anything in the current POML readme or docs that makes it abundantly clear to me what POML's key differentiator is, why I should use it, or when I should consider to vs other alternatives (of which I previously listed several).

Making things more confusing, POML is listed under a Microsoft github account, and Microsoft itself has a variety of prompt templating and management projects. So again - what is POML's unique selling proposition?

To make this comment actionable, my request would be:

  • As a developer evaluating prompt management solutions I want to immediately understand POML's unique benefits and advantages So that I can quickly determine if it's the right choice for my needs

  • As someone considering various prompt templating tools I want clear documentation showing when to choose POML versus alternatives So that I can make an informed decision based on my specific requirements

  • As a developer exploring Microsoft's prompt management offerings I want clear differentiation between POML and other Microsoft-related projects So that I can understand its place in the broader ecosystem

ahgraber avatar Aug 17 '25 13:08 ahgraber

I believe that POML is designed to provide two core capabilities:

  1. Prompt structure abstraction: Prompts can be defined once in POML and then automatically rendered into multiple target formats. For example, if a model requires JSON-like prompts, the same POML definition can be exported as JSON without manual rewriting.

  2. Template variables for modularity: Common text fragments can be externalized and referenced across multiple prompts. This reduces duplication and simplifies prompt maintenance, testing, and iteration.

That said, I dont know of there are other tools of the ecosystem that already provides these features.

joaquinwojcik avatar Aug 17 '25 14:08 joaquinwojcik

I want to thank everyone for their comments and supports. I and my colleagues will keep working on this.

From what I see from a lot of discussions and comments, POML's readme and docs are currently very vague and misleading; I've seen a lot of misunderstandings about what POML is actually doing.

ultmaster avatar Aug 17 '25 17:08 ultmaster

Thanks @ultmaster! I was reading up more yesterday and was thinking along the similar lines; there seem to be a lot of different standards, all from Microsoft without explicit information about the differences and what to use when.

one-kash avatar Aug 26 '25 12:08 one-kash