Native support for templating messages

[hos]

We have a custom message distribution service, which should send messages to a segment of our users. I'm planning to add an option for templating, for example:

{{username}}, you won {{amount}}{{currency}} for being awesome.

This is easy task to do using some templating libraries or i18n libraries, if the text don't contain formatting. But with formatting it will break. The reason for that, is that depending on the length of the variables inside the template, the offset of entities may become invalid/wrong.

Note: the message "templates" are stored as json, serialized message object which bot will get as a message.

So I have multiple questions here:

1. Are there some simple solutions I don't see? For example convert to html, replace variables. (I prefer to keep entities to debug them later easier).
2. Are there any plans to add an utility function to grammy which will fix the entities. (check the diffs and make the shifts in entities)?
3. Is this a good fit for plugin ideas?
4. Do you see this as a different library?

Thank you very much for your great work on this project. This library is one of my most beloved libraries in javascript ecosystem.

[rojvv]

Doesn’t https://github.com/grammyjs/i18n/ already do this?

[KnorpelSenf]

The parse-mode plugin has the fmt helper that does exactly this. However, it's currently not ideal to combine it with templates that are stored in files, you'll effectively have to define them as code.

I'd like to support some sort of formatted strings natively in the library so that no plugin is needed for it. That way, other plugins can depend on the utilities. This should enable us to improve the formatting support inside other plugins.

[hos]

@rojvv will do that for predefined messages, which you have in your code, also with translations. This requires

1. Deploy for each new message distribution.
2. Access to the code to craft and send a message. For example if you have a marketing team, they will need to ask you to add that one time message to the code.

The main "feature" here is that i want to allow users, to send a telegram message to the bot and use it as a template, reusing all the attached media and text formatting. This will allow to add templates dynamically, add some target audiences to this template and run the distribution tasks with no developer involved. This of it like mailchimp, sendgrid alternative for telegram. But instead of running paid service, make a library which can be added to any project on nodej.js, deno (bun?).

[hos]

@KnorpelSenf I'm using fmt for all my formatting in code, were I messages text and format is not going to change. What I don't get yet, is how to "normalize" entity's offset, length properties after the variables are inserted and the length of the original words have been changed.

Currently I have implemented this, (no variables/templating support)

// ~pseudo-code
.on('msg', async (ctx) => {
  template.message = ctx.msg;
});

.command('/send', async (ctx) => {
const templateId = ctx.text.split(' ')[1]

// run a function every second for 30 users
const text = template.message.caption || template.message.text;
const entities = template.message.caption_entities?.length ?template.message.caption_entities : template.message.entities;
// check if it contains photo or media send media

// My plan is to replace placeholders in the template at this point
const finaltext = liquidjs(text, { ...variables })

bot.api.sendMessage(chatId, text, {
  entities // entities may be invalid here
})

I'm thinking about a function which will fix any text shifts providing only

const newEntities = fix(originalText, outputText, oldEntities)

If using custom template replacing logic, it should be easy to fix the entities, something like this (AI generated code)

function normalizeEntities(template, variables, entities) {
    let resultText = "";
    let offsetAdjustment = 0;
    const newEntities = [];

    const placeholderRegex = /{{(.*?)}}/g;

    let lastIndex = 0;
    template.replace(placeholderRegex, (match, key, index) => {
        // Append text before the placeholder
        resultText += template.slice(lastIndex, index);

        // Add the replacement value
        const replacement = variables[key] || "";
        resultText += replacement;

        // Calculate the length difference
        const lengthDifference = replacement.length - match.length;

        // Adjust entities affected by this replacement
        entities.forEach((entity) => {
            const entityEnd = entity.offset + entity.length;

            if (entity.offset >= index + offsetAdjustment) {
                // Entity starts after the replacement
                entity.offset += lengthDifference;
            } else if (entityEnd > index + offsetAdjustment) {
                // Entity overlaps with the replacement
                entity.length += lengthDifference;
            }
        });

        // Update offsetAdjustment and lastIndex
        offsetAdjustment += lengthDifference;
        lastIndex = index + match.length;
    });

    // Append remaining text
    resultText += template.slice(lastIndex);

    // Return normalized text and updated entities
    return { text: resultText, entities };
}

[KnorpelSenf]

I see. There is currently no library that works like liquidjs but also preserves entities. While I want basic string operations to be supported with grammY in the future, I don't think that a full templating engine is in scope.

It would be cool to have this as a plugin, though! This use case isn't really covered in any way yet. If you end up writing something like this, please drop us a link here or in https://t.me/grammyjs so can consider making it an official plugin. In any case, it can be listed as a third-party plugin on the website.