Avoid sending long or confusing keys to the LLM

[mindplay-dk]

While trying to understand what lingo.dev does and what it's exchanging with a (custom) LLM, I noticed this:

In order to have legible source code, we use English source text for keys in our app.

When lingo.dev sends this data in a prompt, it's including these very long keys in the request/response.

This seems really unncessary and practically doubles the number of required input tokens.

Instead of sending long keys, why don't you map the keys to simple base-1 sequential keys instead? (1, 2, 3, etc.)

This would:

1. cut the number of input tokens almost in half (cost savings)
2. speed things up (at least a little - input tokens are pretty fast, but still)
3. avoid potential interference from keys being added to context (!)
4. save the environment a little 😇🍃

To explain my third point a little, what I mean is, an LLM will see English text in keys as English text, which does affect the translation context - I've noticed some LLMs (e.g. LLama 4) have a tendency to "overreact" or go completely off the rails if you repeat stuff. The keys aren't relevant context and really don't belong in the query. Whereas sequential numerical keys in JSON data are almost guaranteed to be ignored (and repeated verbatim) by any LLM.

This could probably be done with an ILoader implementation that gets plugged into (at least) the JSON bucket loader, I think?

[seppviljar]

Just curious - how would you decide which ones are unnecessary? Length? When key == value? Something else?

[mindplay-dk]

In our case, all of the keys are unnecessary - the English keys are an implementation detail.

Sometimes there's a typo in the English text in the source code, and our staff updates the English "translation" from a control panel - so the English source texts are either (worst case) wrong or (best case) redundant.

Even if our keys had meaning (user.auth.login etc.) I wouldn't personally trust that this meaning has any relevance to translation, since the internal, technical language within a codebase is often different from the "official" product terminology. In my experience, product language often evolves, independently of technical language, over time.

So in either case, it might be useful to have this feature, probably as a configurable option.

I wouldn't trust a heuristic to guess whether the keys are important or not.

(and personally I would default to the assumption that they're not.)

[maxprilutskiy]

Makes total sense. I want us to ship it!

Here's some design notes:

1. omitKeys: boolean flag, that will be off by default, that replaces keys with numeric ids, right before those are passed into LLM.
2. the flag should be added in new, 1.9 config schema
3. the logic should be implemented as a separate loader that functions only when omitKeys is true, and does nothing otherwise
4. the loader should be applied to all bucket types (index.ts)
5. the new loader should have unit tests that validate its behavior in isolation

This one is a medium complexity i'd say, lmk if anyone is interested!

[namanguptagit]

Thanks for the clear explanation! I’d like to work on this feature.

My plan is to add an omitKeys option in the new 1.9 config schema, as mentioned. When omitKeys is set to true, the loader will replace long string keys with simple numeric ones (1, 2, 3, …) before sending data to the LLM.

Here’s roughly how I’ll approach it:

1. Create a new loader (maybe OmitKeysLoader) that activates only when omitKeys is true.
2. The loader will map original keys to numeric IDs and reverse the mapping when needed.
3. Integrate this loader into index.ts so it works across all bucket types.
4. Add unit tests to confirm the transformation logic works correctly and doesn’t affect other data.

Once the base version works, I’ll open a PR for feedback and improvements.

[mindplay-dk]

2. The loader will map original keys to numeric IDs and reverse the mapping when needed.

In my own translator, I also validate the keys for completeness - it's a somewhat marginal case, but LLMs do sometimes miss a key, or they mess up the JSON syntax (which is why we use something like jsonrepair) causing a key to go missing. (I have a console.warn for retries, and it does happen, with every LLM I've tested.)

What I do, is I treat the incoming keys as a queue - if a key goes missing, that key gets pushed back into the queue, so the translator can pick it up and retry it in a subsequent next batch. This way, the request isn't wasted, we don't end up doing extra "small requests" just to retry a single key, and no keys are missed.

Not sure how well these ideas fit with Lingo, but I thought I'd share them. 🙂

(I ended up writing my own translator because my needs were much simpler, and it was just faster for me than figuring out how to work this into an existing translator. @namanguptagit kudos for attempting to build this for everyone to enjoy. 🙂)

[namanguptagit]

Happy to help ☺️