mailtrain icon indicating copy to clipboard operation
mailtrain copied to clipboard

Encoding problem with merge tags and Gmail

Open marcinkunert opened this issue 5 years ago • 25 comments

I'm experiencing a problem with encoding on Gmail side.

Example email content:

Cześć [FIRST_NAME]!

Where the FIRST_NAME is "Michał". The user will see it as follows:

Cześć Michał!

The source of the email looks like follows:

Cze=C5=9B=C4=87 Michał!

The non-merge-tag characters are encoded email like, but the merge tag special character is encoded the html way and Gmail can't handle it properly. Thunderbird displays the message properly.

I've already tried changing the encoding (first line of email body) to:

  • <meta charset="utf-8">
  • <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  • <meta charset="utf-8" http-equiv="Content-Type" content="text/html; charset=utf-8">

No luck though. Any idea how to tackle this issue?

marcinkunert avatar May 10 '19 12:05 marcinkunert

Same issue, just posted a duplicate I believe. If someone can test that on v2, it would be great (If it works, we will migrate :)).

phtmgt avatar Jul 04 '19 14:07 phtmgt

Reproducible in V2

D4rk4 avatar Sep 16 '20 14:09 D4rk4

Any plans to address that? It's not like this is a minor issue, 250M people use Cyrillic. I will try to find the issue in the source code.

phtmgt avatar Sep 29 '20 05:09 phtmgt

Any plans to address that? It's not like this is a minor issue, 250M people use Cyrillic. I will try to find the issue in the source code.

Try this - https://github.com/D4rk4/mailtrain/commit/da2e945906774b49dd9aa0df73dc3fc934b2e837

Това коригира грешка с кирилица. Но не съм сигурен, че разработчиците някога ще поправят това.

D4rk4 avatar Sep 29 '20 13:09 D4rk4

Thanks for the solution, @D4rk4 ! Will try it promptly on V2 code.

phtmgt avatar Sep 30 '20 16:09 phtmgt

Thanks for the solution, @D4rk4 ! Will try it promptly on V2 code.

It's a very dirty hack. I'm not a NodeJS developer. If you experienced with NodeJS, please just replace he.encode to another solution with Cyrillic support.

D4rk4 avatar Oct 04 '20 20:10 D4rk4

The whole notion of HTML encoding characters in the body of an e-mail is plain wrong. It doesn’t only botch Cyrillic, it messes with all non-standard characters. So, we don’t need to replace he.encode, we need to get rid of it.

phtmgt avatar Oct 05 '20 05:10 phtmgt

The encoding with he has been adopted from Mailtrain v1. I'm not sure why @andris9 introduced it there, but I'm quite confident he must have had some reason for it. I'll be happy to drop the encoding, but first I would like to have understanding why it was there and what could get bad by removing it. We should also properly test it with different mail clients (Outlook is definitely one of those).

bures avatar Oct 05 '20 06:10 bures

@bures without HTML encoding Outlook working with Cyrillic merge tags like a charm. Also tested in Gmail.

D4rk4 avatar Oct 05 '20 09:10 D4rk4

@D4rk4 - Just to double check, you mean Outlook desktop app, right? (Not the web-based Outlook in M365.)

bures avatar Oct 05 '20 10:10 bures

@andris9 - Please, any idea why the html entity encoding was introduced there? Thanks.

bures avatar Oct 05 '20 10:10 bures

@D4rk4 - Just to double check, you mean Outlook desktop app, right? (Not the web-based Outlook in M365.)

OWA and desktop app.

D4rk4 avatar Oct 05 '20 13:10 D4rk4

Thanks. Win or Mac?

bures avatar Oct 05 '20 13:10 bures

Thanks. Win or Mac?

Win

D4rk4 avatar Oct 05 '20 13:10 D4rk4

Up! Still not fixed in V2

D4rk4 avatar May 23 '21 16:05 D4rk4

@bures Any progress with remove "he.encode"?

D4rk4 avatar Aug 27 '21 07:08 D4rk4

So, any progress?

D4rk4 avatar May 06 '22 11:05 D4rk4

I'm using a widely patched version of MT without he.encode a couple of years, without any issues with encoding. Please remove he.encode, from the main tree...

D4rk4 avatar May 06 '22 12:05 D4rk4

@bures

D4rk4 avatar May 06 '22 12:05 D4rk4

@D4rk4 you can make a pull request.

talheim-it avatar May 06 '22 12:05 talheim-it

@talheim-it this pull request? https://github.com/Mailtrain-org/mailtrain/pull/969

D4rk4 avatar May 06 '22 12:05 D4rk4

@D4rk4 is it v2 approved? If yes I will check to reach @bures for verification.

I just had not time to dig into the code.

talheim-it avatar May 06 '22 12:05 talheim-it

I'm do new one for fresh code

D4rk4 avatar May 06 '22 12:05 D4rk4

@D4rk4 thanks, mention me and I am behind to treach him!

talheim-it avatar May 06 '22 12:05 talheim-it

https://github.com/Mailtrain-org/mailtrain/pull/1252

D4rk4 avatar May 06 '22 12:05 D4rk4

We are going to start with the development and testing of mailtrain v3 in the next weeks.

You are welcome to help us with the testing as soon as the first release candidate is available.

talheim-it avatar Mar 16 '24 08:03 talheim-it