kisli
Implementation of RFC 6532 sends email without activating SMTPUTF8
Your implementation of RFC 6532 may cause the sending of an email without activating SMTPUTF8 although it is mandatory according to RFC 6532:
messages in this format require the use of the SMTPUTF8 extension [RFC6531] to be transferred via SMTP
When an SMTP server announces that it is capable of SMTPUTF8, SMTPTransport::send calls https://github.com/kisli/vmime/blob/182e8f5dd8cc66f5636f684b7677ea1d5ac7574a/src/vmime/net/smtp/SMTPTransport.cpp#L428
Under certain circumstances, this keeps word::generate from outputting encoded words when it has to auto detect if encoding is neccessary:
https://github.com/kisli/vmime/blob/182e8f5dd8cc66f5636f684b7677ea1d5ac7574a/src/vmime/word.cpp#L460
For example, if you are in RFC 6532 mode and use a 'simple' subject header with umlauts, wordEncoder::isEncodingNeeded says not to encode:
https://github.com/kisli/vmime/blob/182e8f5dd8cc66f5636f684b7677ea1d5ac7574a/src/vmime/wordEncoder.cpp#L245-L284
Eventually, this leads to output of UTF-8 encoded text because of:
https://github.com/kisli/vmime/blob/182e8f5dd8cc66f5636f684b7677ea1d5ac7574a/src/vmime/word.cpp#L487-L500
SMTPUTF8 must be activated in this situation. It is, however, activated only if SMTPTransport::sendEnvelope deems it necessary:
https://github.com/kisli/vmime/blob/182e8f5dd8cc66f5636f684b7677ea1d5ac7574a/src/vmime/net/smtp/SMTPTransport.cpp#L210-L223
Since SMTPTransport::sendEnvelope only considers email addresses and knows nothing of the yet to be generated message, we're in a mess!
Is this ever going to be fixed? We're facing this problem increasingly because of the popularity of Office365 resp. Exchange servers (see also #209).
Our quick fix has been to revert your efforts to detect if SMTPUTF8 is really necessary and instead use it whenever the server supports it (#186). I know that this violates the following recommendation of RFC 6531:
If the SMTPUTF8-aware SMTP client is aware that neither the envelope nor the message being sent requires any of the SMTPUTF8 extension capabilities, it SHOULD NOT supply the SMTPUTF8 parameter with the MAIL command.
A proper solution would probably include some form of pre-pass over all headers (see also section 3.6 of RFC 6531):
The MAIL command parameter SMTPUTF8 asserts that a message is an internationalized message or the message being sent needs the SMTPUTF8 support. There is still a chance that a message being sent via the MAIL command with the SMTPUTF8 parameter is not an internationalized message. An SMTPUTF8-aware SMTP client or server that requires accurate knowledge of whether a message is internationalized needs to parse all message header fields and MIME header fields [RFC2045] in the message body.
But even then VMime doesn't honour section 3.5 of RFC 6531:
An SMTPUTF8-aware SMTP client MUST NOT send an internationalized message to an SMTP server that does not support SMTPUTF8. If the SMTP server does not support this option, then the SMTPUTF8-aware SMTP client has three choices according to Section 3.2 of this specification.
Couldn't we fix this easily by calling ctx.setInternationalizedEmailSupport(true) only if SMTPUTF8 extension is available and we detected it should be used (ie. if needSMTPUTF8 == true)?
I came back to this after several years because someone else has reported a similar problem, which was solved for now by setting their "use smtputf8" variable if both any address in the message has a non-ascii localpart and the next-hop server supports it.
I became curious what the concrete problem was with setting ctx.setInternationalizedEmailSupport(true) based only on whether the addresses require it, ignoring any other header fields. I know it's against the rules in the RFC, but surely that wasn't the problem you observed?
