[Bug]: Unable to make SSL connection to SMTP server

[skarpik]

trafficstars

Is there an existing issue for this?

• [X] I have searched the existing issues

What happened?

Created a test DNN v9.13.00 website. Attempted to set up an SSL connection to the SMTP server. Tested SMTP connection. Got failure message. See Current Behavior below.

Steps to reproduce?

1. Created DNN v9.13.00 website on my workstation using NVQuickSite.
2. Set up SMTP connection with the following settings: +++++++ SMTP server: strontium.namespro.ca SMTP port: 465 SMTP SSL: On Username: (my userid)@basilthetortoise.com Password: (my password) +++++++
3. Clicked on "Save" then "Test SMTP Settings". Connection failed with a "There is a problem with the configuration of your SMTP Server..." error message (see Current Behavior below).

Current Behavior

Expected Behavior

Used an online SMTP test service to make sure that the SMTP server was operating properly and there was no error in the credentials. The output is below:

Connected to smtps://strontium.namespro.ca:465/ << 220 outbound.strontium.namespro.ca ESMTP MailEnable Service, Version: 10.29-10.29- ready at 10/08/23 13:08:12

EHLO [172.31.11.248] << 250-strontium.namespro.ca [54.212.131.181], this server offers 4 extensions << 250-AUTH CRAM-MD5 PLAIN LOGIN << 250-SIZE 40960000 << 250-HELP << 250 AUTH=LOGIN AUTH CRAM-MD5 << 334 PDYxNjYwLjQzMzc1MDQ2OEBuYW1lc3Byby13aDI5Pg== bnA2MDUxM0BiYXNpbHRoZXRvcnRvaXNlLmNvbSAzNTdmMGU5YWNkZDI3ZmIyMTE5NjZmZGI3Y2Q4MjQwMg== << 235 Authenticated CRAM-MD5 MAIL FROM:[email protected] SIZE=576 << 250 Requested mail action okay, completed RCPT TO:[email protected] << 250 Requested mail action okay, completed DATA << 354 Start mail input; end with <CRLF>.<CRLF> From: [email protected] Date: Sun, 08 Oct 2023 19:08:09 퍍 Subject: SMTP test from strontium.namespro.ca Message-Id: 22F42BMX6LU4.72FL4XX3TPWL2@WIN-AUIR3RRGP88 To: [email protected] MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=-wg/fb0gvPkmBW78cwmOnxQ=="

--=-wg/fb0gvPkmBW78cwmOnxQ== Content-Type: text/plain; charset=utf-8

Test message --=-wg/fb0gvPkmBW78cwmOnxQ== Content-Type: text/html; charset=utf-8 Content-Id: 22F42BMX6LU4.NUO6X6245L1Y@WIN-AUIR3RRGP88

Test message --=-wg/fb0gvPkmBW78cwmOnxQ==-- . << 250 Requested mail action okay, completed

Relevant log output

none

Anything else?

nothing

Affected Versions

9.13.0 (latest release)

What browsers are you seeing the problem on?

Microsoft Edge

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[valadas]

@skarpik would you mind installing a clean 9.12.0 and trying the same config. There was some changes regarding email providers in 9.13.0 and want to learn if it may be a new bug in 9.13.0 or not.

Thanks.

[david-poindexter]

@skarpik I am pretty sure you may be running into an issue with your SMTP server not allowing "relay" from your Outlook domain. ;-)

[skarpik]

@skarpik would you mind installing a clean 9.12.0 and trying the same config. There was some changes regarding email providers in 9.13.0 and want to learn if it may be a new bug in 9.13.0 or not.

Thanks.

Just build a DNN v9.12.0 website and tried an SSL connection to strontium.namespro.ca:465 (hosted on namespro.ca). No luck.

Just in case there is something odd with the namespro.ca SMTP, I tried connecting to mail.stevekarpik.com:465 (SMTP hosting on DNN4Less) and got the same bad result.

Just for interest, I tried connecting via strontium.namespro.ca:2525 (the non-SSL address for the SMTP server at namespro.ca) and it worked perfectly.

It really looks to me to be an SSL issue in present in both DNN v9.12.00 and DNN v9.13.00.

Is it worth testing any earlier versions?

[skarpik]

@skarpik I am pretty sure you may be running into an issue with your SMTP server not allowing "relay" from your Outlook domain. ;-) I don't think that's the problem. The email is being sent from an SMTP server sitting on namespro.ca (a Canadian ASP .NET web host). It doesn't matter whether the email is going to [email protected] or [email protected] (an email address sitting on DNN4Less). The only thing that is different here from my other websites is that I am using an SSL connection to the SMTP server. All of my other websites connect to the SMTP server without SSL.

[jeremy-farrance]

Are you able to test your source server; the server DNN is running on? Last time I ran in to "it worked without SSL" it was because the server my DNN instance was on had not been TLS hardened and was still allowing TLS below 1.2; which almost all modern SMTP receivers will not allow. Anyhow, if you manage the server yourself and can remote on to the desktop, then download and run IIS Crypto. It will show clearly your TLS and related situation.

This is one of many things worth ruling out as your narrow down towards the real issue (if this is not it). In my experience, there are usually at least 2 issues so digging it to the client side and server side SMTP logs and seeing the back and forth word for word is the best way "to get at the truth."

Also, I have to say, if you are running in a non-production situation (dev, staging, testing, etc), then using that SMTP drop without SSL is probably okay and a good (time saving) choice.

[skarpik]

A little background might help set the context...

This issue came up in a "just for fun" website. I use DNN4Less for all of my clients. However, I could see some time in the future a client insisting that the web server had to live in Canada. So I decided to build https://basilthetortoise.com/ using namespro.ca as the web host. Since it was just for fun, I decide to use as many DNN and 2SXC features as possible. One feature that I included was an SSL connection to the SMTP. Most of the websites that I do don't need to send email, so I don't set an SMTP server. And that is how I discovered this problem.

When new DNN or 2SXC versions come out, I usually build a local website on my workstation and use a non-SSL connection to an SMTP server. So for https://basilthetortoise.com/ being experimental and fun, I happened upon the SSL issue purely by accident. In trying to debug the problem, I figured that https://basilthetortoise.com/ might have some features that could confuse the issue, so I built the most simple local DNN instances with NVQuickSite (DNN version 9.12.00 and 9.13.00). Both exhibit the same problem as https://basilthetortoise.com/.

This SSL SMTP problem doesn't really impact me right now in a practical way. But I tend to be obsessive about hunting down bugs, hence, I posted this issue to Github.

Right now I'm flummoxed. Unless there are logs that I don't know about, DNN isn't giving me any hints on what is going on.

[mitchelsellers]

Given that this is a new host, is it possible they have a firewall rule that is preventing communication as well?

You may also want to try port 587 instead per guidance here (https://stackoverflow.com/questions/20228644/smtpexception-unable-to-read-data-from-the-transport-connection-net-io-connect)

[bdukes]

Have you tried switching the mail provider in the web.config to the MailKit provider? The old/default Core provider has a lot of limitations around supported ports and security protocols supported.

[skarpik]

Have you tried switching the mail provider in the web.config to the MailKit provider? The old/default Core provider has a lot of limitations around supported ports and security protocols supported.

No I haven't. Certainly willing to try. Could you please point me to some documentation to tell me what changes need to be made to the web.config. Thank you.

[bdukes]

I'm not sure that we have official document on switching to the MailKit provider. However, it is relatively straightforward. You should find a <mail> section in the web.config that looks something like this:

Update the defaultProvider attribute from CoreMailProvider to MailKitMailProvider (note that updating the web.config will cause your site to restart).

[skarpik]

Thanks @bdukes. I think I am making some progress but no positive news yet.

I'm focusing on port 587 for the SSL. When I use the default mail provider (CoreMailProvider) and test the SMTP connection, the website looks like it is thinking about doing something for several seconds but eventually it's all over and no test message has been sent and there is no error message (or message of any kind) in the Admin Logs.

When I switch over to MailKitMailProvider, I still can't send the test email. However, I do get an error message in the Admin Logs (see below). I guess that is progress.

I am quite sure that there is nothing wrong with the SMTP server because it tests out correctly using an SMTP Test Tool (https://www.gmass.co/smtp-test).

Here is the error message for MailKitMailProvider:

AbsoluteURL:/API/PersonaBar/ServerSettingsSmtpHost/SendTestEmail DefaultDataProvider:DotNetNuke.Data.SqlDataProvider, DotNetNuke ExceptionGUID:77e4bbae-1e5a-434d-8d75-addf95067c74 AssemblyVersion: PortalId:-1 UserId:-1 TabId:-1 RawUrl: Referrer: UserAgent: ExceptionHash:jB53Gnt6yMm0+jKaQVLBGaPYoXY= Message:The SMTP server has unexpectedly disconnected. StackTrace: at MailKit.Net.Smtp.SmtpStream.<ReadAheadAsync>d__39.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MailKit.Net.Smtp.SmtpStream.<ReadResponseAsync>d__46.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MailKit.Net.Smtp.SmtpStream.ReadResponse(CancellationToken cancellationToken) at MailKit.Net.Smtp.SmtpClient.<ConnectAsync>d__93.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at DotNetNuke.Services.Mail.MailKitMailProvider.SendMail(MailInfo mailInfo, SmtpInfo smtpInfo) InnerMessage: InnerStackTrace: Source:MailKit FileName: FileLineNumber:0 FileColumnNumber:0 Method: Server Name: namespro-wh29

[david-poindexter]

@skarpik I am pretty sure you may be running into an issue with your SMTP server not allowing "relay" from your Outlook domain. ;-) I don't think that's the problem. The email is being sent from an SMTP server sitting on namespro.ca (a Canadian ASP .NET web host). It doesn't matter whether the email is going to [email protected] or [email protected] (an email address sitting on DNN4Less). The only thing that is different here from my other websites is that I am using an SSL connection to the SMTP server. All of my other websites connect to the SMTP server without SSL.

@skarpik my point here was that your Host Email is configured as [email protected]. So, that means the FROM address for any SMTP test notifications will be that email. If the SMTP server is not set up to RELAY emails, it can lead to this behavior.

[skarpik]

To test whether there is a problem with using an outlook.com email address, I changed my host email to [email protected] (stevekarpik.com is hosted on DNN4Less). I first tried with port 2525 (that is what namespro.ca uses for non-SSL connections). It worked perfectly.

I then changed to port 587 which @bdukes said is the recommended SSL port (also recommended by support at namespro.ca). The test failed and I got an error message (see below). I know that the user id/password/port combination works because it worked perfectly on https://www.gmass.co/smtp-test.

I'm racking my brain on this one but getting nowhere. All the evidence seems to point to some problem with the DNN SMTP code for servers that require SSL.

I'm willing to try to debug this one but I'd need a few pointers on how to set up a test environment.

---

AbsoluteURL:/API/PersonaBar/ServerSettingsSmtpHost/SendTestEmail DefaultDataProvider:DotNetNuke.Data.SqlDataProvider, DotNetNuke ExceptionGUID:34ed1156-7363-482e-b12f-e249f1d1aeb9 AssemblyVersion: PortalId:-1 UserId:-1 TabId:-1 RawUrl: Referrer: UserAgent: ExceptionHash:jB53Gnt6yMm0+jKaQVLBGaPYoXY= Message:The SMTP server has unexpectedly disconnected.

StackTrace: at MailKit.Net.Smtp.SmtpStream.<ReadAheadAsync>d__39.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MailKit.Net.Smtp.SmtpStream.<ReadResponseAsync>d__46.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at MailKit.Net.Smtp.SmtpStream.ReadResponse(CancellationToken cancellationToken) at MailKit.Net.Smtp.SmtpClient.<ConnectAsync>d__93.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at DotNetNuke.Services.Mail.MailKitMailProvider.SendMail(MailInfo mailInfo, SmtpInfo smtpInfo) InnerMessage: InnerStackTrace: Source:MailKit FileName: FileLineNumber:0 FileColumnNumber:0 Method: Server Name: namespro-wh29

[david-poindexter]

@skarpik perhaps we can try to help this Friday during our open source co-coding session on Discord?

[mitchelsellers]

@skarpik I strongly suspect a TLS issue as noted by @jeremy-farrance

[skarpik]

@jeremy-farrance and @mitchelsellers - I suspect that both of you are pointing in the direction of a solution. I will join the Open Coding session on Discord this Friday that @david-poindexter mentioned above. This problem with SSL SMTP presents an excellent opportunity for me to learn how to debug and solve this sort of problem.

[skarpik]

I believe that I may have a solution. Following a suggestion from @bdukes, I have focused my attention on MailKitMailProvider rather than CoreMailProvider.

MailKitMailProvider works properly if you cert for the SMTP is consistent (i.e. the name of the server matches the name on the cert). However, if you are using shared hosting and a Let's Encrypt cert, this will not be the case and MailKit will throw an exception. For instance, for my website stevekarpik.com that is hosted on DNN4Less, the name of the webserver is mail.stevekarpik.com but the actual SMTP server has a different name.

Proposed Solution:

My solution has two parts:

1. Explicitly set the SSL protocols used by the SMTP client object. This may no be necessary, however, given the advice from @jeremy-farrance, it seemed to be to be prudent.
2. Create a ServerCertificateValidationCallback event handler that lets MailKit ignore the name mismatch on the cert.

The original and revised code are below. I have not been able to test the solution for the Async code.

Notes:

• The inspiration for this solutions comes from notes associated with the MailKit repository: Source: https://github.com/jstedfast/MailKit/blob/master/FAQ.md#ssl-handshake-exception
• As best as I can tell, SslProtocols.Tls13 is not defined in the version of .Net that we are using. Consequently, I use (SslProtocols)12288. However, it may be that we don't need to reference TLS1.3.
• The event handler for ServerCertificateValidationCallback is very primitive. It may be that a more appropriate event handler will look at the exception and process it in a less blunt way.

I have tested this one two servers (DNN4Less.com and namespro.ca) and the SMTP test passes with flying colours. I have not looked at any SMTP servers that use OAuth.

As I mentioned above, the DNN v9.13.0 code likely works perfectly for situations where the SSL certificate for the SMTP server is consistent and unambiguous. However, for situations where there is shared hosting and an Let's Encrypt SSL cert, MailKit is is unhappy. The problem can be fixed, however, by creating an appropriate ServerCertificateValidationCallback handler.

I am a relative GitHub novice, so I don't know the appropriate steps to publish my proposed solution so that others can test it and make sure there are no unwanted side-effects.

Original Code (DNN v9.13.0):

public override string SendMail(MailInfo mailInfo, SmtpInfo smtpInfo = null)
{
	try
	{
		var (host, port, errorMessage) = ParseSmtpServer(ref smtpInfo);
		if (errorMessage != null)
		{
			return errorMessage;
		}

		var mailMessage = CreateMailMessage(mailInfo, smtpInfo);

		using (var smtpClient = new SmtpClient())
		{
			smtpClient.Connect(host, port, SecureSocketOptions.Auto);

			if (smtpInfo.Authentication == "1" && !string.IsNullOrEmpty(smtpInfo.Username) && !string.IsNullOrEmpty(smtpInfo.Password))
			{
				smtpClient.Authenticate(smtpInfo.Username, smtpInfo.Password);
			}

			var (provider, portalId) = this.GetOAuthProvider(smtpClient, smtpInfo);
			if (provider != null)
			{
				provider.Authorize(portalId, new OAuthSmtpClient(smtpClient));
			}

			smtpClient.Send(mailMessage);
			smtpClient.Disconnect(true);
		}

		return string.Empty;
	}
	catch (Exception exc)
	{
		return HandleException(exc);
	}
}

Revised Code:

public override string SendMail(MailInfo mailInfo, SmtpInfo smtpInfo = null)
{
	try
	{
		var (host, port, errorMessage) = ParseSmtpServer(ref smtpInfo);
		if (errorMessage != null)
		{
			return errorMessage;
		}

		var mailMessage = CreateMailMessage(mailInfo, smtpInfo);

		using (var smtpClient = new SmtpClient())
		{
			// *** replacement code (October 15, 2023 - SK) ***
			// Source: https://github.com/jstedfast/MailKit/blob/master/FAQ.md#ssl-handshake-exception
			// Note: (SslProtocols)12288 is the enum value for SslProtocols.Tls13
			// ---------------------------------------------------------------------------------------
			smtpClient.ServerCertificateValidationCallback = (s, c, h, e) => true;
			if (smtpInfo.EnableSSL)
			{
				smtpClient.SslProtocols = SslProtocols.Ssl3 | SslProtocols.Tls | SslProtocols.Tls11 | SslProtocols.Tls12 | (SslProtocols)12288;
			}

			smtpClient.Connect(host, port, smtpInfo.EnableSSL);

			// *** end of replacement code ***
			// ---------------------------------------------------------------------------------------
			if (smtpInfo.Authentication == "1" && !string.IsNullOrEmpty(smtpInfo.Username) && !string.IsNullOrEmpty(smtpInfo.Password))
			{
				smtpClient.Authenticate(smtpInfo.Username, smtpInfo.Password);
			}

			var (provider, portalId) = this.GetOAuthProvider(smtpClient, smtpInfo);
			if (provider != null)
			{
				provider.Authorize(portalId, new OAuthSmtpClient(smtpClient));
			}

			smtpClient.Send(mailMessage);
			smtpClient.Disconnect(true);
		}

		return string.Empty;
	}
	catch (Exception exc)
	{
		return HandleException(exc);
	}
}

[mitchelsellers]

I think that if we were to utilize something like this, at a minimum it would need to be a setting to allow users to bypass the certification validation check, as in many other situations that test is critical and a security feature that should not be ignored.

Maybe even a list of options and set i

• TLS 1.3 + Certificate Validation
• TLS 1.3 w/o Certificate Validation
• TLS 1.2 etc....

[bdukes]

I agree that we'd need to provide an option for whether to ignore the certificate validation (does the core implementation ignore the certificate automatically?).

I would tend to think that we leave the protocols set to the default, unless someone has a specific case where they need to override that.

[skarpik]

@mitchelsellers - I agree that my solution leaves much to be desired from a security perspective. The dilemma for me is that if we stick with the status quo, DNN websites that use shared hosting and have less than optimal SSL certs would not be able to use SSL at all but would have to use an unsecured option. Your solution gives the DNN user an option if they are going to degrade security but it might be hard for people to wrap their heads around.

I am hoping that someone can come up with something better than I propose. However, the fact that my solution comes directly from the MailKIt documentation makes me think that this is not an uncommon situation and the workaround is rather unsatisfactory.

Maybe we need some way of telling the user that the SMTP server has a name mismatch and asking whether they want to continue. I run into this with FileZilla (an FTP program). Often I make connections where the ftp server has a different name than my website URL but it is the URL for the webhost and I proceed because it all looks sensible.

[skarpik]

Here is an example of what FileZilla tells me:

[jeremy-farrance]

I have noticed a number of situations over the years where the user was given the option to connect and use SSL even though some aspect of the certificate's validation was "not up to snuff." One from a couple years ago was a VS Code extension, liximomo's SFTP. He got SSL working on the FTP and of course, quickly found out LOTS of people wanted the SSL more than they wanted to do the work to make the SSL cert valid.

Here is the option in the config file that he added (and made a few dozen very vocal people very happy):

  "port": 21,
  "secure": true,
  "secureOptions": {
    "rejectUnauthorized": false
  },

Reject unauthorized set to false because a) I want the security but b) I realize the SSL name doesn't match (and I don't want or need to fix it because its my server).

Anyhow, sorry this got long, in short, I am just saying this is a valid pattern (and path to happiness). "Not an uncommon situation."

Also note, in this case/example, there is no UI here and its not well documented. You just figure it out from the clues, discover that there is an option to do SSL without the name being valid.

[skarpik]

I think that @jeremy-farrance's observation is true in many cases the most important thing is that the data transfer is encrypted.

I have a client where visitor's to the website can sign up for programs. When they press "submit" on the webform, their information is sent by email to the program coordinator. There is no money involved (i.e. no credit card info) but I'd feel better if the personal information of the web visitors was encrypted (they send their name, address, email address and telephone number). My client hasn't complained that we're using non-secure SMTP but I'd feel better if the information was encrypted.

If my client had the budget to pay for hosting and a cert that meets all the requirements, that's the route I'd follow. But they're a non-profit with a super tight budget. So I'm looking for the best solution for them - probably one that is on the pragmatic side.

[mitchelsellers]

I think the simple solution here might be the middle ground between what we have now, and what I posted earlier which is closer to what @jeremy-farrance has posted.

For example two options

• Use SSL
• Require Trusted Certificate

The second option is only there if you have selected "Use SSL" and we would need to provide good help text. We could also go with "Reject Untrusted Certificates" or similar to flip it to the negative side of things.

Given the risk of what can happen without the verification, it should be noted that it is not recommended to disable this unless it is being done for a trusted/known reason. The primary thing that could be caught by this is a Man In the Middle type attack where downstream the communications are redirected, as well as expired or untrusted SSL certificates.

[bdukes]

Sounds good to me. I was thinking of wording like "Allow Invalid Certificate"

[mitchelsellers]

Allow Invalid Certificate

Yes! That is much cleaner

[skarpik]

@bdukes and @mitchelsellers - this would be a terrific solution. The default would be the fully secure configuration unless the user agreed to accept invalid certificates.