NTLM: Order of signature and sealed message in gss_wrap/gss_unwrap

[filipnavara]

Describe the bug

The gss-ntlmssp NTLM GSSAPI implementation and Heimdal disagree on the message order in the gss_wrap and gss_unwrap calls. gss-ntlmssp encodes the messages in (SIGNATURE | MESSAGE) order while Heimdal uses the (MESSAGE | SIGNATURE) order. This makes the implementations mutually incompatible.

I've originally found the issue in the context of .NET implementation used for SMTP GSSAPI authentication. To summarize my findings, the behavior is underspecified and there are two possible interpretations:

• The Windows API uses SecPkgContext_Sizes.cbSecurityTrailer name to describe the signature size which seems to imply its a trailer. EncryptMessage API simply calls the buffer type SECBUFFER_TOKEN which is non-descriptive.

• The actual on-the-wire data for GSSAPI SASL authentication scheme when connecting to Microsoft Exchange SMTP server uses the (NTLM signature | message) ordering. The GSSAPI SASL authentication scheme is described by the RFC 2222 specification and later updated as RFC 4752. The relevant part is in RFC 4752, section 3.1. at the end where it describes the last message exchange to be directly manipulated through gss_unwrap and gss_wrap calls. The implication of that is that for the NTLM provider the order in both gss_wrap and gss_unwrap should be with the signature at the beginning of the wrapped message, not at the end.

I am leaning towards the later implementation. It would be consistent with the RFC implementation, on-the-wire ordering and the order that gss-ntlmssp uses.

[lhoward]

I'm not sure if RFC 4752 really has any implications for how other mechanisms construct their tokens, GSS is (mostly) opaque to the SASL layer. And you can't assume much from the way the Kerberos SASL mechanism is implemented as, although it has the name GSSAPI it specifically refers to the Kerberos GSS mechanism.

Anyway, you're right and it's surprising this has gone unnoticed for so long. Windows allows the caller to specify the trailer buffer explicitly so, there is an API impedance mismatch. And indeed when used with DCE RPC, the trailer should go at the end of the sealed message.

I would prefer to fix this by implementing _gss_ntlm_wrap_iov() and _gss_ntlm_unwrap_iov(). The token should always be placed in GSS_IOV_BUFFER_TYPE_HEADER (confusingly, this is where DCE RPC expects to find its security trailer). _gss_ntlm_wrap() and _gss_ntlm_unwrap() can be implemented as wrappers on GSS_IOV_BUFFER_TYPE_HEADER || GSS_IOV_BUFFER_TYPE_DATA (should be legal to omit padding and trailer buffers as there are none). This will also allow support for AEAD (GSS_IOV_BUFFER_TYPE_SIGN_ONLY).

[filipnavara]

RFC 2222 originally specified the GSSAPI mechanism without the specific Kerberos reference, that was only added later in the updated specification (RFC 4752). I agree it's a bit of a stretch to imply the internal mechanics of specific GSSAPI mechanism from SASL specification and on-the-wire content but due to lack of other documentation I could not find better description.

There's definitely an API impedance mismatch because both SSPI and the Gss_WrapEx in NTLM specification operate much more like the gss_wrap_iov API (and the gss_unwrap_iov counterpart).

In fact I even tried to change the code in .NET runtime's GSSAPI interface to use gss_wrap_iov and gss_unwrap_iov but unfortunately not all implementations provide it. Heimdal and gss-ntlmssp were missing it, Apple fork has it but marks it as private API (but uses it in the DCE RPC implementation). I didn't have any working implementation to test against, so I abandoned the attempt for the moment.

Apple implementation of gss_wrap also suffers from the same issue (not a coincidence, it's a Heimdal fork) but in addition it has a bug in _gss_ntlm_wrap that incorrectly sets the buffers and causes memory corruption. That leads me to believe that almost no one actually tried to use this API with NTLM.

[lhoward]

The Apple code appears to wrap the non-IOV entry points around their IOV equivalents, and I think this is the right approach. Perhaps we should merge the Apple code. A third party would need to contribute this as a pull request though, as none of the Heimdal core team have the necessary time to work on this. (Hence the comment from @jaltman in a separate PR about removing NTLM completely.)