bc-java

bc-java copied to clipboard

Hi,

I've tried my luck on the mail list, assuming the problem to be on my side but after downloading the bc-sources and using the debugger I think I've found an actual bug in BouncyCastle.

I've created a test case - feel free to add it to the sources if useful - that shows the problem SMIMESigning_Test.zip

I create a MimeBodyPart containing a mail in different combinations, then I sign it (first encapsulated, then as multipart). After that I immediately verify the signature against the certificate. Everything works except when I create a mail that is organized the following way:

• A multipart/mixed part containing
  • A multipart/alternative part containing
    • A text/plain body part
    • A text/html body part
• A text/plain file attachment with disposition "attachment"

Essentially a mail you regularily see nowerdays containing HTML and an alternate plain text part plus an attachment.

When signing encapsulated the signature verification check succeeds here but using multipart, the check fails with the exception

org.bouncycastle.cms.CMSSignerDigestMismatchException: message-digest attribute value does not match calculated value at org.bouncycastle.cms.SignerInformation.doVerify(SignerInformation.java:529) at org.bouncycastle.cms.SignerInformation.verify(SignerInformation.java:613) at __Run_SMIMESigning.performSignatureCreationAndCheck(__Run_SMIMESigning.java:117) at __Run_SMIMESigning.testMailSigning(__Run_SMIMESigning.java:75)

I added some debug-infos in my testcase and you can see that the calculated digest is the same for encapsulated and multipart:

signed digest: -602279b7bb1596241f5af136cb025a355ec08b1d895d590ff732aae42fb216cb [...] signed digest: -602279b7bb1596241f5af136cb025a355ec08b1d895d590ff732aae42fb216cb

But the verification results into different ones: digest: -602279b7bb1596241f5af136cb025a355ec08b1d895d590ff732aae42fb216cb [...] digest: -e41b571e93a088f1616a4b85b786decd80f703ea3c45d5d8041f501e1c77036

So I downloaded the sources and extended SignerInformation.doVerify the following way:

OutputStream      mydigOut = calc.getOutputStream();
System.out.println("start digesting");
OutputStream digOut = new OutputStream() {
    @Override
        public void write(int b) throws IOException {
            System.out.print((char) (b & 0xff));
//           System.out.print(Integer.toHexString(b & 0xff) + " ");
            mydigOut.write(b);
        }
    };

so the data being digested is printed on STDOUT. This shows a difference in the digested data (stripped down to the relevant part):

Encapsulated:

start digesting
Content-Type: multipart/mixed; 
    boundary="----=_Part_22_1902260856.1560873376436"
[...]
<h1>Some HTML</h1><p>Some text with non-ascii: =E2=82=AC</p>
------=_Part_21_1151593579.1560873376436--

------=_Part_22_1902260856.1560873376436
Content-Type: text/plain; charset=utf8; name=filename.txt
[...]
------=_Part_22_1902260856.1560873376436--

Finished digesting

Multipart signing:

start digesting
Content-Type: multipart/mixed; 
    boundary="----=_Part_22_1902260856.1560873376436"
[...]    
<h1>Some HTML</h1><p>Some text with non-ascii: =E2=82=AC</p>
------=_Part_21_1151593579.1560873376436--
------=_Part_22_1902260856.1560873376436
Content-Type: text/plain; charset=utf8; name=filename.txt
[...]
------=_Part_22_1902260856.1560873376436--

Finished digesting

For some reason an empty line between the end of the multipart/alternative and the multipart/mixed part got lost.

On the producing side I can work around that problem by using encapsulated signing or do compression before signing but I assume that receiving a multipart/signed mail might run into the same problem where such workaround is not available (that would be another testcase but I'll wait for that till I got some feedback here).

Have I found a bug or is it me doing something fundamentally wrong when using BouncyCastle to sign mails?

Which version of JavaMail is this using? I can reproduce the problem but not in the way shown above.

From the manifest.MF file:

Bundle-Version: 1.5.0

Haven't tried it with 1.6, yet because this version currently breaks things here in other (non-mail-related) parts.

Updated to BC 1.62 and JavaMail 1.6.2 has no effect on the outcome of the testcase.

I can reproduce the problem but not in the way shown above.

By that, do you mean that you can reproduce the problem using a different test or that you used my testcase and it failed but differently? If the latter, what's the difference?

I've got some progres here, the effect is created by SMIMEUtils.outputPostamble:

InputStream in;
try
{
    in = ((MimeBodyPart)parent).getRawInputStream();
}
catch (MessagingException e)
{
    return;   // no underlying content rely on default generation
}
[...]

If the given MimeBodyPart throws a MessagingException because of no contained content, the method simply returns. With a content - and even an empty one as far as I understood - you have the output of at least a line break. When changing above block to

InputStream in;
try
{
    in = ((MimeBodyPart)parent).getRawInputStream();
}
catch (MessagingException e)
{
    lOut.writeln();
    return;   // no underlying content rely on default generation
}
[...]

my test cases run through.

Hm, I'd like to give this issue a bump. Is there anything I can provide or help with in order to get progress here?

1,5 years after the initial post, I'd like to give this another bump. Is there anything I can provide or help with in order to get progress here?

I'd be curious to see if this also happens with our generic SMIME library (see org.bouncycastle.mime in the PKIX jar). We've had a string of problems over the years with the way JavaMail loads and re-encodes it's data.

The other thing that might be worth trying is the bcjmail and jakarta mail instead.

I'm not sure how to blame JavaMail here. The empty line between two MIME-boundaries is "normal" and even if it weren't, it shouldn't be ignored while calculating the digest for a signature check.

I also faced the behavior using jakarta.mail 1.6.5 and bcmail-jdk18on 1.72 when I tried to verify a self-created message instantly. Verification of the sent message with Outlook on the other hand was successful.

I've got some progres here, the effect is created by SMIMEUtils.outputPostamble:

InputStream in;
try
{
    in = ((MimeBodyPart)parent).getRawInputStream();
}
catch (MessagingException e)
{
    return;   // no underlying content rely on default generation
}
[...]

If the given MimeBodyPart throws a MessagingException because of no contained content, the method simply returns. With a content - and even an empty one as far as I understood - you have the output of at least a line break. When changing above block to

InputStream in;
try
{
    in = ((MimeBodyPart)parent).getRawInputStream();
}
catch (MessagingException e)
{
    lOut.writeln();
    return;   // no underlying content rely on default generation
}
[...]

my test cases run through.

When creating parts and composing messages no input streams are set up, therefore the MessagingException is thrown and the Bouncy Castle implementation fails. I assume it is not meant for this use case as all examples from bcmail obtain a full message from some input stream. A quick fix that worked for me was to call the copy constructor on the self-created message (which sets up input streams internally) and verify the messages copy instead.

I assume that receiving a multipart/signed mail might run into the same problem

If you also implement the receiving side with Jakarta Mail and Bouncy Castle, I allege no, because both IMAPMessage and POP3Message have reliable getContentStream implementations when the message retrieval is successful. As said earlier, verification with Outlook was successful too.