Forwarding Attachment from Store Timeout

[nicemaker]

Hi,

while I was writing this issue I figured out the problem. And I am not sure if you can actually help here. But maybe you have an idea how to deal with the problem

Our App supports User-to-User emails via masked email addresses. We store the emails temporarly via a store( notify=MYENDPOINT) route to pick them up later and forward them to the real email address.

All that works excellent, except with attachments.

The problem seems to be that mailgun returns the wrong attachment file size and mailgun.Attachment() needs the knownLength attribute.

I get this from mailgun store:

{ url: 'https://sw.api.mailgun.net/v3/domains/sandbox37e724a7518b47c2bd7a99e585558490.mailgun.org/messages/eyJwIjpmYWxzZSwiayI6IjRmOTRiMTc2LTA0OTktNDYzMy04MDE1LTcyYjMyZjE3MTcyZSIsInMiOiJlN2M1YmZhMjEyIiwiYyI6InRhbmtiIn0=/attachments/0',
  'content-type': 'application/pdf',
  name: 'MyFile.pdf',
  size: 204585 }

I modify then the url with apiKey and username:

const url = "https://api:[email protected]/v3/domains/sandbox37e724a7518b47c2bd7a99e585558490.mailgun.org/messages/eyJwIjpmYWxzZSwiayI6ImM5ZTdhZGNjLTFkYWYtNGU1OC1hYjNhLTg0ZTg2MTRlOWRkOCIsInMiOiJkNzJlZDdhNmRlIiwiYyI6InRhbmtiIn0=/attachments/0";

and create new Attachment:

const attachment =  new mg.Attachment(
        { data: request( url )
        , contentType: 'application/pdf'
        , filename: 'MyFile.pdf'
        , knownLength: 204585
        })

That doesn't work. If i check the file size locally it seems to be 149360. If I use the above with knownLength=149360 it works.

It could have been sooo easy. Any idea for a smart workaround?

Cheers

Bjorn

[nicemaker]

Hmm, looks like there is now way around to actually get the correct file size first, Mailgun replied:

Attachment Size in routed or posted information. In this context attachment size is given not in bytes but in symbols. When attachment is sent via email it's usually encoded with base64 (which is often larger than how the data was originally submitted). The size specifies the length of the encoded attachment, including attachment headers. Once you download an attachment you should be able to get its expected size

Is the knownLength really required from mailgun-js point of view?

[bojand]

Hello, sorry it took long to look into this. As the attachment documentation states the knownLength parameter is only required in case of a Stream attachment, which is what you're doing here. This is required parameter, and it is expected to be correct, so I can see how this would not be working in case the passed value is not the true size. The only workaround I can see is that you download the attachment first from the url you have and then you create the attachment using the received data passing it in either as a string or Buffer. Hope this helps.

[nicemaker]

Thanks for looking into this. I also looked briefly into your sourcecode and understood that there is now way to avoid the knownLength.

I posted a feature request on mailgun for an API call that allows user to user email forwarding withoout downloading the attachments. It will probably get lost, but if you like the idea give it an upvote please: https://mailgun.uservoice.com/forums/156243-general/suggestions/31709494-easy-forward-user-to-user-emails-with-masked-email

[berndhartzer]

I am running into the exact same problem that you described in the original post @nicemaker. Out of interest, how did you end up solving this? Did you just download the attachment and pass it as a string or buffer as @bojand suggested?

[nicemaker]

@berndhartzer Finally we skipped attachments for the first release of the App. We will add it to a future release and in the moment I don't see any other way than downloading it. Please support my suggestion above, maybe on one point they come up with a better implementation of simple u2u forwards.

To explain a little better what the problem is,Mailguns support answer was:

Attachment Size in routed or posted information. In this context attachment size is given not in bytes but in symbols. When attachment is sent via email it's usually encoded with base64 (which is often larger than how the data was originally submitted). The size specifies the length of the encoded attachment, including attachment headers. Once you download an attachment you should be able to get it expected size

[berndhartzer]

@nicemaker Thanks for the info, I guess downloading the image will have to be my approach for now. I added a vote to your suggestion on the Mailgun forums too.

[dgwight]

I ran into this issue too and ended up downloading the attachments. I'm not sure how it will scale, but it seems to be working just fine for now. I believe this issue will be different if you have the routes forward instead of store and notify

function downloadAttachment (attachment) {
  return axios.get(attachment.url, {
    responseType: 'arraybuffer',
    auth: {
      username: 'api',
      password: process.env.MAILGUN_API_KEY
    }
  }).then(res => {
    return new mailgun.Attachment({
      data: res.data,
      filename: attachment.name,
      contentType: res.headers['content-type'],
      knownLength: res.headers['content-length']
    })
  })
}

function relay (from, to, email) {
  const attachmentArray = email.attachments ? JSON.parse(email.attachments) : []
  return Promise.all(attachmentArray.map(attachment => {
    return downloadAttachment(attachment)
  })).then(attachments => {
    return mailgun.messages().send({
      to: to,
      from: from,
      subject: email.Subject,
      text: email['body-plain'] || ' ',
      attachment: attachments
    })
  })
}