Uploaded PDFs can be downloaded, but not viewed inline

[todda00]

When setting up a collection for PDF files, they upload OK and are downloadable via a link like this:

<a href='{{this.url store='original' download=true}}'>download</a>

When trying to get the PDF to open in the browser with a link like this:

<a target="_blank"  href='{{this.url store='original'}}'>view</a>

The file will never appear unless it is a very small file. The headers seem to be asking for an incomplete range, when looking at the headers there are 2 file requests present - here are the request and response headers for the first:

GET /cfs/files/media/F7zYD73qekE38qPBP/per-jkaxshaft-102013.pdf?store=original HTTP/1.1
Host: localhost:3000
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Cookie: TJS=7ae9b2f1-fc6c-46c0-84bd-a1aa28b1b1ea; _ga=GA1.1.1101389818.1410962009

And the response headers:

HTTP/1.1 200 OK
vary: Accept-Encoding
access-control-allow-origin: http://meteor.local
access-control-allow-methods: PUT
access-control-allow-headers: Content-Type
content-type: application/pdf
content-disposition: inline
content-range: bytes 0-1521851/1521852
c: a
p: u
content-length: 1521852
last-modified: Mon, 22 Dec 2014 15:08:58 GMT
accept-ranges: bytes
date: Mon, 22 Dec 2014 15:40:59 GMT
connection: keep-alive

Request for the second entry:

GET /cfs/files/media/F7zYD73qekE38qPBP/per-jkaxshaft-102013.pdf?store=original HTTP/1.1
Host: localhost:3000
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36
Accept: */*
Referer: http://localhost:3000/cfs/files/media/F7zYD73qekE38qPBP/per-jkaxshaft-102013.pdf?store=original
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Cookie: TJS=7ae9b2f1-fc6c-46c0-84bd-a1aa28b1b1ea; _ga=GA1.1.1101389818.1410962009
Range: bytes=0-32767

And the response:

HTTP/1.1 206 Partial Content
vary: Accept-Encoding
access-control-allow-origin: http://meteor.local
access-control-allow-methods: PUT
access-control-allow-headers: Content-Type
content-type: application/pdf
content-disposition: inline
content-range: bytes 0-32767/1521852
c: a
p: u
content-length: 32768
last-modified: Mon, 22 Dec 2014 15:08:58 GMT
accept-ranges: bytes
date: Mon, 22 Dec 2014 15:40:59 GMT
connection: keep-alive

The status of the second comes back as :

(failed)
net::ERR_CONTENT_LENGTH_MISMATCH

My best guess would be having to do with the initial request being 1 byte short, asking for:

content-range: bytes 0-1521851/1521852

instead of:

content-range: bytes 0-1521852/1521852

But that is a guess and I am not sure exactly what that line should look like in a good implementation.

I have attempted this with S3, FileSystem, and GridFS with the same results.

[raix]

This actually looks correct - a buffer[0-1521851] is 1521852 in length

Question is if the data sent to the client actually matches the length...

Try setting FS.debug = true

[todda00]

Here is the console output, there is no content length info:

I20141222-13:33:33.439(-5)? token: eyJhdXRoVG9rZW4iOiJ3OC1LZEhGR2NBaW1tUmZnZGpVM1RqOHNrQVJtd3dtMEljTXA0MlJIOC1LIn0=
I20141222-13:33:33.441(-5)? GET FILERECORD: F7zYD73qekE38qPBP
I20141222-13:33:33.441(-5)? createReadStream original
I20141222-13:33:33.442(-5)? createReadStreamForFileKey original
I20141222-13:33:33.913(-5)? token: eyJhdXRoVG9rZW4iOiJ3OC1LZEhGR2NBaW1tUmZnZGpVM1RqOHNrQVJtd3dtMEljTXA0MlJIOC1LIn0=
I20141222-13:33:33.919(-5)? GET FILERECORD: F7zYD73qekE38qPBP
I20141222-13:33:33.921(-5)? createReadStream original
I20141222-13:33:33.921(-5)? createReadStreamForFileKey original

I am not sure if it is normal to have 2 requests at almost the same time, this was a result of clicking on 1 link. Here is the associated collection object for the uploaded PDF:

{
   "createdByTransform":true,
   "_id":"F7zYD73qekE38qPBP",
   "copies":{
      "original":{
         "name":"per-jkaxshaft-102013.pdf",
         "type":"application/pdf",
         "size":1521852,
         "key":"media/F7zYD73qekE38qPBP-per-jkaxshaft-102013.pdf",
         "updatedAt":"2014-12-22T15:08:58.993Z",
         "createdAt":"2014-12-22T15:08:58.993Z"
      },
      "thumb":{
         "name":"per-jkaxshaft-102013.png",
         "type":"image/png",
         "size":57965,
         "key":"media/F7zYD73qekE38qPBP-per-jkaxshaft-102013.png",
         "updatedAt":"2014-12-22T15:09:00.259Z",
         "createdAt":"2014-12-22T15:09:00.259Z"
      }
   },
   "meta":{

   },
   "original":{
      "name":"per-jkaxshaft-102013.pdf",
      "updatedAt":"2014-12-22T14:44:07.000Z",
      "size":1521852,
      "type":"application/pdf"
   },
   "uploadedAt":"2014-12-22T15:08:57.725Z",
   "collectionName":"media"
}

[todda00]

For what its worth, the file size of the file on my local system is 1521852 bytes and the file size uploaded at S3 is the same, 1521852.

[raix]

I'm not sure whats triggering the two requests - could you try to copy paste the links in to a separate tab and see if that makes a difference?

[todda00]

same result when the link is copied to a new tab. 2 entries in the console log.

For the download link, there is only 1 request and it works properly.

[raix]

could it be tat the browser fires up the pdf reader and this starts a partial download.

Try in other browsers too

[todda00]

That might be the issue, here are my results:

Chrome - 2 requests, does not load. FF - 2 requests, does not load. Safari - 1 request, loads fine and displays in the browser.

Is there a way to disable partial downloads?

[raix]

We should prop. fix the partial support instead - since it seems as if the partial data is not sent correctly - since this is failing on all SA's it must be somewhere in the access-point package I think.

[aldeed]

I had reworked the partial requests stuff a few months back in https://github.com/CollectionFS/Meteor-cfs-access-point/commit/9e08aa229054b2bd02c016886c390303d04c4f24 and I think that fixed some issues and introduced others. Every browser seems to handle it differently, so we need to figure out the right way to do it that works with every browser.

[raix]

@aldeed I'm turning in now - but I'm not sure all of the SA's actually support the start/end options? (I could be wrong just had a quick look)

[aldeed]

You're probably right. I might be able to take a look tomorrow if you don't get to it first.

[raix]

@aldeed I'm on gridfs - I've refactored the access-point slightly - Currently reading up on standards,

[aadamsx]

@todda00, I'm just starting out with this package, and it seems I'm trying to do the same thing as you... using FileSystem, I'd like to be able to "render" the PDF in the browser. Is there any way you can help me by sharing an example of your collection creation and any schema you have associated with this setup, and maybe a few pointers on how to get start?

Also, did you ever get this issue resolved?

[firdausramlan]

Is there any way to disable 206 partial request? I've tried setting max-ranges 0 in Nginx but to no avail.

[todda00]

@aadamsx No, rendering in browser is still not working at this time, unless the PDFs are very small and don't need to be broken into partial downloads. Schema (Assuming you are referring to the SimpleSchema package??) doesn't play well with CollectionFS, you could probably get it to work if you add every expected field into the schema, but I found it not worth attaching a schema to the file collection, I can handle validation by other means for files.

As far as collection creation, for FileSystem it should be pretty straightforward, something like this:

var storePdf = new FS.Store.FileSystem('pdf');

Media = new FS.Collection("media", {
  stores: [storePdf],
  filter: {
    allow: {
      contentTypes: [
      'application/pdf'
      ]
    }
  }
})

[firdausramlan]

I manage to view pdf inline by adding this configuration in Nginx reverse proxy:

proxy_hide_header Accept-Ranges;
proxy_hide_header Content-Range;

Looking forward partial request 206 handled by CollectionFS.

[raix]

I'm sorry about lagging at the moment - I'll try to find time (we could disable the feature until its fully implemented and tested)

Its basically an issue of implementing the partial option in the storage adapters. FS.FileSystem should be fairly straight forward, gridfs too - and s3 requires a bit more work.

In general we need some kind of qa/test of the storage adapter api - simply to have a way making sure it doesnt accidentally break.

[aadamsx]

Thanks @todda00

To start, I ended up just using a server side Iron Router route and Node.js to build the file, write it to a server side directory, and render it on a route call.

I'll take a look at this package again once some of this stuff you're having issues with gets sorted out I suppose (as I don't want to run into the roadblock you did).

[adamgins]

HI folks, I think I am hitting this problem too?

I am using cfs:s3 0.1.2 Amazon Web Services S3 storage adapter for CollectionFS

Safari displays the PDF. In Chrome it never seems to get there. Cordova in app browser it does not work either.

I am using HAPROXY (not NGINX).. anyone know how to workarounds? @raix are the changes above to fix this?

As a workaround, I tried download=true, mentioned above, but that just seemed to break my call to CFS... ie got nothing (blank) back from the {{url store="attachments" download=true}}

Not sure how this would work in a Cordova app?

[adamgins]

@raix @aldeed hi ya, just wondering what the status was on this, pls?

[aldeed]

I'm not sure, but there has been work done to improve byte range support. I think some of the SAs are still missing proper support for it, though.

[raix]

Filesystem supports partial: https://github.com/CollectionFS/Meteor-CollectionFS/blob/master/packages/filesystem/filesystem.server.js#L59

GridFS supports partial: https://github.com/CollectionFS/Meteor-CollectionFS/blob/master/packages/gridfs/gridfs.server.js#L80-L82

Question is if the Amazon Official API supports this too: https://github.com/CollectionFS/Meteor-CollectionFS/blob/master/packages/s3/s3.server.js#L126-L129 We need to add the RANGE option might be in the format of a string: bytes 21010-47021/47022 unit start-end/total http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getObject-property

The SA createStream is called at: https://github.com/CollectionFS/Meteor-CollectionFS/blob/master/packages/access-point/access-point-handlers.js#L193 setting options {start: range.start, end: range.end}

[zevsuld]

I have same issue, updated latest version of cfs. If pdf file is a little bit big size, it is not loading just display blank page. I am using cfs:filesystem, if I change to cfs:gridfs is it possible to display pdf file properly.

[zevsuld]

@firdausramlan thank you for sharing your workaround. Redeployed my project with meteor up with nginx and used reverse proxy settings as you did. It works fine for all browser. Look forward to CollectionFS support partial download.

[zaklampert]

+1. Would be great to be able to open pdfs inline.

@zevsuld or @firdausramlan can you elaborate on setting up a mup nginx reverse proxy?

[veered]

I think that this may be related to a Chrome bug: https://code.google.com/p/chromium/issues/detail?id=442318

[zevsuld]

is this issue not fixed yet? I am currently using workaround, but really need to upload and display inline pdf file which over size of 1mb.

[Sojourneer]

I have a similar problem, but for small files. I am deploying to meteor.com, and decided for the dropbox store for fallback access to the files. (Files uploaded to the fs store disappeared on re-deploy).

With dropbox, download via the url() fails for small text files. (My) manual access via dropbox is fine.

[Sojourneer]

I tested on a 9 byte text file and traced it through the cfs:dropbox and dropboxjs libs, then lost it somewhere in compress.js (part of npm connect module in meteor webapp package). The version of connect is two years old,so I decided to try to upgrade at least the latest compress.js file (connect 2.10.0 release); I also had to install npm negotiator (latest version).

Small files are rendered successfully now, but unfortunately PDFs still aren't.

[jonarnaldo]

I'm having the exact same problem with S3, but for videos and images. Any further help much appreciated.