GridFS: 5593bfe857f397093508f1c5 does not exist

[neopostmodern]

I've hacked together some code that uploads GIFs, decomposes them into frames and eventually inserts all of them into GridFS. It works.

But there is one specific GIF for which it fails:

(STDERR) 
(STDERR) /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/base.js:246
(STDERR)         throw message;      
(STDERR)               ^
(STDERR) Error: 5593bfe857f397093508f1c5 does not exist
(STDERR)     at /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/gridfs/gridstore.js:200:22
(STDERR)     at /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/collection/query.js:164:5
(STDERR)     at /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:778:35
(STDERR)     at Cursor.close (/home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:1009:5)
(STDERR)     at Cursor.nextObject (/home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:778:17)
(STDERR)     at commandHandler (/home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:741:14)
(STDERR)     at /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/db.js:1903:9
(STDERR)     at Server.Base._callHandler (/home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
(STDERR)     at /home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
(STDERR)     at [object Object].MongoReply.parseBody (/home/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1dgxmmd++os.linux.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
=> Exited with code: 8

Please notice that the 5593bfe857f397093508f1c5 is the exact same on my development machine and the meteor test server, referenced above.

Upload code:

Gifs.insert file, (error, fileObj) ->
# Inserted new doc with ID fileObj._id, and kicked off the data upload using HTTP

    if error?
        Materialize.toast error, 4000, "red"
        state.set 'status', 'default'
        return

    uploadTracker = Tracker.autorun ->
        reactiveFileObj = Gifs.findOne fileObj._id

        if reactiveFileObj.isUploaded()
            uploadTracker.stop()

            Meteor.call 'multichannel.initialize-project', { fileId: fileObj._id }, (error, result) ->
                if error?
                    Materialize.toast error, 4000, "red"
                    state.set 'status', 'default'
                    return

                Materialize.toast "Successfully created project", 2000
                Router.go 'home'

'multichannel.initialize-project': (options) ->
    if not @userId?
        throw new Meteor.Error 401, "Please log in to create a multichannel"

    gif = Gifs.findOne options.fileId

    @unblock()

    Projects.insert(
        name: gif.name()
        originalGif: gif._id
        frames: []
        userId: @userId
        likes: 0
    , (error, projectId) =>
        if error?
            console.dir error
            return

        Meteor.users.update _id: @userId,
            $push:
                projects: projectId

        frameIndex = 0

        GifExplode.Do gif.createReadStream('gifs'), (frame) ->
            _frameIndex = frameIndex
            frameIndex += 1
            fsFrame = new FS.File()
            fsFrame.attachData(frame, { type: 'image/gif' })
            fsFrame.name(Meteor.uuid() + ".gif")
            Gifs.insert(
                    fsFrame
            , (error, frameObject) ->
                if error?
                    console.dir error
                    return

                Projects.update(_id: projectId,
                    $push:
                        frames:
                            $each: [
                                gif: frameObject._id
                                index: _frameIndex
                            ]
                            $sort: index: 1
                            $slice: -50 
                , (error) ->
                    if error?
                        console.dir error
                        return
                )
            )
    )

The frame in the callback is a stream.

Or see the entire source code.

[jimmiebtlr]

I believe I'm experiencing the same issue, but with a different string.

Error: 55945c2b6389174f15989a5b does not exist

Which occurs fairly rarely on upload of an image, and crashes the meteor server.

[neopostmodern]

This seems to be a duplicate of https://github.com/CollectionFS/Meteor-CollectionFS/issues/408

[fishdude]

Getting the same thing here.

 Error: 55efccefe3653fa204dd0996 does not exist

[adamgins]

Hi, looks I have something similar:

I just started running my app on Galaxy and I am seeing the error below when uploading an image.

I am using Collections FS with S3

2015-10-08 06:22:16+11:00about to mark all notifications read for:PpEu7cHpEShno2zEC
ab79
2015-10-08 06:22:46+11:00 at [object Object].MongoReply.parseBody (/app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
ab79
2015-10-08 06:22:46+11:00 at /app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
ab79
2015-10-08 06:22:46+11:00 at Server.Base._callHandler (/app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
ab79
2015-10-08 06:22:46+11:00 at /app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/db.js:1903:9
ab79
2015-10-08 06:22:46+11:00 at commandHandler (/app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:741:14)
ab79
2015-10-08 06:22:46+11:00 at Cursor.nextObject (/app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:778:17)
ab79
2015-10-08 06:22:46+11:00 at Cursor.close (/app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:1009:5)
ab79
2015-10-08 06:22:46+11:00 at /app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:778:35
ab79
2015-10-08 06:22:46+11:00 at /app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/collection/query.js:164:5
ab79
2015-10-08 06:22:46+11:00 at /app/bundle/programs/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/gridfs/gridstore.js:200:22
ab79
2015-10-08 06:22:46+11:00Error: 56157104bd99051c008fc328 does not exist
ab79
2015-10-08 06:22:46+11:00Application exited with code: 7 signal: null
ab79
2015-10-08 06:22:46+11:00Application process closed with code: 7 signal: null

My app currently runs OK on on AWS Opsworks/Docker... so this is just happening on Galaxy.

[vioan]

I've got a similar error for my app:

W20151026-18:11:16.942(1)? (STDERR)
W20151026-18:11:16.946(1)? (STDERR) /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/base.js:246
W20151026-18:11:16.946(1)? (STDERR)         throw message;
W20151026-18:11:16.946(1)? (STDERR)               ^
W20151026-18:11:16.950(1)? (STDERR) Error: 562e5eb4411fe142a6c59e47 does not exist
W20151026-18:11:16.951(1)? (STDERR)     at /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/gridfs/gridstore.js:200:22
W20151026-18:11:16.951(1)? (STDERR)     at /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/collection/query.js:164:5
W20151026-18:11:16.951(1)? (STDERR)     at /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:778:35
W20151026-18:11:16.952(1)? (STDERR)     at Cursor.close (/Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:1009:5)
W20151026-18:11:16.954(1)? (STDERR)     at Cursor.nextObject (/Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:778:17)
W20151026-18:11:16.955(1)? (STDERR)     at commandHandler (/Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/cursor.js:741:14)
W20151026-18:11:16.955(1)? (STDERR)     at /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/db.js:1903:9
W20151026-18:11:16.957(1)? (STDERR)     at Server.Base._callHandler (/Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
W20151026-18:11:16.958(1)? (STDERR)     at /Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
W20151026-18:11:16.958(1)? (STDERR)     at [object Object].MongoReply.parseBody (/Users/[...]/.meteor/packages/cfs_gridfs/.0.0.33.1juhiel++os.osx.x86_64+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)

[dnish]

Having the same error, whole server crashes:

[server-crash] 56324147e575535820228f38 does not exist

[griv]

Having the same error here (local Mac dev): Error: 564e67463289527a61dc4262 does not exist

[tagrudev]

Same thing in here

App 29314 stderr: Error: 564ecf6ab3f9228272e2f31a does not exist
App 29314 stderr:     at //[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/gridfs/gridstore.js:200:22
App 29314 stderr:     at //[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/collection/query.js:164:5
App 29314 stderr:     at //[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:778:35
App 29314 stderr:     at Cursor.close (//[...]//npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:1009:5)
App 29314 stderr:     at Cursor.nextObject (//[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:778:17)
App 29314 stderr:     at commandHandler (//[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/cursor.js:741:14)
App 29314 stderr:     at //[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/db.js:1903:9
App 29314 stderr:     at Server.Base._callHandler (//[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/base.js:453:41)
App 29314 stderr:     at //[...]//server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/connection/server.js:488:18
App 29314 stderr:     at [object Object].MongoReply.parseBody (/[...]/server/npm/cfs_gridfs/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)

It restarts the server, which is a big pain, I've searched all of the collections and couldn't find this id.

I wonder where is it coming from ?

[tagrudev]

That's kind of a big problem since it restarts the server anyone ?

[dnish]

@tagrudev Clear all temp collections of CollectionFS

[tagrudev]

@dnish you mean all temp mongo collections ? If so I've already tried that, it's not working

[dnish]

Yeah all ones that are related to CollectionFS like chunks or the temp ones.

[tagrudev]

I've already tried cleaning the collectons, but that doesn't solve the issue (as I said I've tried finding the id that the collectionFS is trying to access and it's nowhere to be found), which is kind of strange - I think the problem comes from CollectionFS trying to access unfinished / failed upload file that's been chunked

[vigile]

Have the same problem!

[louis49]

This fix works : https://github.com/welelay/Meteor-CollectionFS/commit/86939dce8bbc986e0941b385d2cd255faf95c321

[jackkav]

@louis49 Your suggested fix and your own fix do not appear to be related, care to explain?

[louis49]

Neither, after using some time I realize that this does not work 100% of the time in very specific conditions: Multinode "Sticky-session" not activated

After applying other unpublished corrections there are some bug (but not crash)

We base our tests on a bulk of 1000 files < 6 MB

[jackkav]

Thanks for the speedy reply, it would appear the temp store doesn't handle multiple application servers accessing it particularly well. In our implementation we are using CollectionFS to store things on the local server filesystem and manage the access. We have been hitting this heavily with tens of thousands of files 10-100MB. The problem doesn't seem to occur when the app and db are on the same server and we only run one app server. I haven't tested it yet but I'm hoping the welelay fix could prevent additional application server from causing the tempstore to crash collectionFS. I also noticed using the FS store still copied each file into the temp store on the db, 13GB in our case. This is problematic when the db is not on the same physical machine, but if the tempstore could be switched off, with welelay's fix, perhaps this won't be an issue.

[louis49]

OK, i found it! I found a working solution, the last bug were finally on a other package of our project. I check all and put the final correction tomorrow.

[ChrisCinelli]

@jackkav If "The problem doesn't seem to occur when the app and db are on the same server and we only run one app server." sounds like a race condition.

[hwillson]

Hi all - Just a quick FYI; I just had this problem as well and clearing the cfs._tempstore.chunks collection fixed it for me:

console> db['cfs._tempstore.chunks'].remove({})

[ScyDev]

In our case it was actually the upload and storing of huge images that caused this problem. An image of 2.5M and roughly 4000x3000px was too much.

[dnish]

@hwillson Yeah, we did this too, but the error occured just a few days later again.

[tobitech]

This was working fine in my Meteor app until I adopted the new project structure in the Meteor guide and I started getting this error - 59c7a6c4ed1108551a335157 does not exist. Please does any one have a solution to this? could it be because my collection definition is inside imports/api folder?