saveEventually / EventuallyQueue doesn't work when killing server -> saveEventually object -> restarting server

[mortenmo]

New Issue Checklist

• [X] I am not disclosing a vulnerability.
• [x] I am not just asking a question.
• [x] I have searched through existing issues.
• [x] I can reproduce the issue with the latest versions of Parse Server and the Parse JS SDK.

Issue Description

Loading an object in client, then calling saveEventually on it after killing the parseserver, then starting it up again doesn't work for me. It seems to be a few different issues. Maybe I'm just doing something really wrong, and if so I'm sorry in advance :)

Test code is really simple. I find object "Foo" (with server up) in the console, kill server, client executes this code:

result[0].set("counter", result[0].get("counter")+1); await result[0].saveEventually(); 

I can see the Parse.EventuallyQueue.getQueue() have the following

'[{"queueId":"save_Foo_T6r7uWTcU9_","action":"save","object":{"counter":13,"createdAt":"2023-09-22T16:00:06.541Z","updatedAt":"2023-09-22T18:48:30.396Z","ACL":{"G2lwffEEaD":{"read":true,"write":true}},"objectId":"T6r7uWTcU9"},"serverOptions":{},"id":"T6r7uWTcU9","className":"Foo","createdAt":"2023-09-22T18:48:44.083Z"}]

When I restart the server, I don't see any attempts to save to server but queue is empty. It seems to fail this test in EventuallyQueue.sendQueueCallback under save:

// Queued update was overwritten by other request. Do not save
      if (
        typeof object.updatedAt !== 'undefined' &&
          object.updatedAt > new Date(queueObject.object.createdAt)
      )

The code is checking is the object's updatedAt is greater than the object's createdAt, which will always be the case. My guess was that this should be object.updatedAt > queueObject.createdAt (when the queue object was created not the original object). hacking on the JS code did get me past this problem but it still wouldn't save.

Next issue is that the EventuallyQueue stores a .toJSON of my object, which includes among other things the ACL object.

With some extra logging, the ParseObject.save gets sent that ACL object. It fails on ParseObject.validate(attrs) (silently) with this:

Error: ACL must be a Parse ACL.
    at ParseObjectSubclass.value (ParseObject.js:1259:16)
    at ParseObjectSubclass.value (ParseObject.js:1578:31)
    at EventuallyQueue.js:386:27

I'm not sure here if the best fix is to delete the ACL out of the json stored in EventuallyQueue or if ParseObject.validate should allow for JSON versions of the ACL, but it does fail my very simple saveEventually regardless.

I don't normally modify ACLs on client, only server using save hooks, but taking them out would be one thing.

Another fix (although still would fail with ACLs changed on client) is to not do (in EventuallyQueue.enqueue):

 queueData[index] = {
      queueId,
      action,
      object: object.toJSON(), // <-- This stores ACL in json format..
      serverOptions,
      id: object.id,
      className: object.className,
      hash: object.get('hash'),
      createdAt: new Date(),
    };

Instead of object being the entire toJSON() storing only the values of the changed keys (_getSaveJSON() ?). Or maybe change the ACL object if present back into a valid version.

Steps to reproduce

Actual Outcome

Data is lost

Expected Outcome

Expected data to be saved when server starts up again.

Environment

Server

• Parse Server version: 5.2.3
• Operating system: OSX
• Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): Local

Database

• System (MongoDB or Postgres): MongoDB
• Database version: 5.5
• Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): local

Client

• Parse JS SDK version: 4.1.0

Logs

[parse-github-assistant[bot]]

Thanks for opening this issue!

• 🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

[mortenmo]

I did make this hacky fix locally to fix the problem and it does, but not sure if its the best fix:

fixParse.ts

type QueueObject = {
    queueId: string;
    action: string;
    object: Parse.Object;
    serverOptions: any;
    id: string;
    className: string;
    hash: string;
    createdAt: Date;
};

console.info('Fixing Parse...');

// @ts-ignore
Parse.EventuallyQueue.enqueue = async function (
    action: string,
    object: Parse.Object,
    serverOptions: any,
): Promise<void> {
    console.log(`Enqueueing ${action} for ${object.id}`);
    const queueData = await this.getQueue();
    // @ts-ignore
    const queueId = this.generateQueueId(action, object);

    // @ts-ignore
    let index = this.queueItemExists(queueData, queueId);
    if (index > -1) {
        // Add cached values to new object if they don't exist
        for (const prop in queueData[index].object) {
            if (typeof object.get(prop) === 'undefined') {
                object.set(prop, queueData[index].object[prop]);
            }
        }
    } else {
        index = queueData.length;
    }
    queueData[index] = {
        queueId,
        action,
        // @ts-ignore
        object: object._getSaveJSON(),
        serverOptions,
        id: object.id,
        className: object.className,
        hash: object.get('hash'),
        createdAt: new Date(),
    };
    // @ts-ignore
    return this.setQueue(queueData);
};

// @ts-ignore
Parse.EventuallyQueue.sendQueueCallback = async function (
    object: Parse.Object,
    queueObject: QueueObject,
): Promise<void> {
    if (!object) {
        // @ts-ignore
        return this.remove(queueObject.queueId);
    }
    switch (queueObject.action) {
        case 'save':
            // Queued update was overwritten by other request. Do not save
            if (
                typeof object.updatedAt !== 'undefined' &&
                object.updatedAt > queueObject.createdAt
            ) {
                // @ts-ignore
                return this.remove(queueObject.queueId);
            }
            try {
                await object.save(
                    queueObject.object,
                    queueObject.serverOptions,
                );
                // @ts-ignore
                await this.remove(queueObject.queueId);
            } catch (e) {
                if (
                    e.message !==
                    'XMLHttpRequest failed: "Unable to connect to the Parse API"'
                ) {
                    // @ts-ignore
                    await this.remove(queueObject.queueId);
                }
            }
            break;
        case 'destroy':
            try {
                await object.destroy(queueObject.serverOptions);
                // @ts-ignore
                await this.remove(queueObject.queueId);
            } catch (e) {
                if (
                    e.message !==
                    'XMLHttpRequest failed: "Unable to connect to the Parse API"'
                ) {
                    // @ts-ignore
                    await this.remove(queueObject.queueId);
                }
            }
            break;
    }
};

export {};

[dplewis]

@mortenmo Do you want to open a pull request?

[dplewis]

Fixed via https://github.com/parse-community/Parse-SDK-JS/pull/2097