Parallel executions seems not to update and reload db model properly.

[Tzvetelin88]

trafficstars

Issue Creation Checklist

[x] I have read the contribution guidelines

Issue Description

Payload for the execution:

let payload = {
  "group": {
    "tasks": [
      {
        "name": "Task 1 Name",
        "id": "1"
      },
      {
        "name": "Task 2 Name",
        "id": "2"
      },
      {
        "name": "Task 3 Name",
        "id": "3"
      },
      {
        "name": "Task 4 Name",
        "id": "4"
      }
    ]
  }
}

When running parallel execution by doing this:

await Promise.all(
  group.tasks.map((task: any) =>
    runTask(task)
  )
); 

where "runTask" is:

// getting existing ID's from DB, let assume bellow object as data.
let exeistingIds = {a:1,b:2}

await executorModel.update({
  id: {
      ...exeistingIds,
      [`${this.payload.id}`]: executionId,
    },
  });
await executorModel.reload();

Problem is when we have parallel executions. If we put brake points at:

await executorModel.update

and

await executorModel.reload();

we see that we hit "await executorModel.update" 4 times, and then 4 times "await executorModel.reload()", which leads to wrong information saved to db.

When we have sequential executions, there is no problem. For each task execution we hit "await executorModel.update" 1 time and then we hit "await executorModel.reload()" one time, and continues.

Issue Template Checklist

Is this issue dialect-specific?

• [x] No. This issue is relevant to Sequelize as a whole.
• [ ] Yes. This issue only applies to the following dialect(s): XXX, YYY, ZZZ
• [ ] I don't know.

Would you be willing to resolve this issue by submitting a Pull Request?

• [ ] Yes, I have the time and I know how to start.
• [ ] Yes, I have the time but I don't know how to start, I would need guidance.
• [ ] No, I don't have the time, although I believe I could do it if I had the time...
• [x] No, I don't have the time and I wouldn't even know how to start.

[fzn0x]

This is related to how asynchronous works.

Do you already try to put reload above update?

[Tzvetelin88]

Yes, the result is the same. If I add additional async function "await testMe()" before "await executorModel.update", it resolves each task in testMe, but still async function from Sequelize update and reload don't hit properly:

async function testMe() {
  return new Promise((resolve) => {
    console.log(`Task ID: ${this.payload.id}`);
    resolve('');
  });
}

maybe the issue is related to some cache or buffer in Sequelize, or some transaction issue?

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 14 days without activity. It will be closed if no further activity occurs within the next 14 days. If this is still an issue, just leave a comment or remove the "stale" label. 🙂

[ephys]

I'd need to see more code to be sure but this doesn't look like a sequelize issue. It looks like you might be modifying the same attribute on the same row multiple times in parallel. The operations aren't atomic so you end up with concurrency issues.

The field you modify looks like JSON. Depending on your dialect, you may have SQL functions you can use to add something to your JSON without needing to fetch it first, which would resolve the concurrency issue. In postgres, that would be jsonb_set, not sure about other dialects

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 14 days without activity. It will be closed if no further activity occurs within the next 14 days. If this is still an issue, just leave a comment or remove the "stale" label. 🙂