noobaa-core icon indicating copy to clipboard operation
noobaa-core copied to clipboard

NC | Config files updates are not isolated. Concurrent updates can override one another

Open dannyzaken opened this issue 5 months ago • 0 comments

Environment info

  • NooBaa Version: 5.17
  • Platform: Non Containerized

Actual behavior

  1. Config file updates are not synchronized, and when performed concurrently on the same file, data corruption can occur.

  2. In most update flows, an object (e.g., a bucket) is read from the config file, then modified, and the complete object is sent to update the config file.

  3. When two such updates occur concurrently, such that both updates read the old data, each modify a different property, and then both attempt to write, the second write will override the first write changes.

  4. a concrete example of two bucket update flows: put bucket tagging: https://github.com/noobaa/noobaa-core/blob/8961f1d524265a4fce5e7951d81d8d96b33ab3bf/src/sdk/bucketspace_fs.js#L435-L445

    put bucket encryption: https://github.com/noobaa/noobaa-core/blob/8961f1d524265a4fce5e7951d81d8d96b33ab3bf/src/sdk/bucketspace_fs.js#L526-L537

    let's take a look at this sequence for example:

    1. process 1 reads the bucket in line 439
    2. process 2 reads the bucket in line 530
    3. process 1 sets bucket.tag and updates the bucket config files (lines 440-441)
    4. process 2 sets bucket.encryption and updates the bucket files (lines 531-533) with data that doesn't have the bucket.tag that was set in step 3.

Expected behavior

  1. 2 concurrent updates should be isolated from one another

Steps to reproduce

More information - Screenshots / Logs / Other output

dannyzaken avatar Sep 10 '24 15:09 dannyzaken