flowfuse icon indicating copy to clipboard operation
flowfuse copied to clipboard

Add support for uploading exported snapshots

Open Steve-Mcl opened this issue 1 year ago • 1 comments

Description

Details Updated 2024/05/15

  • All FE and BE API parts refer to the action as "import" (not "upload") as a counterpart to the "export" API actions whereas the FE buttons are labelled "Upload Snapshot" to align with the existing "Download Snapshot" actions in the UI
  • Adds the endpoint POST api/v1/snapshots/import in the Snapshots API
    • The schema for the POST body is fairly strict to ensure that the snapshot is valid and can be imported
      • It could be made further strict by validating the flows array and settings.xxx objects, but this is not done in this PR
  • Adds RBAC role snapshots:import for which only the owner of the instance or device can import snapshots
    • This may be a bit restrictive, since member is permitted to create snapshots?
  • The "Upload Snapshot" buttons are added to the instance and device pages (not the application page: reason provided on the issue)
    • The buttons placed adjacent to the "Create Snapshot" buttons on both the table search row AND the empty state actions slot
    • The buttons are only visible to users with the snapshots:import role
    • fixes the gap between the buttons in the empty state actions slot (in line with the SCSS gap setting for the table search actions slot)
  • Adds shared component frontend/src/components/dialogs/SnapshotImportDialog.vue
    • this is reused on both the instance and device pages
  • Ads a utility function isSnapshot in forge/routes/api/snapshot.js
    • this is used to do basic validation the snapshot before attempting to import it (browser does not have native support for JSON schema validation, so this is a simple check to ensure the snapshot has the required properties and are of the correct type)
  • Adds audit log entry for the import action
    • corrects icon for snapshot log entries
  • Adds unit tests for the new endpoint for both instances and devices
  • Adds E2E tests for the new endpoint for both instances and devices

Unit tests added

  Snapshots API
    Import snapshot
      instance
        ✔ Owner can import snapshot with credentials
        ✔ Owner can import snapshot without credentials
        ✔ Returns 400 for missing snapshot
        ✔ Returns 404 for non-existant instance
        ✔ Returns 400 for missing credentialSecret
        ✔ Returns 400 for bad/invalid snapshot (missing flows)
        ✔ Returns 400 for bad/invalid snapshot (missing settings)
        ✔ Returns 400 for incorrect credentialSecret
        ✔ TeamB member cannot import snapshot for instance belonging to TeamA - 404
        ✔ TeamA Owner cannot import snapshot for instance belonging to TeamB - 404
        ✔ Member cannot import snapshot - 403
        ✔ Viewer cannot import snapshot - 403
      device
        ✔ Owner can import snapshot with credentials
        ✔ Owner can import snapshot without credentials
        ✔ Returns 400 for missing snapshot
        ✔ Returns 404 for non-existant device
        ✔ Returns 400 for missing credentialSecret
        ✔ Returns 400 for bad/invalid snapshot (missing flows)
        ✔ Returns 400 for bad/invalid snapshot (missing settings)
        ✔ Returns 400 for incorrect credentialSecret
        ✔ TeamB member cannot import snapshot for device belonging to TeamA - 404
        ✔ TeamA Owner cannot import snapshot for device belonging to TeamB - 404
        ✔ Member cannot import snapshot - 403
        ✔ Viewer cannot import snapshot - 403


  24 passing (4s)

E2E tests added

  FlowForge - Devices - With Billing
    upload snapshot with credentials
    upload snapshot without credentials
  FlowForge - Instance Snapshots
    upload snapshot with credentials
    upload snapshot without credentials
  FlowForge shows audit logs
    for when a snapshot is imported
pre 15-05-2024

  • Adds new endpoint POST api/v1/snapshots
    • we may want to make this more explicit e.g /upload?
  • Adds front end API uploadSnapshot
  • Adds frontend common dialog frontend/src/components/dialogs/SnapshotUploadDialog.vue

NOTES:

  • Only implemented on application->device and on instance pages. The Application->snapshots page was not included in this iteration as it would require presenting a full list of devices and instances for the user to select from.
  • This is a super early (but working) draft PR, issues include:
    • No audit logging
    • Double notifications on upload error
    • Device info not present in the response from the newly created snapshot (table column missing info until refresh)
    • Validation on credential secret should probably re-use the validation methods created in export snapshot

Related Issue(s)

#3628

Checklist

  • [x] I have read the contribution guidelines
  • [x] Suitable unit/system level tests have been added and they pass
  • [ ] Documentation has been updated
    • [ ] Upgrade instructions
    • [ ] Configuration details
    • [ ] Concepts
  • [ ] Changes flowforge.yml?
    • [ ] Issue/PR raised on FlowFuse/helm to update ConfigMap Template
    • [ ] Issue/PR raised on FlowFuse/CloudProject to update values for Staging/Production

Labels

  • [ ] Includes a DB migration? -> add the area:migration label

Steve-Mcl avatar May 08 '24 18:05 Steve-Mcl

Codecov Report

Attention: Patch coverage is 85.24590% with 9 lines in your changes are missing coverage. Please review.

Project coverage is 79.26%. Comparing base (d99d46c) to head (2d73c87). Report is 2 commits behind head on main.

Files Patch % Lines
forge/routes/api/snapshot.js 84.37% 5 Missing :warning:
forge/db/controllers/Snapshot.js 85.18% 4 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3842      +/-   ##
==========================================
+ Coverage   79.22%   79.26%   +0.03%     
==========================================
  Files         281      281              
  Lines       12660    12721      +61     
  Branches     2813     2837      +24     
==========================================
+ Hits        10030    10083      +53     
- Misses       2630     2638       +8     
Flag Coverage Δ
backend 79.26% <85.24%> (+0.03%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar May 08 '24 18:05 codecov[bot]

Closing this large PR in favour of separate tasks due to growing larger than anticipated.

See new task list in parent story #3628

Steve-Mcl avatar May 16 '24 09:05 Steve-Mcl