FlowFuse
Static Asset Service
Description
When building on FlowFuse, we now have the capability to have persistent file storage. This epic captures user stories that could be unlocked given the new, unlocked, functionality:
Which customers would this be available to
Team + Enterprise Tiers (EE)
### Design Tasks
- [ ] https://github.com/FlowFuse/flowfuse/issues/4352
### User Stories
- [ ] Serving of static assets (e.g. images) to FlowFuse Dashboard
- [ ] Serving of npm libraries to FlowFuse Dashboard
- [ ] Providing access to static assets (e.g. csv files, tensor flow models) in flows running on FF
### Engineering Tasks
- [ ] https://github.com/FlowFuse/nr-launcher/issues/273
- [ ] https://github.com/FlowFuse/flowfuse/issues/4357
- [ ] https://github.com/FlowFuse/flowfuse/issues/4358
- [ ] https://github.com/FlowFuse/flowfuse/issues/4359
- [ ] https://github.com/FlowFuse/nr-launcher/issues/274
- [ ] Support multiple file upload
Listing items currently out of scope of the design. However this will lay the groundwork needed for these items:
### Out of Scope
- [ ] Versioning of files when creating snapshots
- [ ] Deploying files to devices
- [ ] Handling static assets in DevOps Pipelines
- [ ] Managing files whilst instance is suspended
See also:
- https://github.com/FlowFuse/flowfuse/issues/3790
Do we want to add some code to the nr-launcher to allow file lists, upload/delete from persistent storage?
Do we want to add some code to the nr-launcher to allow file lists, upload/delete from persistent storage?
If that's what would be required - then yes 🙂 Hadn't broken this down beyond user stories into engineering tasks yet, but I suspect, based on your question, that would be Step 1?
Well, it's the easiest way to get a consistent view of the available storage (especially on k8s)
The core functionality here is the ability to manage the files stored in persistent storage for an instance. That requires:
- Launcher API for basic file operations (listing, uploading, deleting, renaming)
- Forge API to expose API to frontend
- Frontend provide a file-browser like UI
The next step is then to allow the user to mark a folder as 'shared' (or whatever term we use). As part of that, they get to pick the url path the folder will be shared as. That will be stored in Instance settings and get mapped to the httpStatic setting in Node-RED by nr-launcher.
NB: support for multiple httpStatic routes came in Node-RED 3.0. We also need to consider security of the routes - whether they automatically gain the same protection as the regular HTTP routes, or if that will need some additional work to make happen.
What is currently out of scope is how files relate to snapshots (ie versioning assets to match the flows) as well as how we manage deploying files to devices.
@knolleary are we saying here that assets would be instance-bound? Or could there be scope for having them application/team bound at a later point? Thinking more of Dashboard use-cases with static images/logos
are we saying here that assets would be instance-bound?
First iteration is instance bound. Having files that are shared to all instances could be a future enhancement once we have the basics in place.
For the dashboard use case (wanting the same static assets in multiple instances), the workaround would be to add them to one instance and access them from there in the others.
Quick designs to give direction when construction the front-end. Will follow up with details on the "Upload" interaction soon
I had more of a file browser ux in mind, rather than a flat list of individual files. The sharing option (whether to expose via http) would be done at the folder level (we can't do individual files)
I didnt quite follow the need to share?
Surely anything in the assets is then available? Or is the idea that you could store stuff for future use, but not have it publicly available just yet?
I had more of a file browser ux in mind, rather than a flat list of individual files.
Yep, absolutely the right direction to go, I had 1 hour to flesh out a few different pages, so this was v rushed
I didnt quite follow the need to share?
For the use-case: Providing access to static assets (e.g. csv files, tensor flow models) in flows running on FF - the files need to be accessible to the flows on the local file system, but that doesn't mean I want them to be HTTP accessible.
One UX issue to consider; if the instance is suspended, then nr-launcher isn't running and we don't have access to the filesystem to manage its files. That might be achievable by mounting the file-system elsewhere temporarily, but that gets complicated (and sure if it would be possible in the docker version). So considering that out of scope for first iteration.
the files need to be accessible to the flows on the local file system, but that doesn't mean I want them to be HTTP accessible.
Got it, so essentially difference between accessible in Node-RED in a flow, or if they're also accessible through Dashboard too?
the files need to be accessible to the flows on the local file system, but that doesn't mean I want them to be HTTP accessible.
Got it, so essentially difference between accessible in Node-RED in a flow, or if they're also accessible through Dashboard too?
essentially difference between accessible in Node-RED in a flow, or if they're also accessible through Dashboard too?
Correct.
One UX issue to consider; if the instance is suspended, then nr-launcher isn't running and we don't have access to the filesystem to manage its files. That might be achievable by mounting the file-system elsewhere temporarily, but that gets complicated (and sure if it would be possible in the docker version). So considering that out of scope for first iteration.
Docker has the whole root "volume" mounted to do clean up, but the k8s forge app doesn't currently have any access to the volumes that are mounted into the Instances
First iteration delivered, only outstanding follow up is the "visibility"/share feature which will come in 2.9
Will open a new Epic to track a "Phase 2" of Static Assets to track the "Out of Scope" work detailed here
Is this feature safe to close? The out of scope tasks have been moved to https://github.com/FlowFuse/flowfuse/issues/4556, not sure if we'd need to do the same for the user stories task list.
