SciCatProject
Question: how to include PSI Ingestor and depositor frontend pages?
Summary
At the 2025-01-21 meeting I showed PSI's new Ingestor and OneDep deposition pages. The main question is how (or whether) we should contribute these features back to the main frontend codebase.
Details
Slides are available here, but I'll repeat the summary:
- The Web Ingestor page doesn’t require tight integration. It’s an independent page, and only requires an additional link from the scicat menubar. The ingestor could be a general tool for facilities with non-trivial data transfer steps and is applicable to all data types.
- The Depositor is tightly integrated into the Dataset Details as a new tab. It is selectively enabled based on dataset properties. And it only applies to cryoEM data, making it not relevant to many facilities.
Implementation options
Three implementation options were discussed.
- Enableable Feature in Core. Features are included in the main scicat image (but must be enabled in the config). Example: the current 'Create Dataset' feature
- Separate Service + links. The feature runs as a separate web service. The only frontend change is to allow custom links (configured similarly to #1700). Example: the DOI landing pages
- Plugin system/microfrontend. Refactor the frontend to allow site-specific code to be loaded at runtime. Example: ICAT microfrontend
| Enableable Feature in Core | Separate Service + links | Plugin system/microfrontend | |
|---|---|---|---|
| Implementation difficulty | + Easy | + Medium (needs separate repo and web service) | - Major refactor |
| Integration with other pages | + Full integration possible | - No integration (link only) | + Tight integration is possible (depending on plugin API) |
| Frontend JS size | - Increases frontend size, even for sites that disable it | + Minimal code size increase | + Depends on implementation |
Some additional links were posted in chat for various plugin approaches (thanks @dylanmcreynolds and @minottic )
- https://module-federation.io/
- https://single-spa.js.org/
- https://webpack.js.org/concepts/module-federation/
Personally I would probably go with either the separate service approach ( note, that without a sso solution inplace the user may needs to login multiple times) or the plugin system as that allows better facility customizations.
as much as I like 3, I think it requires a huge amount of work and it's personally not what I would start with. IMO, a solution in between, especially for the depositor which is well scoped, is web components (or loading JS scripts dynamically if we want an even quicker solution), which could be enabled selectively as part of deploying the FE through its config.
Related:
- #1519
- #1642
- #986
- #1700
- https://github.com/SciCatProject/scicat-backend-next/issues/862
Number 1 is the easiest from our point of view. Are there strong opinions against this? There are many features implemented this way already.
I agree with @minottic that solution 3 requires a major refactoring and we are lacking resources at the moment. After digesting all the information provided, I would try with solution 1 provided that we generalize and well-define the functionalities. Regarding the dataset ingestion, I propose that this feature should be the evolution of the rudimentary "create dataset" functionality that is currently present in FE, but is clearly limited and confusing. About the depositor, I would suggest that this should implemented as "dataset actions" which can be configurable with a set of rules similar to the "datafiles" actions which are already in the FE code
Thanks, Spencer for putting it down here. I'd go for option 2 as this emphasizes the strength of scicat to integrate with other systems easily and institutes can decide if they want it or not. I'm not much in favor of option 1, as other instituts would have to carry addtional load they may not want - you mentioned yourself it may be not so useful to "many facilities" due to its specific use case . However I think with option 2, the respective site admin can be inspired and adopt to his case. Option 3 would be off the table due to its implementation efforts.
I vote option 1 for the Web Ingestor. It's a feature that already exists in the scicat core (create dataset form) with a greater enhancement. For the Depositor option 3. I don't quite see the use case at the moment, so would prefer it to be as flexible as possible
I vote option 1 for the Web Ingestor. It's a feature that already exists in the scicat core (create dataset form) with a greater enhancement.
I very much agree with this!
And still think the depositor should be a web component...
Based on the discussion in today's meeting, we will prepare a PR for option 1 (using the new customization options for buttons). This will make it easier to test functionality and make sure it's suitable for inclusion in core.