archiveweb.page
archiveweb.page copied to clipboard
Consider refactoring into two libraries
This repo is becoming complex and now has several deployments, including the extension, electron app, and embedded record mode, and a CDP-protocol wrapper for recording
Perhaps it would make sense to refactor this into two libraries:
-
a core 'recorderlib' which consists of a service worker + CDP and fetch recording library + ipfs writing functionality that extends wabac.js and offers recording mode, using CDP and/or fetch, WACZ/WARC serialization, signing. Could be used in both browser, electron and plain node.
-
the ui, which extends replayweb.page, and use recorderlib to perform the recording. The various UI manifestations will continue to live here
Hi We are building extension which only handles the capture, so in the long term I would think of splitting it up to:
- capturing the site Apis
- archive browser One more thing regarding replay , In my case replaying of social media modern site is my first concern , So having flash support for example makes the package heavier than we need. I am reviewing the code and I would say I am really impressed, well done!!
Part of this has already been done: the service worker part of the extension, which extends wabac.js, has been moved to awp-sw library.
It is possible to split things further, such as all of the UI from the recording logic..
Hi We are building extension which only handles the capture, so in the long term I would think of splitting it up to:
Hi, do you mean you are building your own extension on top of archiveweb.page, or your are using archiveweb.page?
- capturing the site Apis
- archive browser
Do you mean just splitting the UI and recording API as suggested above?
One more thing regarding replay , In my case replaying of social media modern site is my first concern , So having flash support for example makes the package heavier than we need. I am reviewing the code and I would say I am really impressed, well done!!
Thanks, yes, currently looking at how to make Flash optional potentially, though its not possible to load code dynamically is discouraged in extensions, so for now, we are shipping the Ruffle player.
Hi, do you mean you are building your own extension on top of archiveweb.page, or your are using archiveweb.page? We are building the extension using archiveweb.page base code , we removed the archive-page-app and its dependencies from the extension making it smaller Do you mean just splitting the UI and recording API as suggested above? Yes
One more thing since we are planned to work on the extension , we think we could benefit the project as well by:
- Migrating to manifest v3
- Adding upload to AWS S3 Bucket (Need to think how it can be added mainly how its configured)
- We will test many sites with the extension - we will post issues if we encounter problems If you have more ideas , I'll be happy to hear I can be reached via mail tsemachh at shefing.com or in slack as well
BTW: Are there any public architecture documents regarding the extension internals ? I have fresh graduate I would like her to dive to the code there.