Automate dispensary libraries updates

[diox]

trafficstars

Updates to libraries in this repo are made by humans (generally @wagnerand), with someone:

• Checking if there are updates manually
• Updating the json file with the relevant version number for the library that needs to be updated
• Generating the hashes by running npm update
• Tagging and releasing a new version

We need to find ways to automate this process. Particular attention needs to be paid to the hashes, to make sure they match what the developers would pick when downloading the files from the website of the library.

[EnTeQuAk]

Maybe worth looking into GitHub Actions regarding all or some of these steps.

On Thu, Feb 20, 2020, at 3:44 PM, Mathieu Pillard wrote:

Updates to libraries in this repo are made by humans (generally @wagnerand https://github.com/wagnerand), with someone:

• Checking if there are updates manually
• Updating the json file https://github.com/mozilla/dispensary/blob/master/src/libraries.json with the relevant version number for the library that needs to be updated
• Generating the hashes by running npm update
• Tagging and releasing a new version We need to find ways to automate this process

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mozilla/dispensary/issues/702?email_source=notifications&email_token=AABB6GLWW3O3SETDGK7RA7TRD2JLXA5CNFSM4KYQHL7KYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IPAKH6A, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABB6GK2Q7UXZQNTUU747SLRD2JLXANCNFSM4KYQHL7A.

[willdurand]

@diox so what do you think we should be doing here? Would adding some documentation be a good start?

I am not sure to understand how this project works but if we had more docs, I guess we could share the workload (e.g., maybe by adding this step to the tagging routine). Of course if we can automate as much as we can (without spending too much time building a rocket), that would be nice.

[diox]

Doc is a good start (we can start with what I've written in the description, which is the extend of my knowledge on the subject at the moment :), but as mentioned in meeting yesterday it's not been updated regularly recently and the sky hasn't fallen. I don't think it's worth adding that workload to the tagging process, I'd much prefer automating as much as we can first.

[willdurand]

@wagnerand would you be able to help?

[wagnerand]

My workflow is as follows:

1. Open src/libraries.json
2. Open all of: https://github.com/angular/angular.js/releases https://github.com/jashkenas/backbone/releases https://github.com/twbs/bootstrap/releases http://download.dojotoolkit.org/ https://github.com/cure53/DOMPurify/releases http://jquery.com/ http://jqueryui.com/ https://github.com/moment/moment/releases http://mootools.net/core http://prototypejs.org/ https://github.com/facebook/react/releases https://github.com/jashkenas/underscore/releases https://github.com/mozilla/webextension-polyfill/releases
3. On each page, check whether there are newer release versions than what is in src/libraries.json. (Note that some libraries, like react, support several versions, we need to check each "branch")
4. For major upgrades, take a quick look at the code changes
5. Add new versions to src/libraries.json
6. Run npm run update
7. Commit and push (Make sure to include src/libraries.jsonand src/hashes.txt)
8. Tag and release as usual

[willdurand]

awesome, I'll update the README based on your comment then, thanks

[wagnerand]

Thanks! The urls are really just an extraction and simplification of the urls defined in src/libraries.json. Maybe we can extract that in the README, so that info doesn't go stale when we add or remove libraries, or update the urls for existing libraries.

[willdurand]

Thanks! The urls are really just an extraction and simplification of the urls defined in src/libraries.json. Maybe we can extract that in the README, so that info doesn't go stale when we add or remove libraries, or update the urls for existing libraries.

That is a good idea but URLs are quite different so I don't know.

[wagnerand]

Oops, I meant to say maybe we can abstract that in the README.

[willdurand]

Oops, I meant to say maybe we can abstract that in the README.

@wagnerand Ha 😅 I was thinking about adding a releasePage attribute in the JSON otherwise (and generate the list of release pages in the README based on that). WDYT?

[wagnerand]

That sounds good. Note that my list is also a simplification. For example, I use the jQuery homepage because that shows the latest version number prominently. Maybe using a github releases page for (eventually) automated version number extraction would work better. Feel free to adapt the links to what works best.

[willdurand]

I removed this issue from the Project because we have some docs now. Of course, it is not automated yet, but it is at least documented and someone else could update the libraries as of today.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this bug should stay open, please comment on the issue with further details. Thank you for your contributions.

[willdurand]

I don't think we can add a lot more automation to this process given we need to review the different updates manually. Lets revisit this decision later if needed.