NeurodataWithoutBorders
proposal to use Git LFS for large files
Problem
With the inclusion of several large PDF and ZIP files, this repo is getting quite large. That has caused problems for me pulling and pushing, and I expect the problem to get worse as we add more reports and possibly other types of large files.
Proposed Solution
I propose we try using Git LFS. There are some nice guides for how to:
- Install Git LFS: https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage
- Configure Git LFS for specific file types: https://docs.github.com/en/repositories/working-with-files/managing-large-files/configuring-git-large-file-storage
- Move existing files in your repository to Git LFS storage: https://docs.github.com/en/repositories/working-with-files/managing-large-files/moving-a-file-in-your-repository-to-git-large-file-storage
It seems Git LFS is not supported with GitHub pages site (see bottom of page here).
There is a potential workaround if we switch from deploying from a branch to deploying with GitHub actions - see discussion and related LFS/Pages issues here. From what I understand, this workaround means every workflow run will fetch the LFS data. If that sounds ok I can work on setting that up?
The PDF and Zip files that are causing the issue I believe are the reports from the events and as such are a) not necessary for the build since they are just pointed to as download links and b) don't really need Git-based version control. I'm wondering, whether it may be easier to simply create a shared GoogleDrive for web assets where we can store those files and update the links on the websites to point to the public files in the drive. For the PDF's (and Zip) files of the reports we could also just upload them to Zenodo to get DOI's for the reports. In this way we could still remove them from the repo but not need to deal with the complications that GitLFS brings with it.
