Allow download of arbitrary link instead of only Github api

[touilleMan]

While using a Github repo as storage for a plugin is convenient most of the time, it is not the case when the plugin need a processing phase before being usable. This typically include removing unit tests folder, adding custom LICENSE or README file etc. This is even more critical with plugins making use of GDnative and which must be compiled. This way the github repository has really few in common with the actual binary distributed to the users through the asset store.

In the current workflow, this means once build, the plugin must be added to the repository (typically creating a new totally empty branch and adding the build as a single commit). This unnecessary increase the size of the github repo and make it uneasy to work with it in the long run. On top of that Github doesn't accept single file bigger than 100mo which is another shortcoming.

Another solution is to create another Github repository dedicated to serve the release builds. However this is still not really convenient (additional workload, growing size of the repo, max single file size).

I've hit this trouble working on PythonScript where I had to create a dedicated release build repo. For instance the 0.10.0 release is 400mo big (91mo once zipped).

To solve this issue, I think we should allow arbitrary download link for the project build.

• given the hash of the build is kept, this wouldn't be a security issue
• the Github api to download the project as a zip is really similar to zip download from an arbitrary url
• Availability could be an issue if the users start uploading build in non-reliable services (e.g. poorly self-hosted), to address this we could filter the allowed domain (for instance only accept github/gitlab etc. domains)

Another solution would be to only allow the download link to be on the github release page of the project (https://github.com/touilleMan/godot-python/releases in my case)

[bojidar-bg]

It is possible to do it, but currently only moderators can use this feature. If deemed useful enough, we can open it up for normals as well. What do you think, @akien-mga?

[Calinou]

Using GitHub Releases for stuff like that sounds fine to me; it's effectively intended to releasing software after all. :wink:

There are various ways to automate the creation of releases on GitHub, which can be used from Travis CI and the like.

[akien-mga]

I'd prefer not to allow arbitrary URLs (like Mega, Dropbox or ShareThisOrThatAndIn6MonthsItsOffline), but we can add support for the download areas affiliated with the git platforms we support (GitHub, GitLab, Bitbucket, etc.).

[bojidar-bg]

Ok, here are a few proposals:

1. Add it as another "repository/download provider":
2. Add another checkbox:

[punto-]

Can we add an optional subdirectory inside the repository to be the root of the download?

On 4 April 2018 at 09:53, Bojidar Marinov [email protected] wrote:

Ok, here are a few ideas:

1. Add it as another "repository/download provider": [image: mockup 1] https://user-images.githubusercontent.com/5276727/38308131-c7295b32-381e-11e8-8d1d-8665f9956e3d.png
2. Add another checkbox: [image: mockup 2] https://user-images.githubusercontent.com/5276727/38308579-3f6b39b6-3820-11e8-98dd-835090baa6b7.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot-asset-library/issues/140#issuecomment-378589391, or mute the thread https://github.com/notifications/unsubscribe-auth/AGVmPRYAta_971Ihkdo8VwieluROTayTks5tlMJjgaJpZM4TGjIg .

[touilleMan]

Can we add an optional subdirectory inside the repository to be the root of the download?

It wouldn't be useful given this means the user had to download the entire project (github API cannot download a sub-directory as a zip) then extract the interesting sub directory. Beside the issue about fast (and useless) growing of the size of the repo would remain.

Le mer. 4 avr. 2018 20:56, punto- [email protected] a écrit :

Can we add an optional subdirectory inside the repository to be the root of the download?

On 4 April 2018 at 09:53, Bojidar Marinov [email protected] wrote:

Ok, here are a few ideas:

1. Add it as another "repository/download provider": [image: mockup 1] < https://user-images.githubusercontent.com/5276727/38308131-c7295b32-381e-11e8-8d1d-8665f9956e3d.png

2. Add another checkbox: [image: mockup 2] < https://user-images.githubusercontent.com/5276727/38308579-3f6b39b6-3820-11e8-98dd-835090baa6b7.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/godotengine/godot-asset-library/issues/140#issuecomment-378589391 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AGVmPRYAta_971Ihkdo8VwieluROTayTks5tlMJjgaJpZM4TGjIg

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot-asset-library/issues/140#issuecomment-378707914, or mute the thread https://github.com/notifications/unsubscribe-auth/ADCjtS08CVSLkPVAwimdCM_RFb3nvXrDks5tlRdUgaJpZM4TGjIg .

[punto-]

On 5 April 2018 at 04:10, Emmanuel Leblond [email protected] wrote:

Can we add an optional subdirectory inside the repository to be the root of the download?

It wouldn't be useful given this means the user had to download the entire project (github API cannot download a sub-directory as a zip) then extract the interesting sub directory. Beside the issue about fast (and useless) growing of the size of the repo would remain.

Are you sure? this thread seems to imply there is a way https://stackoverflow.com/questions/7106012/download-a-single-folder-or-directory-from-a-github-repo , I haven't looked at it in detail tho, but a couple of the linked tools definitely use the github api. In any case, that's a limitation of github, doesn't have to be ours too. When the underlying source is github the directory can be forced to the root.

Le mer. 4 avr. 2018 20:56, punto- [email protected] a écrit :

Can we add an optional subdirectory inside the repository to be the root of the download?

On 4 April 2018 at 09:53, Bojidar Marinov [email protected] wrote:

Ok, here are a few ideas:

1. Add it as another "repository/download provider": [image: mockup 1] < https://user-images.githubusercontent.com/5276727/ 38308131-c7295b32-381e-11e8-8d1d-8665f9956e3d.png

2. Add another checkbox: [image: mockup 2] < https://user-images.githubusercontent.com/5276727/ 38308579-3f6b39b6-3820-11e8-98dd-835090baa6b7.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/godotengine/godot-asset-library/issues/140# issuecomment-378589391 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AGVmPRYAta_ 971Ihkdo8VwieluROTayTks5tlMJjgaJpZM4TGjIg

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/godotengine/godot-asset-library/issues/140# issuecomment-378707914>, or mute the thread <https://github.com/notifications/unsubscribe- auth/ADCjtS08CVSLkPVAwimdCM_RFb3nvXrDks5tlRdUgaJpZM4TGjIg> .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/godotengine/godot-asset-library/issues/140#issuecomment-378840991, or mute the thread https://github.com/notifications/unsubscribe-auth/AGVmPYqfpZxmQaoa4gahc3Z_Ylez7OIVks5tlcNWgaJpZM4TGjIg .

[cmfcmf]

For GitHub, you can also use a .gitattributes file, like done here. Then downloads using the "Download Zip" button don't include the files listed in the .gitattributes file.

Edit: I realise this fixes only part of the problem described in this issue.

[touilleMan]

Are you sure? this thread seems to imply there is a way

It seems to me that git internals make it impossible for doing such a thing. However there might be custom ways of doing it (I saw they were talking about github implicitly converting your repo into svn before sending you the subpart, another dirty way would be to crawl the webpage to download from there...). So it might be more possible than what thought, but definitely not something standard...