RemoteRScriptRepository and GitHubScriptRepository

[nuest]

Implement an abstract class RemoteRScriptRepository that can manage scripts from remote locations, namely URLs or URIs. On startup scripts are copied into a local directory (based on an an abstract method public abstract Collection<InputStream> getScripts() and then loaded. The loading functions can probably be re-used from the LocalR...Repository. The remote repository should allow to set a fixed interval on which it updates the local copies of the scripts.

The GitHubScriptRepository will allow to define a GitHub username and repository name and from that information download the R files from that repository.

• JGit is EDL (http://eclipse.org/jgit/ - https://wiki.52north.org/bin/view/Documentation/ThirdPartyLicensesForGPL) and on Maven Central (http://mvnrepository.com/artifact/org.eclipse.jgit/org.eclipse.jgit) so we can just clone the repo!
• Maybe calling git clone on the command line via JNDI is a more lightweight alternative, but it requires git to be installed.
• Or the GitHub API (https://developer.github.com/v3/repos/#get), e.g. https://api.github.com/repos/52North/WPS - but I didn't find an option for file download...

[autermann]

Or the GitHub API (https://developer.github.com/v3/repos/#get), e.g. https://api.github.com/repos/52North/WPS - but I didn't find an option for file download...

You don't need the API for that:

https://github.com/$USER/$REPO/archive/$BRANCH_OR_TAG_OR_COMMIT.{zip,tar.gz}

[nuest]

Right - that gives me access to a zipped copy of all the current files, e.g. https://github.com/52North/WPS/archive/master.zip - and I can unzip that and look for all the .R files in it.

Thanks for the tip! I think this is a good solution for a first prototype.

[nuest]

The "remote" could also be any file URL that is local for that matter and use the same behaviour for additional (or the current R/scripts directory as well!) script locations and resource locations - just copy them all into a temporary folder, no mischief happening in that controlled location.

[nuest]

Together with #76 this would allow to move the demo R scripts out of the 52n-wps-webapp module into the 52n-wps-r - module into src/../resources, which would be an important improvement for decoupling of the webapp module.

[nuest]

@matthias-mueller fyi - I want to tackle this issue soon to be able to collaborate more easily on R script development, for merely practical reasons. Conceptually this is probably related to your mc-work.

[matthias-mueller]

Yes; let's have a brief talk then to see if we can align the workspace structure.

[nuest]

By now I would prefer using JGit from the start, it allows to also update the local clone outside of the WPS, for example.

Related and maybe also interesting: rgist package to access gists from R.

[matthias-mueller]

Okay, let's see where this goes. I see four or five things that we have previously discussed within the repository concept:

1. Access to executable workspaces or code
2. Access to related source code (especially important for byte code)
3. Versioning
4. Documentation
5. Link with github due to its widespread use

I'd be glad if we could keep things as simple as possible, i.e. not force our users into complex solutions if they just want to upload a simple script. Although I have a strong affinity to using GIT, we should still be able to handle plain (non-git) workspaces and content.

[nuest]

Yes - this is an addition to the simple "drop-in folder" solution that WPS4R already supports.

If you would like to start a discussion on the repository concept in general, could you open a new issue?