Add Beacon Server API support

[hershman]

With the change over to Flask (see issue #38 and PR #40 for discussion and progress) and our support for plain VCF files using the TabixBackend we have the infrastructure to very easily make a Beacon server. This could be trivial to deploy using the PyPI package (see issue #36) and would require only a path to a directory of VCF files to configure.

[hershman]

I am working on PR for Beacon Spec (0.2)[https://docs.google.com/document/d/154GBOixuZxpoPykGKcPOyrYUcgEXVe2NvKx61P4Ybn4/edit#heading=h.234otr74rjti].

[kerrydc]

Isn't this a bit premature? I mean, a Beacon is something that we don't need until we have a real, deployable app. Outstanding API methods, benchmarking, testing, security, logging, and so on are all higher priority. It is likely the Beacon spec will continue to evolve while we finish up implementing this basic functionality, rendering prior work obsolete. Kerry

On Wed, Jan 14, 2015 at 10:19 AM, Steve Hershman [email protected] wrote:

I am working on PR for Beacon Spec (0.2)[ https://docs.google.com/document/d/154GBOixuZxpoPykGKcPOyrYUcgEXVe2NvKx61P4Ybn4/edit#heading=h.234otr74rjti ].

— Reply to this email directly or view it on GitHub https://github.com/ga4gh/server/issues/41#issuecomment-69931169.

[jeromekelleher]

Not sure I agree @kerrydc - a Beacon could be something that we can turn into a real deployable app much more quickly than the full API. The sooner we ship working code, the better, and I think a lot of people would be interested in a simply deployable beacon. Some of us could concentrate on testing, logging etc for this simpler instance while the development for the larger API is also going on by others.

Feel free to open an early PR once you're ready for comments @hershman.

[maximilianh]

I think Beacons should not depend on any of the other GA4GHs API. Beacons might often run on the external public legacy webserver, not necessarily on an internal cloud. Getting security approval is easier with a simple one-file webservice than a host of libraries, tools and dependencies.

[hershman]

Hi @maximilianh, Thanks for joining the discussion. Your easily deployable beacon at https://github.com/maximilianh/ucscBeacon is really awesome!

It would silly for the reference server to not also be able to host a beacon in addition. I have an implementation that will not be hard to clean up and catch up to the current Master branch and Beacon spec https://github.com/hershman/server/tree/beacon

It would be ideal if the same code base could be used, but the frameworks seem different enough and the amount of implantation seems small enough that I'm excited to forge ahead if the community is.

[bartgrantham]

@maximilianh, I think most would agree that Beacons shouldn't require the use of the variants API. But the variants API reference server does a lot of the dirtywork of wrapping file formats and making them queryable. A basic Beacon server is a small layer on top of this hard work, and it's a great out-of-the-box example of what someone can do with the variants API server.

Also, some of the 0.2 Beacon API features, such as allele frequency, point to new directions for the variant API so it's helpful to have this feedback loop.

[maximilianh]

Oh sure, if it is optional, then this is of course fine. I was mostly thinking of new users at first, those that are not using a full variants API stack yet and just want to share their allele data in some way.

[haussler]

+1

On Wed, Jan 14, 2015 at 12:57 PM, Maximilian Haeussler < [email protected]> wrote:

Oh sure, if it is optional, then this is of course fine. I was mostly thinking of new users at first, those that are not using a full variants API stack yet and just want to share their allele data in some way.

— Reply to this email directly or view it on GitHub https://github.com/ga4gh/server/issues/41#issuecomment-69990092.

[hershman]

I posted rough implementation for discussion as PR #103 .

[david4096]

A GA4GH client can be used to respond to beacon requests from any GA4GH resource and act as a beacon. Work in this direction has been started here https://github.com/kozbo/Beacon-on-GA4GH-API/blob/master/beacon.py

[kozbo]

I propose that the time has come to look at how to merge the Beacon API into the main line GA4GH APIs. People who set up a server will often want to set up a Beacon too. The APIs that the reference server (or any GA4GH server) provides will allow interested parties to issue more sophisticated queries. There are many areas of overlap, but there are also some areas that require harmonization to allow the two APIs to work as one. See the work begun in the schemas here. Some areas that need work are:

1. There wasn't a good mapping for "Organization" in the API so I added a setting in a config file
2. Beacon is meant to be an open API so security controls should apply to the other APIs but not to the Beacon APIs themselves. This should be easy enough to configure with our new Auth0 implementation.

NOTE: I have changed the title of this long-running issue to reflect this new direction. The old title implies that this issue belongs in another repo.