gos
gos copied to clipboard
gosling-lang
Support for Ibisdata
Just learned of gosling today via twitter. I used to work at heavy.ai on large-scale dataviz and have been pretty close to the Vega/VegaLite ecosystem for a few years, including collaborations with that team.
First of all, wanted to say that this is a FANTASTIC project - kudos to everyone involved for a truly next-gen way of taking genomics data and turning it into a comprehensive set of visualizations. I've always felt that this area has been underserved and you've all done a tremendous job!
I work at sneller.io now - which may be of some interest. We allow for SQL on very large-scale JSON collections - see here. It may be of interest if you have data collections of this sort.
Next - One of the projects I worked on in the past is IbisData - This allows you to use SQL databases/DWs as backends in a pythonic data workflow without needing SQL directly. It might be interesting to explore how to back gosling/gos with Ibis so you can put large genomic data sets in central storage and use Ibis to only pull in the data needed on demand.
Hope this helps - again, fantastic work and will stay close to this!
Thanks for the kind words and interest in gos! I will definitely look into IbisData.
One thing I'm very interested in is (optional) further integration with Python backends to support better interactive visualization of genome-mapped data during analysis in notebook environments. This idea of a data layer with Ibis to pull data on demand is very exciting.
When I was at OmniSci (now heavy.ai), we worked on exactly this goal, by creating an IbisTransform in Vega that used in conjunction with Altair to allow in-notebook exploration of very large datasets using Ibis - https://summit.omnisci.com/sessions/using-altair-ibis-and-vega-for-interactive-exploration-of-omnisci. We did this work with folks from Quansight founded by Travis Oliphant (of numpy fame).
Happy to answer questions on how this might work...
See this ungated version of the same presentation with Saul - https://www.youtube.com/watch?v=DTl32fWhm6c