data pipeline configuration proposals

[max-mapper]

entries we have so far:

• https://github.com/datproject/datscript/blob/master/example.ds
• https://github.com/datproject/datscript/blob/master/example-bionode.ds
• https://github.com/ekg/datscriptish/blob/master/example.dsh

the idea is to use a runtime like https://www.npmjs.org/package/gasket to run these. they are supposed to be less like full scripting languages and more a data structure/configuration that defines the relationship between unix commands in a pipeline

cc @karthik @sckott

[max-mapper]

Update

We have drafted a combined proposal here: https://github.com/datproject/gasket/issues/17

@melaniecebula has updated the hackfile parser to reflect the proposal: https://github.com/mafintosh/hackfile

Now @melaniecebula is working on a hackfile -> gasket converter https://github.com/melaniecebula/datscript#usage and will also be updating gasket to implement the new functionality required by the proposal.

[max-mapper]

food for thought http://vincebuffalo.com/2013/08/08/the-mighty-named-pipe.html

[karthik]

Good stuff!

On Tue, Mar 3, 2015 at 12:06 PM, Max Ogden [email protected] wrote:

food for thought http://vincebuffalo.com/2013/08/08/the-mighty-named-pipe.html

Reply to this email directly or view it on GitHub: https://github.com/datproject/discussions/issues/16#issuecomment-77023503