Utku Demir
Utku Demir
Correct syntax: ``` {-# LANGUAGE TemplateHaskell #-} main = $$(error "foo") ``` Hindent gives: `Parse error: $$` ---- Also on some cases it mixes them up with infix operators: From:...
This is an umbrella issue to gather useful datasets which can be used freely. Things to consider: * We should be able to access them freely & quickly. It helps...
k8s seems to be everywhere, and many major cloud providers support it. We can simply package the binary as a Docker container (with an added advantage that it doesn't require...
* Run read and write ends of the conduits concurrently. * When reading a `Partition` created by `` operations, consume the smaller partitions in parallel.
Parquet is a commonly used data format, but sadly Haskell ecosystem is lacking a mature library. If we implement a library to encode/decode Parquet files, we can both use it...
YARN is the most common way to schedule Spark & Hadoop on a cluster. Supporting it as an executor will enable us to run side-by-side with existing data processing pipelines.
Currently, we expect users to write a `Conduit` to read data from external sources. This is quite easy, however it would be even better to provide some combinators to use...
Currently the CI only tests Nix builds. We should also test stack and cabal. We can migrate away from Travis while doing this.
This is one of the more exciting features. Apache Spark has support for running SQL queries at runtime in an untyped fashion. It is quite useful when exploring the data...