alldefector

Not yet... `deepdive sql` has a fallback python script to do the formatting: https://github.com/HazyResearch/deepdive/blob/13e8788b9bf39e2d3aa83d0883a6391c9c9d6592/database/db-driver/postgresql/db-query#L42 https://github.com/HazyResearch/deepdive/blob/13e8788b9bf39e2d3aa83d0883a6391c9c9d6592/database/pgtsv_to_json However, I think it may not be reliable with long values (when I tried half...

BTW, the post-processing approach was used by Vincent: https://github.com/infinitespace/memex-search/blob/master/app.ddlog#L202

Here is another indexing speed optimization: https://github.com/HazyResearch/mindbender/commit/8d4169ab6784236f21e3caf7c794830f54b66357

No, ES seems to have a very flexible thread pool scheme in one node and can saturate all cores. I suspect that even if there is only one shard, it's...

Another key performance knob is ES_HEAP_SIZE: https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html But default ES's heap size is 0.25-1G. We may want to use a different default.

Just to second @netj and @chrismre, 1. We shouldn't hard code the auth params in the compiled auth-api.coffee. It should be configurable out of the big final binary. Env vars...

Confirmed: anonymous users can bypass this middleware and successfully query the ES proxy (even if the request has not csrftoken cookie). I'm not familiar with nodejs enough to fix it.