Brett Naul

ADAP-464: Add allow_non_incremental_definition option to BigQuery materialized views

7

Closes #672. ### Problem Non-incremental materialized views are much more flexible than incremental and have a lot of uses: see e.g. https://medium.com/google-cloud/hidden-gems-of-bigquery-part-4-five-types-of-views-ddfbc05118fb#bc99 ### Solution Add `allow_non_incremental_definition` flag following the pattern...

Uncaught exception for incorrect application name

3

Makes sense that this would fail but it was a bit tricky to debug because of the error. Changing our `STORMPATH_APPLICATION` config variable led to the following error: ``` ERROR:cesium.Flask.flask_app:Exception...

Lots of little things: - Inconsistent capitalization from one tab to another - One vs. two word choices not being consistently followed (e.g. dataset, data set) Should do a thorough...

Re-implement custom features

2

Presumably still using Docker.

Improve handling of invalid data / feature values

2

NaNs, etc.

Use multiple workers

2

Right now we hard-code a single 1-thread worker; should use the # of cores or something along those lines.

Especially when an error occurs, having the full (potentially detailed) description of what failed is important for debugging on the front end.

Slow dask task creation for complex functions

2

### Bug summary I've been seeing very slow task creation (~1/s) when `.map`ing a classmethod from a not particularly crazy class. Using plain old Dask `client.map()` on the same function...

Autoscaling removes workers in other worker groups besides default

4

**Describe the issue**: Per https://kubernetes.dask.org/en/latest/operator_resources.html#daskautoscaler, only the default worker pool should autoscale. But I'm observing other worker groups also getting scaled down at the end of a computation. **Minimal Complete...