sql-runner

sql-runner copied to clipboard

Placeholder for Golang / library upgrades

7

* Should be broken out into individual tickets * Will have to pay particular attention to any Postgres driver bumps for compatibility testing

alexanderdean

Update docs

8

Currently using v0.5.2 I have a playbook with 2 steps. I can see the unloads finish on redshift, however the last step never executes and I get a timeout error...

dcrasto

Decide whether log should go to stdout

1

Ideally failures would go to stderr...

alexanderdean

Add better documentation for how to use the -runQuery argument

2

NielsKSchjoedt

Add consul template function

1

jbeemster

sql-runner fails on IF EXISTS statements

19

For example, web-incremental step 00-preparation: https://github.com/snowplow/snowplow/blob/master/5-data-modeling/sql-runner/redshift/sql/web-incremental/00-preparation/00-preparation.sql#L25 Gives this output ``` 2015/10/07 04:12:30 EXECUTING 00-preparation (in step 00-preparation @ snowplow): /opt/snowplow/sql_runner/analytics/sql/web-incremental/00-preparation/00-preparation.sql 2015/10/07 04:12:30 FAILURE: 00-preparation (step 00-preparation @ target snowplow), ERROR:...

gabrielcrowdtilt

Add support for preconditions

2

### Use case 1 Imagine that: * You have a process landing new data in Redshift roughly every 40-60 minutes * You want to kick off a SQL Runner playbook...

alexanderdean

Make timeout for individual query user-configurable

1

Currently it's hard-coded to 8 hours: https://github.com/snowplow/sql-runner/blob/0ae72da4357bd7627eda99602d0139faaf437d06/sql_runner/postgres_target.go#L25 It would be nice to set this as part of the playbook, given that different playbooks might have very different timeout expectations (a...

alexanderdean

Make message on no affected rows more explicit

1

E.g. message: ``` 2018/03/26 23:05:18 SUCCESS: 00a-events (step 01-model/00a-events @ target com.acme redshift), ROWS AFFECTED: -1 ``` Means that query [can't affect any rows in principle](https://github.com/go-pg/pg/blob/v5.3.3/types/result.go#L33-L35), but `-1` is slightly...

chuwy

alexanderdean