digdag
digdag copied to clipboard
treasure-data
Workload Automation System
Digdag version: 0.9.42 We have the `_error` parameter and main tasks in our workflow. example: ``` timezone: Asia/Tokyo _error: echo>: here is _error configuration. +test: +task1: sh>: sleep 2; exit...
Google BigQuery table can be created with Google Sheet as follows.  When querying this kind of table, bq> operator needs drive read scope: "https://www.googleapis.com/auth/drive.readonly". (https://github.com/googleapis/google-api-java-client-services/blob/master/clients/google-api-services-sheets/v4/1.30.1/com/google/api/services/sheets/v4/SheetsScopes.java#L33) However, digdag does not...
Digdag users want to share files between tasks on server mode. Use cases * Download from a database. And use it. * ~~Download file from a database. and execute embulk...
I am trying to run digdag on kubernetes. Most functions work properly on kubernetes. However, it is difficult to handle the scale-out and scale-in behavior of the digdag worker agent....
I want to support the sidecar pattern for `kubernetes command executor`. This is because supporting the side car allows you to perform tasks more flexibly, such as monitoring pods(the task...
While rb> operator has a solution for installing dependent packages with bundler as discussed in https://github.com/treasure-data/digdag/issues/318, it would be nice if we could have Python dependency installation for py> operator....
This issue is a list of document improvement TODO. - [ ] Secrets overview #424 - [x] Plugin list #925 (Not official, but useful) - [ ] Missing config file...
### Description `td_for_each>` operator does not support query saved on Treasure Data. ### Request `td_for_each>` supports to use a query saved on Treasure Data , or add new operator like...
gcs_wait> operator should have timeout related parameters. Design and specification should be common with s3_wait> as much as possible. s3_wait> timeout is implementing in #1280.
About the kubernetes command executer in the v0_10 branch. I hope to able to set node affinity property in k8s command executor. https://github.com/treasure-data/digdag/blob/5ad5fd91617b8bcde9dc48e7cc112880e5b21f53/digdag-standards/src/main/java/io/digdag/standards/command/kubernetes/DefaultKubernetesClient.java#L142