digdag
digdag copied to clipboard
td.database isn't used when td> operator with local secrets `td.database` given
When I set a secret td.database
, td>
operator fails even if td.database
variable is set in dig file.
The following procedure assuming TD access info is stored in ~/.td/td.conf
.
_export:
td:
database: sample_datasets
+td_task:
td>:
query: "select 1"
# assume job-monitroing DB doesn't exist
$ digdag secrets --set td.database=job-monitoring
$ digdag run test.dig
2019-10-06 11:32:29 +0900: Digdag v0.9.37
2019-10-06 11:32:29 +0900 [INFO] (main): Database migration started
2019-10-06 11:32:29 +0900 [INFO] (main): Applying database migration:20190318175338
2019-10-06 11:32:29 +0900 [INFO] (main): Database migration successfully finished.
2019-10-06 11:32:29 +0900 [WARN] (main): Reusing the last session time 2019-10-06T00:00:00+00:00.
2019-10-06 11:32:29 +0900 [INFO] (main): Using session /Users/aki/src/workflow-examples/machine-learning-box/house-price-prediction/.digdag/status/20191006T000000+0000.
2019-10-06 11:32:29 +0900 [INFO] (main): Starting a new session project id=1 workflow name=test session_time=2019-10-06T00:00:00+00:00
2019-10-06 11:32:30 +0900 [INFO] (0018@[0:default]+test+td_task): td>:
2019-10-06 11:32:30 +0900 [INFO] (0018@[0:default]+test+td_task): td-client version: 0.8.11
2019-10-06 11:32:30 +0900 [INFO] (0018@[0:default]+test+td_task): td>:
2019-10-06 11:32:32 +0900 [WARN] (0018@[0:default]+test+td_task): failed to start job: domainKey=4e57cafd-29c1-44a1-a298-2a4df8b7d8bd
com.treasuredata.client.TDClientHttpNotFoundException: [TARGET_NOT_FOUND] [404:Not Found] API request to /v3/job/issue/presto/job-monitoring has failed: Resource not found
at com.treasuredata.client.TDRequestErrorHandler.defaultHttpResponseErrorResolver(TDRequestErrorHandler.java:119)
at com.treasuredata.client.TDHttpRequestHandler.resolveHttpResponseError(TDHttpRequestHandler.java:88)
at com.treasuredata.client.TDHttpClient.submitRequest(TDHttpClient.java:410)
at com.treasuredata.client.TDHttpClient.submitRequest(TDHttpClient.java:460)
at com.treasuredata.client.TDHttpClient.call(TDHttpClient.java:551)
at com.treasuredata.client.TDHttpClient.call(TDHttpClient.java:517)
at com.treasuredata.client.TDClient.doPost(TDClient.java:233)
at com.treasuredata.client.TDClient.submit(TDClient.java:624)
at io.digdag.standards.operator.td.TDOperator.submitNewJob(TDOperator.java:224)
at io.digdag.standards.operator.td.TDOperator.lambda$submitNewJobWithRetry$8(TDOperator.java:262)
at io.digdag.standards.operator.td.TDOperator.submitNewJob(TDOperator.java:242)
at io.digdag.standards.operator.td.TDOperator.lambda$submitNewJobWithRetry$9(TDOperator.java:268)
at io.digdag.util.RetryExecutor.run(RetryExecutor.java:166)
at io.digdag.util.RetryExecutor.run(RetryExecutor.java:142)
at io.digdag.standards.operator.td.TDOperator.submitNewJobWithRetry(TDOperator.java:268)
at io.digdag.standards.operator.td.TDOperator.submitNewJobWithRetry(TDOperator.java:262)
at io.digdag.standards.operator.td.TdOperatorFactory$TdOperator.startJob(TdOperatorFactory.java:235)
at io.digdag.standards.operator.td.BaseTdJobOperator.lambda$runTask$0(BaseTdJobOperator.java:59)
at io.digdag.standards.operator.td.TDOperator.runJob(TDOperator.java:319)
at io.digdag.standards.operator.td.BaseTdJobOperator.runTask(BaseTdJobOperator.java:59)
at io.digdag.util.BaseOperator.run(BaseOperator.java:35)
at io.digdag.core.agent.OperatorManager.callExecutor(OperatorManager.java:330)
at io.digdag.cli.Run$OperatorManagerWithSkip.callExecutor(Run.java:706)
at io.digdag.core.agent.OperatorManager.runWithWorkspace(OperatorManager.java:271)
at io.digdag.core.agent.OperatorManager.lambda$runWithHeartbeat$2(OperatorManager.java:144)
at io.digdag.core.agent.LocalWorkspaceManager.withExtractedArchive(LocalWorkspaceManager.java:25)
at io.digdag.core.agent.OperatorManager.runWithHeartbeat(OperatorManager.java:142)
at io.digdag.core.agent.OperatorManager.run(OperatorManager.java:125)
at io.digdag.cli.Run$OperatorManagerWithSkip.run(Run.java:688)
at io.digdag.core.agent.MultiThreadAgent.lambda$null$0(MultiThreadAgent.java:131)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-10-06 11:32:32 +0900 [ERROR] (0018@[0:default]+test+td_task): Task +test+td_task failed.
[TARGET_NOT_FOUND] [404:Not Found] API request to /v3/job/issue/presto/job-monitoring has failed: Resource not found (td client http not found)
2019-10-06 11:32:32 +0900 [INFO] (0018@[0:default]+test^failure-alert): type: notify
error:
* +test+td_task:
[TARGET_NOT_FOUND] [404:Not Found] API request to /v3/job/issue/presto/job-monitoring has failed: Resource not found (td client http not found)
Task state is saved at /Users/aki/src/workflow-examples/machine-learning-box/house-price-prediction/.digdag/status/20191006T000000+0000 directory.
* Use --session <daily | hourly | "yyyy-MM-dd[ HH:mm:ss]"> to not reuse the last session time.
* Use --rerun, --start +NAME, or --goal +NAME argument to rerun skipped tasks.