etl icon indicating copy to clipboard operation
etl copied to clipboard

Add ETL active updates and heartbeats.

Open gfr10598 opened this issue 5 years ago • 1 comments

Part of m-lab/dev-tracker#501

Use gardener update/ to send per task updates. Use gardener heartbeat/ to send per job heartbeat, once per minute. This allows gardener to detect ETL instance crashes.

gfr10598 avatar Dec 15 '19 15:12 gfr10598

Lots of prototyping in the universal-sandbox-gfr branch. Had stuff working with alpine-glibc, but Stephen suggested avoiding that dependency. However, with

FROM ubuntu
RUN apk add --no-cache ca-certificates

I see errors on the etl side like:

active.go:170: Error streaming Get https://storage.googleapis.com/storage/v1/b/archive-mlab-sandbox?alt=json&prettyPrint=false&projection=full: x509: certificate signed by unknown authority

Trying again with:

FROM frolvlad/alpine-glibc
RUN apk update && apk add ca-certificates && rm -rf /var/cache/apk/*

gfr10598 avatar Jan 14 '20 23:01 gfr10598