neonKUBE icon indicating copy to clipboard operation
neonKUBE copied to clipboard

telemetry: neon-desktop/neon-cli upload cluster deployment logs on failures

Open jefflill opened this issue 2 years ago • 0 comments

neon-cli and neon-desktop need a way to upload cluster deployment logs for cluster deployment failures. This will be a new neonCLOUD headend service.

  • [ ] Honor the NEONKUBE_DISABLE_TELEMETRY=true environment variable by disabling this feature
  • [ ] Lets just ZIP the files in the log folder and upload those to a new headend REST API
    • [ ] Can we upload potentially large ZIP files in one go or should we transmit as multiple parts?
    • [ ] Record these as Minio and return some kind of link to the blob to be included in an error log
    • [ ] Seems like we'll need the cluster definition too but we'll need to redact any secrets
    • [ ] Add a metadata file to the ZIP with information about user client-id etc. and/or tag the Minio blob with this info
  • [ ] Add a headend API to the service and client that accepts the ZIP file and saves it to Minio, possibly with metadata tags, and returns a link

REFACTOR:

@marcusbooyah and I discussed this and we'll send this log ZIP file to neon-desktop-service and have that actually manage the uploads to the headend.

One challenge is how we as developers will be able to download these ZIP files. I assume that we'll download these via the Minio Dashboard in headend cluster and that the "link" embedded in the telemetry log will somehow reference this in some way like bucket/blob name or something.

  • [ ] Add a log ZIP API to neon-desktop-service

    I think we'll need to have the apps (neon-cli and neon-desktop) pass a GUID that will be used to name the Minio blob so that we can tag related log events with the same GUID for correlation purposes.

jefflill avatar Sep 10 '22 19:09 jefflill