metaflow icon indicating copy to clipboard operation
metaflow copied to clipboard

feature: delete logs

Open saikonen opened this issue 1 year ago • 2 comments

introduces logs scrub run_id/step/task_id command for scrubbing logs of tasks.

by default it deletes both stdout/stderr for the latest attempt of a task.

Final feature set:

  • logs [pathspec] works as previously, with the new default subcommand custom click group
  • logs show [pathspec] is the default subcommand for logs cmd group
  • logs scrub [pathspec] will overwrite logs with a redacted message

also adds support for both show/scrub commands to specify --attempt [number] to target a specific attempt of a task, instead of the latest.

saikonen avatar Apr 16 '24 11:04 saikonen

@saikonen we could introduce a top-level delete command that has a sub command - logs? wdyt? also, the default behavior could be for --latest-attempt and users can pass in --all-attempts to wipe out logs for all attempts?

savingoyal avatar Apr 16 '24 15:04 savingoyal

this now introduces logs as a command group, with a default fallback to the new logs show command in case no subcommmand is specified.

the following command still works the same

python LongLog.py logs 1713532810109387/start/1
python LongLog.py logs 1713532810109387/start/1 --stderr

but now we can also support the following

python LongLog.py logs delete 1713532810109387/start/1

saikonen avatar Apr 19 '24 13:04 saikonen