lakeFS
lakeFS copied to clipboard
Progress bars can be *fully* disabled.
In #7732 the animated progress bars one sees when running commands like lakectl local clone
or lakctl fs download
can be partially disabled. I say partially because if you're running these commands and having the logs go to a logging service, like Google Cloud Logging, DataDog, SigNoz, etc. the logs still do not follow a format that works well for those tools.
Specifically, If you pass --no-progress it doesn't display the ###
animations anymore but it does still use \r
and \n
characters to update the output. For example, initially it might display something like:
download vet-video-labels/clljmlh8j0z5q~ ...
download vet-video-labels/clljmnw6f113m~ ...
download vet-video-labels/clljmdchw2ppv~ ...
download vet-video-labels/clljmjdop2qtg~ ...
But then, later, it moved back up a few lines to update the display to:
download vet-video-labels/clljmlh8j0z5q~ ... done!
download vet-video-labels/clljmnw6f113m~ ... done!
download vet-video-labels/clljmdchw2ppv~ ...
download vet-video-labels/clljmjdop2qtg~ ...
Most logging solutions (e.g. Google Cloud Logging, DataDog, SigNoz, etc.) assume a log message is a single line . That is, it expects a string of Unicode characters terminated by a \n
. Once it sees the \n
that's the log message that gets stored. When the progress messages use meta-characters to move back upwards in the terminal to update existing lines those are new log lines and they're very hard for the likes of DataDog, Cloud Logging, etc. to parse.
I'd like to request that there is a way to get logs that don't do any kind of "updating display" - just log downloading X
and then, when the download is complete, log X download complete
(or something like that).