Add ability to redirect --debug logs to file

[toddysm]

What is the areas you would like to add the new feature to?

Notation CLI

Is your feature request related to a problem?

I am not able to redirect the --debug output to a file. The following command still prints on the console and does not populate the log file:

notation sign --signature-format cose --key $KEY_NAME --debug docker.io/toddysm/kubeconeu23-python:3.11.2-slim > trace.log

What solution do you propose?

Provide ability to redirect the output to a file

What alternatives have you considered?

None

Any additional context?

No response

[yizha1]

@toddysm in general, standard output (stdout) is used for normal program output, while standard error (stderr) is used for error messages and diagnostic output.

Notation CLI follows the common practice which is to write the verbose or debug log to stderr, so you can use 2> to redirect stderr to a file, for example

notation sign --signature-format cose --key $KEY_NAME --debug docker.io/toddysm/kubeconeu23-python:3.11.2-slim 2> trace.log

You can also redirect both stdout and stderr to different files, for example, > stdout.log 2> stderr.log

So, it is not a bug of Notation CLI, but we do think there is a need to document this for our users, so the next step is to transfer this issue to notatryproject.dev repo and request documentation. What do you think?

[toddysm]

That makes sense but the confusion here is that > debulg.log did not have any content in it when the --debug switch is used. Shouldn't at least some messages go there?

[yizha1]

That makes sense but the confusion here is that > debulg.log did not have any content in it when the --debug switch is used. Shouldn't at least some messages go there?

> debug.log means redirecting the stdout to file debug.log, but there is no content written to stdout with --debug flag, since all the debug logs go to stderr, so we need use 2> debug.log for redirecting stderr to debug.log.

Another way is to redirect stderr to stdout using 2>&1, then redirect stdout to a file, an example notation sign <image> --debug 2>&1 >debug.log.

Does it help?

[toddysm]

No, my question is why all messages go to stderr when debug is on? We should send only error messages to stderr and all other messages to stdout (because they are not errors). The meaning of stderr is to show only error messages (or at least this is my expectation). Does this make sense?

[yizha1]

No, my question is why all messages go to stderr when debug is on? We should send only error messages to stderr and all other messages to stdout (because they are not errors). The meaning of stderr is to show only error messages (or at least this is my expectation). Does this make sense?

@shizhMSFT Could you help here to explain why all messages go to stderr when debug is on?

[shizhMSFT]

According to man stderr,

Under normal circumstances every UNIX program has three streams opened for it when it starts up, one for input, one for output, and one for printing diagnostic or error messages.

Since debug logs are diagnostics, they all go to stderr.

[toddysm]

I just ran curl -v https://www.google.com > verbose.log and I didn't need to redirect the output to 2> to get the output in the file as well as on the console. We can redirect output to multiple outputs.

In comparison, when I ran notation sign --signature-format cose --key $TEST_KEY_NAME ghcr.io/toddysm/net-monitor:kubeconeu-demo-v1 --debug > notation.log the console showed all the output but the notation.log file was empty.

Running the following command notation sign --signature-format cose --key $TEST_KEY_NAME ghcr.io/toddysm/net-monitor:kubeconeu-demo-v1 > notation.log (note it is without --debug) had the same result - notation.log was again empty.

In all the cases we are NOT redirecting the output to a file, which is problematic. The second part is that of convenience and expectations. All other tools can redirect to file with > only and don't need 2>.

Hope this helps clarify where the frustration from the experience comes from.

@sajayantony FYI ^^

[sajayantony]

I think enabling debug output to stderr might be ok but can be argued that --debug is userinput and hence should go to stdout.

But the larger issue I'm seeing here is as a problem is because the verbose logging (which should go to stdout) isn't helping here.

[toddysm]

OK, I did some more investigations and it seems the behavior is consistent for stderr and stderr. I think my user confusion was that for curl I was still getting stdout messages in the case of error response from the server while we are not doing that in Notation. I needed some time to understand how we treat messages and where we send them (which I still believe is not a great user experience).

In essence, my scenario was: "how can I get the detailed output from the console sent to a file that I can send to somebody for troubleshooting". The way to achieve this is to use the --debug option (at the moment, more user-friendly would be to use --verbose as @sajayantony proposes above) and redirect the stderr using 2>.

Now that I know that I think I am fine but it required some brainpower to think which message goes where - which I think can be a hurdle for user experience. I would suggest that we add this to the troubleshooting steps. Also, simplifying the behavior may be good.

For example, are "warnings" errors or standard output. Nevertheless, we can close this unless we want to use for renaming the switch.

[yizha1]

Thanks @toddysm @sajayantony. I think it is clear now for the redirection stderr. I would like to involve more people for discussion on naming, since we haven't released v1 yet. If we know --verbose is the way to go, and --debug will not be used. Should we rename it before v1 release to avoid breaking changes later. @shizhMSFT @FeynmanZhou @vaninrao10 @priteshbandi @iamsamirzon Looking forward for your comments.

[github-actions[bot]]

This issue is stale because it has been opened for 60 days with no activity. Remove stale label or comment. Otherwise, it will be closed in 30 days.