Log sprawl improvements
After adding the verbose logging features, this tool can generate ALOT of logs when doing a batch migration.
There a few things we could consider to make the logs a bit less overwhelming and more useful:
- Include the name of the command in the log file name (e.g. timestamp.create-team.octoshift.log)
- If running a generated script that includes many commands, combine all the logs together into 2 files (timestamp.octoshift.log + timestamp.octoshift.verbose.log)
- Put the logs in a logs/timestamp/*.log directory
We may not need to do everything above, but we should at least do some of it.
Should reconsider whether the non-verbose log file should include verbose content when we run with the --verbose flag.
Today the --verbose flag will cause verbose output on the console, and the non-verbose log file matches what is on the console. So you will have identical contents in the non-verbose and verbose log files.
We might want to consider having the non-verbose log file never contain verbose output, then the --verbose flag would only affect what happens on the console.
Another great suggestion from here: https://github.com/github/CS-Release-Readiness/issues/484#issuecomment-1028486312
In the console output include the path to the log files generated.
We could also consider making the timestamps inside the verbose log files more precise (include seconds/milliseconds). this would be helpful when trying to diagnose performance issues.
It would be nice to be able to re-direct the logs to a specified folder.
ado2gh migrate-repo yada yada yada --logPath ".$ado_project\migration\Logs" ado2gh rewire-pipeline yada yada yada --logPath ".$ado_project\pipeline\Logs"
With Regard to the options @dylan-smith lists in the original entry on this issue.
- Yes: Having the command name on the log file name would be very useful.
- Yes: Being able to combine logs into one file would also be great, Although I prefer the folder option I mentioned above.
- No: I don't want a timestamp folder I want to be able to specify my own folder names for the logs. I don't care when it ran I want to know what was running. (Org/Team Project/Repo Name/Command are more important to me)
It would be nice to be able to re-direct the logs to a specified folder.
I have a customer who is also asking for this feature.
Another potential issue with our current logging strategy is if you run multiple CLI commands within the same second they can end up trying to write to the same log file. This may seem unlikely, but what some customers do is have multiple migration scripts that they run in parallel in separate terminal windows. This increases the likelihood that 2 commands from different sessions may run in the same second.
I'm told when this happens that it doesn't crash (which is somewhat surprising to me) but both logs get intermingled in the same file (have not tested this yet).
Some of the suggestions above may be a good solution to this (e.g. an arg to direct the log files to a specific folder, and you would specify a different location for each of the parallel sessions/terminals to eliminate any possibility of conflict). Making the timestamp more granular might also help, but makes the filenames even more cryptic.