prowler
prowler copied to clipboard
prowler-cloud
[Bug]: Output issue
Steps to Reproduce
I have found issue that has been close off before (https://github.com/prowler-cloud/prowler/issues/614) and it was mentioning using output command as following in order to remove coloring from cli output: Prowler aws -M text html csv -D myS3bucket
This results to me as issue with "text" argument and just stops scan
Expected behavior
Output in CLI should be only black/white, no colors
Actual Result with Screenshots or Logs
2024-01-31T11:21:51.853406066Z prowler aws: error: argument -M/--output-modes: invalid choice: 'text' (choose from 'csv', 'json', 'json-asff', 'html', 'json-ocsf')
How did you install Prowler?
Cloning the repository from github.com (git clone)
Environment Resource
Kubernetes CronJob on Rancher. This is working perfectly just output issue to cli
OS used
AL2
Prowler version
Prowler 3.11.3
Pip version
pip 23.3.2
Context
I need advice on proper command argument to remove green/red coloring from CLI output. This is needed for allow Loki/Grafana to pick up logs and display them in dashboard but color coding is making a mess of output in grafana.
Hi @Imiler, thank you for reaching us out.
The --only-logs flag may solve your issue since this would print only Prowler logs by the stdout.
Let me know if that works for you.
Thank you,
Unfortunately this is not what we needed. This completely removes anything showing in CLI. They way loki/grafana gets data is that it searched CLI output for results (FAIL, PASS) and from those keywords in query it creates dashboard. This way we have nothing in logs at all, just CSV and HTML exports in file.
Any way to keep same output as it is with normal "Prowler aws" command but for it to be without color.?
This is also link to grafana dashboard, you can see in json on line 210 where it looks for specific word in logs, in this case "EC2". but since in normal output EC2 and results of findings are color coded this just gets out as mess.
That's a good point @Imiler, we will include a flag to remove the colors in the cli output. Any naming that comes to your mind? Thanks!
Hey, thank you for fast response and assistance.
For naming, suggestion i have is --no-color and maybe (-N if its not used already) as its pretty self explanatory.
@Imiler out of curiosity, why aren't the output files (csv, json...) useful to grafana/loki?
It could be useful but they require additional steps to make those files available for grafana to pick them up as in our case we store those to S3 bucket and from what i know Grafana cannot query s3 to get those files. We would need to ingest those to Database or make additional resources to provide those files to Grafana.
In this situation we have dashboard template: [https://grafana.com/grafana/dashboards/19722-aws-prowler-dashboard/] which fetches information from cli output.
But for some reason when running prowler and getting cli output in color and then when Grafana queries it gets messed up. this is example from dashboard: │ ec2 │ \u001B[31mFAIL (97)\u001B[0m │ \u001B[91m0\u001B[0m │ \u001B[31m18\u001B[0m │ \u001B[33m77\u001B[0m │ \u001B[34m2\u001B[0m │"
Why not to get it from the logs? The integration pattern you are following is probably not the best in performance and reliability. Our findings results source of truth are our output files and logs.
I will try to do it that way as well but for now if possible to get just no color output to help with this case.
This dashboard template i am using seems to be focused on catching that first output but could be more underneath to sort out but im unable to even see what results i get from cli since its all messed up
Hi @Imiler currently we are working on a new CLI but to fix your issue now you can use the following command to remove the ANSI color codes: prowler <...> | sed -r "s/\x1B\[([0-9]{1,3}(;[0-9]{1,2};?)?)?[mGK]//g".
Please, feel free to reopen the issue if this doesn't work for you.
Thanks for using Prowler 🚀
