logdata-anomaly-miner
logdata-anomaly-miner copied to clipboard
Replace Deepsource with open source components
Following modules should be included in the CI-Pipeline:
- Bandit(Security-Checks)
- mypy (already included)
- prospector(wrapper for: pylint, pep8, pep257, pyflakes, mccabe, dodgy, pyroma, vulture) -> flake8 > pyflakes, mccabe includes flake8,
- trivy (Security-Checks for the Repository)
- flake8
Whitelisting comments must be replaced with the ones from the new components.
not sure if using wrappers like prospector is a good idea for us. Keeping it modular for the moment.
not sure if using wrappers like prospector is a good idea for us. Keeping it modular for the moment.
what are your doubts?
not sure if using wrappers like prospector is a good idea for us. Keeping it modular for the moment.
what are your doubts?
I got scared after running it locally within a pre-commit hook. Pylint seems to be too pedantic about issues where none exist. If we use prospector (or other wrapper tools), it is not easily possible to just turn off Pylint. However, it is very easy to include the wanted tool separately.
not sure how to run trivy: https://github.com/aquasecurity/trivy There is no apt package for it right? Also no pre-commit hook
also probably overkill to run with every test..
I just want to remind you that this task is assigned to me. As far as I know you can disable pylint with:
pylint:
run: false
And each tool can be configured separately. see https://prospector.landscape.io/en/latest/profiles.html
But you doubts let me thing about the advantages of using prospector other than having one single config-file? Maybe we can really spare it.
Deepsource seems to be running now, but I would still progress in replacing it with open-source tools.
I just want to remind you that this task is assigned to me. As far as I know you can disable pylint with:
pylint: run: falseAnd each tool can be configured separately. see https://prospector.landscape.io/en/latest/profiles.html
sorry, I thought I can do it as it requires removing all of the deepsource comments and creation of stages in the jenkinsfile. I am already almost finished with it..
Please review the changes when I open the PR.
got mypy, bandit and flake8 tests in the Jenkins CI and also extended the pre-commit hook, notably with PEP257 checks with docformatter. There are also quite some changes to the format of docstrings..