apache
Ambari-26147: Add Ruff integration to ambari
What changes were proposed in this pull request?
The entire Ambari project has accumulated a large amount of Python code, which is the result of the collective efforts of multiple contributors to make Ambari better.
| Language | Files | Lines | Code | Comments | Blanks |
|---|---|---|---|---|---|
| Java | 3330 | 690679 | 427116 | 156389 | 107174 |
| JavaScript | 2125 | 512742 | 374238 | 87081 | 51423 |
| Plain Text | 68 | 500082 | 0 | 497409 | 2673 |
| JSON | 758 | 479988 | 479689 | 0 | 299 |
| Python | 1096 | 218057 | 175499 | 9547 | 33011 |
| XML | 1135 | 156581 | 130300 | 20189 | 6092 |
During this process, a problem has emerged: everyone has different coding styles, which leads to inconsistencies in the submitted code. This can result in various inconveniences, such as increased understanding costs of the underlying principles and greater difficulty in identifying the root causes of issues. As the codebase grows, these problems become increasingly apparent. Therefore, we are introducing ruff, along with some additional processes, to format the code and alleviate these issues.
What is Ruff?
An extremely fast Python linter and code formatter, written in Rust. Its speed is one of its advantages. More information can be found here: ruff. (Please fill in changes proposed in this fix)
Ruff rules are derived from the Apache Superset project's rule file.
Ruff integration plan:
- Project integration with Ruff, formatting all Python files, ensuring all tests pass, and the cluster runs normally. Completed.
- Integration of Ruff detection into Jenkins. Completed.
Notice: This PR's Ruff rules ignore some rule checks because these rules involve significant changes that will be addressed in subsequent PRs. The current PR ensures that all Python indentation and basic formatting conform to standards.
Ruff Usage Instructions in Ambari Code Submission
Install Ruff:
pip install ruff
Format Code with Ruff:
ruff format .
Check Code Format Manually:
ruff check .
Check Code Format with Maven:
mvn exec:exec@ruff-check -Pruff-check
Fix Code Format with Maven:
mvn exec:exec@ruff-format -Pruff-format
How was this patch tested?
manual test CI/CD After formatting thousands of files with Ruff, I deployed a simple cluster to verify that Ambari runs smoothly.
(Please explain how this patch was tested. Ex: unit tests, manual tests) (If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)
Please review Ambari Contributing Guide before opening a pull request.
Wow, this is huge and yet it seems worth having. Skimmed through at high level, looks good. I wonder if anyone else is also interested to review this, otherwise let's get this in and see how things go.
Nice work!
Hi @JiaLiangC, Which environment are you deploying the cluster for testing? It seems you're having success with the setup. I recently tried to build and deploy an Ambari cluster on Rocky Linux 8 and encountered two blocking issues: AMBARI-26183 (Fixed) AMBARI-26187 (Still open)
@arshadmohammad Good catch! Your deployment failed because of this PR: https://github.com/apache/ambari/pull/3770, which introduced numerous bugs in the frontend. Although the subsequent PR https://github.com/apache/ambari/pull/3846 fixed many issues, around 60 of them, there are still many bugs remaining. One of them is the blocking issue you encountered, which is a bug caused by the frontend upgrade. I tested using the code from before this PR was merged: AMBARI-25289 Upgrade Jquery and Bootstrap to latest versions #3770.
Hi @JiaLiangC, Which environment are you deploying the cluster for testing? It seems you're having success with the setup. I recently tried to build and deploy an Ambari cluster on Rocky Linux 8 and encountered two blocking issues: AMBARI-26183 (Fixed) AMBARI-26187 (Still open)
@arshadmohammad Already resolved by this pr https://github.com/apache/ambari/pull/3849.
@arshadmohammad I have added the usage of Ruff involved in this PR to the description. Additionally, all code changes in this PR are the result of ruff format.
Hi @JiaLiangC , many changes include indentation / spacing between the lines, the current codebase uses two spaces for indentation / spacing instead of the typical four spaces / tabs. Is it possible to configure ruff to consider two spaces as the indentation and spacing size as it is the standard followed. Doing so may help reduce the size of the change as well and have clear diff for the actual changes.
@vishalsuvagia Thank you for the reminder. I've changed the indent-width to 2 and committed it. This indeed makes it easier to review. However, there are still a massive 900 files, which I believe makes manual review very difficult. We can only rely on unit tests and subsequent bug reports after usage. Currently, the unit tests have passed, and I've deployed a complete cluster in my environment without finding any issues so far. Theoretically, code formatting tools like ruff shouldn't change the code semantics.