Update black to the latest version incrementally

[artbataev]

What does this PR do?

Update black using incremental changes only for files changed in PRs.

• the black package is updated to the latest version
• all collections are excluded, so black . will reformat nothing
  • it is possible also to exclude collection from the list, once it is reformatted though PRs
• there will be 2 approaches to reformat the files manually:
  • pass the file(s) explicitly (e.g., black my_file.py)
  • use a pre-commit hook
• the automatic reformatting is done by GitHub Actions only for PRs and only for changed files

Tests (target branch is update_black):

• #9017 - Test from fork (code reformatted, pushed back to the branch from fork)
• #8996 - Test from fork, with updates
• #9019 - Test from branch in the main repo

Approach:

• create a custom GitHub Action responsible for reformatting the code. For black use only changed files
• custom token is used to push changes, since only custom token can trigger workflows after reformatting

Q&A About the Approach

Why do we need a custom GitHub Action if we have pre-commit.ci?

The pre-commit hook is still active and works, but pre-commit.ci reformats all the code (the pre-commit.ci team explicitly denied changing this behavior, see https://github.com/pre-commit-ci/issues/issues/90), thus it is replaced with a custom GitHub Action, which does the same, but only for the changed files (in the PR)

Why do we need isort GitHub action?

Since both isort and black can change files, this should be done in one action to avoid concurrent changes. So, the GitHub Action will execute both formatters and commit changes if needed (same behavior as for pre-commit.ci).

Why do we need a custom GitHub token?

When GitHub action pushes the commit with the default secret token, it cannot trigger other workflows. See the discussion: https://github.com/orgs/community/discussions/25702 This can only be done with custom tokens.

Collection: [CI]

Changelog

• Add specific line by line info of high level changes in this PR.

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this 

Jenkins CI

To run Jenkins, a NeMo User with write access must comment jenkins on the PR.

Before your PR is "Ready for review"

Pre checks:

• [x] Make sure you read and followed Contributor guidelines
• [ ] Did you write any new necessary tests?
• [ ] Did you add or update any necessary documentation?
• [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • [ ] Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• [x] New Feature
• [ ] Bugfix
• [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed. Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

• Related to # (issue)

[github-actions[bot]]

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

[github-actions[bot]]

This PR was closed because it has been inactive for 7 days since being marked as stale.

[artbataev]

@titu1994, @ericharper To make this finally work we need a separate token with minimal permissions:

• Read access to metadata
• Read and Write access to actions and code

The token will have fewer permissions than the default GITHUB_TOKEN, so I do not think that using custom token introduces any new issues.

The reason why we need a custom token is that GITHUB_TOKEN cannot trigger workflows, see https://github.com/orgs/community/discussions/25702 for details.

I tested the solution on the separate repository, and everything works (code reformatted, pushed if needed, after push the workflows are triggered).

[artbataev]

Test changes from fork: #8996