test: combine and send ci coverage report to PR comment

[bowenliang123]

Description

• use coverage to run pytests and cobine the coverage report.
• use python-coverage(https://github.com/marketplace/actions/python-coverage) to send the coverage report to PR's comment, including the overall coverage rate, the coverage of modified or new files. eg. (https://github.com/marketplace/actions/python-coverage#pr-message--job-summary-)

Type of Change

Please delete options that are not relevant.

• [ ] Bug fix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] This change requires a documentation update, included: Dify Document
• [x] Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
• [ ] Dependency upgrade

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• [ ] TODO

Suggested Checklist:

• [x] I have performed a self-review of my own code
• [ ] I have commented my code, particularly in hard-to-understand areas
• [ ] My changes generate no new warnings
• [ ] I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods
• [ ] optional I have made corresponding changes to the documentation
• [ ] optional I have added tests that prove my fix is effective or that my feature works
• [ ] optional New and existing unit tests pass locally with my changes

[bowenliang123]

The error HttpError: Resource not accessible by integration is due to lack of write permission for workflow actions.

Docs for enabling or managing Github workflow permissions: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#configuring-the-default-github_token-permissions

[bowenliang123]

cc @crazywoola

[LeoQuote]

Please consider using codecov instead of inventing wheel again

[bowenliang123]

Please consider using codecov instead of inventing wheel again

This PR is not inventing the wheel again, but generally reused the pytest framework and coverage outputs. And it's not introducing outside dependencies or cost.

codecov, as you mentioned, is an outside platform instead, which could have lead to further constraint or possible fee to the Dify project itself.

If Dify project decide to connect to third-party platform like codecov. I would like to follow.

[LeoQuote]

You deserve better software, the product you choose is raw, what if you want separated coverage report for unit test and integration test? What if you want the coverage not to drop down in every pull request? And coverage is not only about coverage rate, but also which file and which line is and is not covered by tests, the product you choose has no help about it.

[bowenliang123]

Why are you so aggressive in your sords to the goal while ignoring the key points I just mentioned about the balance of third-party platform constraint and the feasibility? Is that codedev the only and the so-called somehow best software for testing coverage in your mind ? It's even far from free or non-profit service itself to the Dify itself as the open-source and business project !

[LeoQuote]

Not interested in arguing about it, if you’re not happy, I’ll mark my comment as off topic.

[bowenliang123]

PLEASE at least respect the Dify project and the Dify community behaviour code. I raised this PR with proper explanation and implementation for mutual benefit. If you have a better solution, PLEASE submit yours. STOP over-criticize other's proposals with no solid approaches or supporting discussions.