tiup
tiup copied to clipboard
pingcap
Support specifying TiSpark version in TiUP cluster
What problem does this PR solve?
Support specifying tispark version. This pr should not be merged before upload spark & tispark to mirror
What is changed and how it works?
- Support specifying TiSpark version
- Add spark_version, tispark_version and scala_version in TiSparkMasterSpec
- valid version: spark_version and tispark_version should be set together or not be set together
- Transfer spark_version, tispark_version, and scala_version to TiSparkInstance(both master and worker)
- Get version info and download spark & tispark
- Choose the TiSpark jar to deploy
- Get version info from mergedtopo when scale-out
- Generate different configs for different tispark_version
- Skip TiSpark when upgrade
Check List
Tests
Manual test (add detailed scripts or steps below)
- spark_version 3.0.3 tispark version 3.0.0 scala version 2.12 (success)
- spark_version 3.0.3 tispark version 3.0.0 (success)
- spark_version 2.4.3 tispark version 2.4.1 (success)
- no paratemer (success)
- spark_version 3.0.3 tispark version 3.0.0 scala version 2.11 (fail because no jar found)
- spark_version 3.0.3 (fail because tispark version not set)
Code changes
- Has exported function/method change
- Has exported variable/fields change
- Has interface methods change
Related changes
- Need to update the documentation
Release notes:
NONE
[REVIEW NOTIFICATION]
This pull request has not been approved.
To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.
The full list of commands accepted by this bot can be found here.
Reviewer can indicate their review by submitting an approval review. Reviewer can cancel approval by submitting a request changes review.
Codecov Report
Patch coverage: 86.14% and project coverage change: -0.05% :warning:
Comparison is base (
8117e9a) 52.56% compared to head (2f6c743) 52.51%. Report is 136 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #1965 +/- ##
==========================================
- Coverage 52.56% 52.51% -0.05%
==========================================
Files 307 307
Lines 35482 35596 +114
==========================================
+ Hits 18648 18691 +43
- Misses 14641 14705 +64
- Partials 2193 2200 +7
| Flag | Coverage Δ | |
|---|---|---|
| cluster | 41.23% <83.17%> (+0.08%) |
:arrow_up: |
| dm | 24.35% <0.99%> (-0.12%) |
:arrow_down: |
| playground | 12.93% <0.00%> (-0.04%) |
:arrow_down: |
| tiup | 14.42% <0.00%> (-0.03%) |
:arrow_down: |
| unittest | 21.01% <9.09%> (-0.01%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Files Changed | Coverage Δ | |
|---|---|---|
| pkg/cluster/manager/upgrade.go | 63.07% <ø> (+3.07%) |
:arrow_up: |
| pkg/cluster/template/systemd/tispark.go | 65.71% <ø> (ø) |
|
| pkg/cluster/template/config/tispark.go | 63.41% <72.73%> (+2.12%) |
:arrow_up: |
| pkg/cluster/spec/bindversion.go | 84.62% <75.00%> (-6.29%) |
:arrow_down: |
| pkg/cluster/manager/deploy.go | 68.71% <77.78%> (+0.11%) |
:arrow_up: |
| pkg/cluster/task/builder.go | 99.19% <83.33%> (-0.81%) |
:arrow_down: |
| pkg/cluster/spec/tispark.go | 67.12% <84.00%> (+1.32%) |
:arrow_up: |
| pkg/cluster/manager/builder.go | 68.50% <96.15%> (-4.59%) |
:arrow_down: |
| pkg/cluster/spec/validate.go | 93.01% <100.00%> (+0.04%) |
:arrow_up: |
| pkg/utils/semver.go | 84.71% <100.00%> (+0.27%) |
:arrow_up: |
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
@shiyuhang0: PR needs rebase.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.