incubator-uniffle
incubator-uniffle copied to clipboard
Make spark3 as activation profile
What changes were proposed in this pull request?
Make spark3 as activation profile
Why are the changes needed?
Idea can not resolved dependencies if there are no activation profile
Cannot resolve org.apache.spark:spark-core_${scala.binary.version}:${spark.version}
Does this PR introduce any user-facing change?
No
How was this patch tested?
test locally
Actually we can manually set active profiles in IDEA like this,
So this change is not necessary, and the solution is limited to spark3 profile.
Codecov Report
Merging #39 (64eb319) into master (8256765) will increase coverage by
0.54%. The diff coverage isn/a.
@@ Coverage Diff @@
## master #39 +/- ##
============================================
+ Coverage 56.29% 56.83% +0.54%
- Complexity 1168 1204 +36
============================================
Files 152 152
Lines 8401 8401
Branches 813 813
============================================
+ Hits 4729 4775 +46
+ Misses 3408 3368 -40
+ Partials 264 258 -6
| Impacted Files | Coverage Δ | |
|---|---|---|
| ...a/org/apache/uniffle/common/RemoteStorageInfo.java | 94.44% <0.00%> (+31.48%) |
:arrow_up: |
| .../java/org/apache/uniffle/common/BufferSegment.java | 100.00% <0.00%> (+44.00%) |
:arrow_up: |
| ...java/org/apache/uniffle/common/PartitionRange.java | 100.00% <0.00%> (+100.00%) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update 8256765...64eb319. Read the comment docs.
I'm neutral for activeByDefault, but recommend to unify the profile style, e.g.
spark-2.3
spark-2.4
spark-3.0
spark-3.1
spark-3.2
I'm neutral for
activeByDefault, but recommend to unify the profile style, e.g.spark-2.3 spark-2.4 spark-3.0 spark-3.1 spark-3.2
Agreed.
yes, the profile name should have same style
I'm neutral for
activeByDefault, but recommend to unify the profile style, e.g.spark-2.3 spark-2.4 spark-3.0 spark-3.1 spark-3.2
It's good to unify the style. But I want to have a default profile for Spark2 and Spark3, the default profile is the version which we mainly support. So we make the some profile names special. Could you give some suggestion to solve this problem?
... the default profile is the version which we mainly support.
With the community growing, other versions of Spark may be adopted by users in large-scale production, how about pointing the version on the use case sharing page(something like Who is using Uniffle)?
... the default profile is the version which we mainly support.
With the community growing, other versions of Spark may be adopted by users in large-scale production, how about pointing the version on the use case sharing page(something like Who is using Uniffle)?
Good idea.