starrocks
starrocks copied to clipboard
[Feature]Use jprotobuf-precompile-plugin for starrocks
Why I'm doing:
The jprotobuf code generation plug-in used by StarRocks provides a pre-compiled alternative, eliminating the need for runtime code generation via the JDK’s compilation API. This improvement enhances the efficiency of RPC information transmission.
What I'm doing:
Add the precompile plugin to the pom file and modify the current compilation check.
What type of PR is this:
- [ ] BugFix
- [x] Feature
- [ ] Enhancement
- [ ] Refactor
- [ ] UT
- [ ] Doc
- [ ] Tool
Does this PR entail a change in behavior?
- [x] Yes, this PR will result in a change in behavior.
- [ ] No, this PR will not result in a change in behavior.
If yes, please specify the type of change:
- [ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
- [ ] Parameter changes: default values, similar parameters but with different default values
- [x] Policy changes: use new policy to replace old one, functionality automatically enabled
- [ ] Feature removed
- [ ] Miscellaneous: upgrade & downgrade compatibility, etc.
Checklist:
- [ ] I have added test cases for my bug fix or my new feature
- [ ] This pr needs user documentation (for new or modified features or behaviors)
- [ ] I have added documentation for my new feature or new function
- [x] This is a backport pr
Bugfix cherry-pick branch check:
- [ ] I have checked the version labels which the pr will be auto-backported to the target branch
- [ ] 3.3
- [ ] 3.2
- [ ] 3.1
- [ ] 3.0
- [ ] 2.5
Please open a github issue to associate to this
[Feature]development.
Sorry, I accidentally closed this PR. It is now associated with the issue #50684.
introduces new CVE
==========
Total: 5 (HIGH: 3, CRITICAL: 2)
┌────────────────────────────────────────────────────────────┬──────────────────┬──────────┬────────┬───────────────────┬────────────────────────────────┬─────────────────────────────────────────────────────────────┐
│ Library │ Vulnerability │ Severity │ Status │ Installed Version │ Fixed Version │ Title │
├────────────────────────────────────────────────────────────┼──────────────────┼──────────┼────────┼───────────────────┼────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ com.google.protobuf:protobuf-java │ CVE-2022-3509 │ HIGH │ fixed │ 3.17.3 │ 3.16.3, 3.19.6, 3.20.3, 3.21.7 │ protobuf-java: Textformat parsing issue leads to DoS │
│ (orc-mapreduce-1.8.7-shaded-protobuf.jar) │ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2022-3509 │
│ ├──────────────────┤ │ │ │ ├─────────────────────────────────────────────────────────────┤
│ │ CVE-2022-3510 │ │ │ │ │ protobuf-java: Message-Type Extensions parsing issue leads │
│ │ │ │ │ │ │ to DoS │
│ │ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2022-3510 │
├────────────────────────────────────────────────────────────┼──────────────────┼──────────┤ ├───────────────────┼────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ org.apache.maven:maven-core (maven-core-2.0.9.jar) │ CVE-2021-26291 │ CRITICAL │ │ 2.0.9 │ 3.8.1 │ maven: Block repositories using http by default │
│ │ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2021-26291 │
├────────────────────────────────────────────────────────────┼──────────────────┤ │ ├───────────────────┼────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ org.codehaus.plexus:plexus-utils (plexus-utils-1.5.15.jar) │ CVE-2017-1000487 │ │ │ 1.5.15 │ 3.0.16 │ plexus-utils: Mishandled strings in Commandline class allow │
│ │ │ │ │ │ │ for command injection │
│ │ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2017-1000487 │
│ ├──────────────────┼──────────┤ │ ├────────────────────────────────┼─────────────────────────────────────────────────────────────┤
│ │ CVE-2022-4244 │ HIGH │ │ │ 3.0.24 │ codehaus-plexus: Directory Traversal │
│ │ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2022-4244 │
└────────────────────────────────────────────────────────────┴──────────────────┴──────────┴────────┴───────────────────┴────────────────────────────────┴─────────────────────────────────────────────────────────────┘
the PR is great, need to fix the introduced CVEs.
The CVE issues have already been resolved.
@mergify rebase
rebase
✅ Branch has been successfully rebased
@mergifyio rebase
rebase
✅ Branch has been successfully rebased
Quality Gate passed
Issues
0 New issues
0 Accepted issues
Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code
[Java-Extensions Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)
[FE Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)
[BE Incremental Coverage Report]
:white_check_mark: pass : 0 / 0 (0%)
@Lrabbit125 this plugin requires some packages can not be in scope of provided. If I want to like comment hudi-coommon or paimon-bundle to provided scope, how can I do that?
@Lrabbit125 BTW, can you elaborate what this plugin is for? or share me some links about principles of this enhancement?
And if I mark paimon-bundle as scope provided, it fails like
java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:568)
at com.baidu.jprotobuf.mojo.PreCompileMojo$1.run (PreCompileMojo.java:326)
at java.lang.Thread.run (Thread.java:842)
Caused by: jodd.io.findfile.FindFileException: Scan entry error: EntryData{com.starrocks.planner.PaimonScanNode'}; <--- java.lang.ClassNotFoundException: org.apache.paimon.table.source.Split
at jodd.io.findfile.ClassFinder.scanEntry (ClassFinder.java:377)
at jodd.io.findfile.ClassFinder.scanClassFile (ClassFinder.java:324)
at jodd.io.findfile.ClassFinder.scanClassPath (ClassFinder.java:307)
at jodd.io.findfile.ClassFinder.scanPath (ClassFinder.java:239)
at jodd.io.findfile.ClassFinder.scanPaths (ClassFinder.java:178)
at jodd.io.findfile.ClassScanner.scan (ClassScanner.java:35)
at jodd.io.findfile.ClassScanner.scanDefaultClasspath (ClassScanner.java:28)
at com.baidu.jprotobuf.mojo.JprotobufPreCompileMain.main (JprotobufPreCompileMain.java:182)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:568)
at com.baidu.jprotobuf.mojo.PreCompileMojo$1.run (PreCompileMojo.java:326)
at java.lang.Thread.run (Thread.java:842)
Caused by: java.lang.Exception: org/apache/paimon/table/source/Split
at com.baidu.jprotobuf.mojo.JprotobufPreCompileMain$1.onEntry (JprotobufPreCompileMain.java:177)
at jodd.io.findfile.ClassFinder.scanEntry (ClassFinder.java:375)
at jodd.io.findfile.ClassFinder.scanClassFile (ClassFinder.java:324)
at jodd.io.findfile.ClassFinder.scanClassPath (ClassFinder.java:307)
at jodd.io.findfile.ClassFinder.scanPath (ClassFinder.java:239)
at jodd.io.findfile.ClassFinder.scanPaths (ClassFinder.java:178)
at jodd.io.findfile.ClassScanner.scan (ClassScanner.java:35)
at jodd.io.findfile.ClassScanner.scanDefaultClasspath (ClassScanner.java:28)
at com.baidu.jprotobuf.mojo.JprotobufPreCompileMain.main (JprotobufPreCompileMain.java:182)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:77)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:568)
at com.baidu.jprotobuf.mojo.PreCompileMojo$1.run (PreCompileMojo.java:326)
at java.lang.Thread.run (Thread.java:842)
@Lrabbit125 If I understand correctly, the main purpose of this pr is to let starrocks run in JRE mode (instead of JDK mode).
And I try my fix https://github.com/StarRocks/starrocks/pull/56732 on JRE17(zulu), and it works.