[BugFix][CVE] Exclude unused avro-ipc that bundles vulnerable jquery 1.4.2 (backport #76270)#76375
Open
mergify[bot] wants to merge 1 commit into
Open
[BugFix][CVE] Exclude unused avro-ipc that bundles vulnerable jquery 1.4.2 (backport #76270)#76375mergify[bot] wants to merge 1 commit into
mergify[bot] wants to merge 1 commit into
Conversation
…1.4.2 (#76270) ## Why I'm doing: `avro-ipc-1.11.4.jar` ships `jquery-1.4.2.min.js` (CVE-2011-4969, CVE-2014-6071 and six more jQuery XSS CVEs, two of them rated critical by downstream scanners) as static assets of Avro's stats server, which StarRocks never runs. Scanning the whole build output, the only class that references `org.apache.avro.ipc` is avro-mapred's `tether/TetheredProcess` (Avro's tethered-MapReduce debugging feature, also unused), so the jar is dead weight on both sides it ships to: - FE classpath: `spark-core_2.12 -> avro-mapred -> avro-ipc` - BE hive-reader lib: direct `avro-mapred` dependency drags in `avro-ipc` + `avro-ipc-jetty` Upgrading Avro does not help: the latest avro-ipc 1.12.1 still bundles the same jquery 1.4.2. ## What I'm doing: Pom/gradle-only, no Java code changes: 1. **Exclude `avro-ipc`** from spark-core (fe) and **`avro-ipc`/`avro-ipc-jetty`** from avro-mapred (hive-reader). Avro core and avro-mapred stay untouched — `org.apache.avro.mapred.FsInput` etc. remain on the classpath for Hive Avro reads. 2. **Keep `org.tukaani:xz` as an explicit dependency**: it used to reach `fe/lib` only through avro-ipc, while commons-compress, clickhouse-jdbc and 5 other shipped jars still reference it. Without this, the exclusion would silently drop the XZ codec. 3. **Ban `avro-ipc`/`avro-ipc-jetty` in the maven-enforcer gates** (fe + java-extensions root poms) so a future dependency bump cannot re-introduce them. 4. **Mirror everything in the Gradle build** (`fe/build.gradle.kts`, `fe/fe-core/build.gradle.kts`). Verification (full FE rebuild + BE hive-reader repackage): - `fe/lib` is byte-identical minus `avro-ipc-1.11.4.jar` (the explicit xz keeps `xz-1.9.jar` in place); no `jquery-1.4*` asset remains anywhere in `fe/lib`. - FE starts clean, metadata queries work, zero `ClassNotFound`/`NoClassDefFound` in fe.log. - `hive-reader-lib` drops `avro-ipc`/`avro-ipc-jetty`; hive-reader's `TestHiveScanner` (real `.avro` file reads through `AvroSerDe`/`AvroContainerInputFormat`) passes with the exclusion in place. - Both maven reactors pass `validate` with the new enforcer bans active. Signed-off-by: Joob1n <Joob1n@outlook.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> (cherry picked from commit 4f9d892)
23 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why I'm doing:
avro-ipc-1.11.4.jarshipsjquery-1.4.2.min.js(CVE-2011-4969, CVE-2014-6071 and six more jQuery XSS CVEs, two of them rated critical by downstream scanners) as static assets of Avro's stats server, which StarRocks never runs. Scanning the whole build output, the only class that referencesorg.apache.avro.ipcis avro-mapred'stether/TetheredProcess(Avro's tethered-MapReduce debugging feature, also unused), so the jar is dead weight on both sides it ships to:spark-core_2.12 -> avro-mapred -> avro-ipcavro-mapreddependency drags inavro-ipc+avro-ipc-jettyUpgrading Avro does not help: the latest avro-ipc 1.12.1 still bundles the same jquery 1.4.2.
What I'm doing:
Pom/gradle-only, no Java code changes:
avro-ipcfrom spark-core (fe) andavro-ipc/avro-ipc-jettyfrom avro-mapred (hive-reader). Avro core and avro-mapred stay untouched —org.apache.avro.mapred.FsInputetc. remain on the classpath for Hive Avro reads.org.tukaani:xzas an explicit dependency: it used to reachfe/libonly through avro-ipc, while commons-compress, clickhouse-jdbc and 5 other shipped jars still reference it. Without this, the exclusion would silently drop the XZ codec.avro-ipc/avro-ipc-jettyin the maven-enforcer gates (fe + java-extensions root poms) so a future dependency bump cannot re-introduce them.fe/build.gradle.kts,fe/fe-core/build.gradle.kts).Verification (full FE rebuild + BE hive-reader repackage):
fe/libis byte-identical minusavro-ipc-1.11.4.jar(the explicit xz keepsxz-1.9.jarin place); nojquery-1.4*asset remains anywhere infe/lib.ClassNotFound/NoClassDefFoundin fe.log.hive-reader-libdropsavro-ipc/avro-ipc-jetty; hive-reader'sTestHiveScanner(real.avrofile reads throughAvroSerDe/AvroContainerInputFormat) passes with the exclusion in place.validatewith the new enforcer bans active.What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check:
🤖 Generated with Claude Code
This is an automatic backport of pull request #76270 done by Mergify.