Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-14399] Remove unnecessary excludes from POMs and simplify Hive POM #12171

Closed
wants to merge 35 commits into from
Closed

[SPARK-14399] Remove unnecessary excludes from POMs and simplify Hive POM #12171

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
ca2aef5
Update test-dependeices script to not accidentally filter out Spark h…
JoshRosen Apr 4, 2016
87dd382
Remove explicit jodd dependency
JoshRosen Apr 5, 2016
ed40fa3
Remove explicit Datanucleus dependency.
JoshRosen Apr 5, 2016
38ed734
Add Maven Enforcer rule to document org.hsqldb:hsqldb exclusion
JoshRosen Apr 5, 2016
aec2f20
Remove seemingly-unnecessary Calcite exclude.
JoshRosen Apr 5, 2016
02dec1b
Remove explicit Calcite dependency.
JoshRosen Apr 5, 2016
1cf73d4
Remove unused Joda dependency.
JoshRosen Apr 5, 2016
e8409d0
Remove unnecessary Jackson excludes:
JoshRosen Apr 5, 2016
bcd4867
Remove unnecessary Jackson excludes.
JoshRosen Apr 5, 2016
44ae3e0
Remove unnecessary jackson-mapper-asl direct dependency in Hive
JoshRosen Apr 5, 2016
e5c221e
Remove unnecessary httpclient exclusions.
JoshRosen Apr 5, 2016
3deb2c2
Remove unnecessary jsr305 excludes.
JoshRosen Apr 5, 2016
7a24ae8
Clean up commons-codec excludes.
JoshRosen Apr 5, 2016
55e7cc8
Remove commented-out portions of the Hive build.
JoshRosen Apr 5, 2016
482840a
Remove unnecessary Janino exclude.
JoshRosen Apr 5, 2016
9eb49f2
Remove unnecessary libfb303 direct dependency.
JoshRosen Apr 5, 2016
35d539d
Remove a bunch of unnecessary logging-related excludes.
JoshRosen Apr 5, 2016
805f55b
Remove a stray httpclient exclude.
JoshRosen Apr 5, 2016
277e56a
Document org.ow2.asm exclusions.
JoshRosen Apr 5, 2016
4f7c0c5
Clarify comments surrounding Hive's Avro dep. classifier.
JoshRosen Apr 5, 2016
1abd1f0
Remove unnecessary io.netty exclusions; add clarifying comment.
JoshRosen Apr 5, 2016
8eb56cd
Add enforcer rule for excluding javax.servlet:servlet-api
JoshRosen Apr 5, 2016
6b64539
Add enforcer rule for mockito-all exclusion.
JoshRosen Apr 5, 2016
b1809b7
Remove another stray slf4j exclude.
JoshRosen Apr 5, 2016
e5ca86e
More unnecessary exclude removal; reudce scope of Kryo exclude.
JoshRosen Apr 5, 2016
393696c
Remove unnecessary Zookeeper excludes
JoshRosen Apr 5, 2016
17afbe2
Remove unnecessary curator excludes
JoshRosen Apr 5, 2016
d736e7e
Remove inherited exclusion.
JoshRosen Apr 5, 2016
51853c7
Fix typo.
JoshRosen Apr 5, 2016
fc938a0
Roll back a subset of Calcite changes to avoid pentaho dep. resolutio…
JoshRosen Apr 5, 2016
36430de
Pin Janino commons-compiler version.
JoshRosen Apr 5, 2016
0bb11be
Re-add joda-time dependency to fix Hive tests
JoshRosen Apr 5, 2016
c5635f5
Remove redundant enforcer plugin version.
JoshRosen Apr 5, 2016
b7c8c6f
Remove avro.mapred.classifier, since it's now fixed at hadoop2
JoshRosen Apr 5, 2016
f1eb02a
Remove unnecessary Guava excludes.
JoshRosen Apr 5, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 0 additions & 10 deletions core/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,10 +211,6 @@
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty</artifactId>
</dependency>
<dependency>
<groupId>com.clearspring.analytics</groupId>
<artifactId>stream</artifactId>
Expand Down Expand Up @@ -261,12 +257,6 @@
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<exclusions>
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</exclusion>
</exclusions>
<scope>test</scope>
</dependency>
<!-- Added for selenium: -->
Expand Down
8 changes: 7 additions & 1 deletion dev/deps/spark-deps-hadoop-2.2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ commons-beanutils-core-1.8.0.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
commons-compiler-2.7.6.jar
commons-compiler-2.7.8.jar
commons-compress-1.4.1.jar
commons-configuration-1.6.jar
commons-dbcp-1.4.jar
Expand Down Expand Up @@ -78,6 +78,12 @@ hadoop-yarn-client-2.2.0.jar
hadoop-yarn-common-2.2.0.jar
hadoop-yarn-server-common-2.2.0.jar
hadoop-yarn-server-web-proxy-2.2.0.jar
hive-beeline-1.2.1.spark.jar
hive-cli-1.2.1.spark.jar
hive-exec-1.2.1.spark.jar
hive-jdbc-1.2.1.spark.jar
hive-metastore-1.2.1.spark.jar
hive-service-1.2.1.spark.jar
httpclient-4.3.2.jar
httpcore-4.3.2.jar
ivy-2.4.0.jar
Expand Down
8 changes: 7 additions & 1 deletion dev/deps/spark-deps-hadoop-2.3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ commons-beanutils-core-1.8.0.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
commons-compiler-2.7.6.jar
commons-compiler-2.7.8.jar
commons-compress-1.4.1.jar
commons-configuration-1.6.jar
commons-dbcp-1.4.jar
Expand Down Expand Up @@ -73,6 +73,12 @@ hadoop-yarn-client-2.3.0.jar
hadoop-yarn-common-2.3.0.jar
hadoop-yarn-server-common-2.3.0.jar
hadoop-yarn-server-web-proxy-2.3.0.jar
hive-beeline-1.2.1.spark.jar
hive-cli-1.2.1.spark.jar
hive-exec-1.2.1.spark.jar
hive-jdbc-1.2.1.spark.jar
hive-metastore-1.2.1.spark.jar
hive-service-1.2.1.spark.jar
httpclient-4.3.2.jar
httpcore-4.3.2.jar
ivy-2.4.0.jar
Expand Down
8 changes: 7 additions & 1 deletion dev/deps/spark-deps-hadoop-2.4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ commons-beanutils-core-1.8.0.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
commons-compiler-2.7.6.jar
commons-compiler-2.7.8.jar
commons-compress-1.4.1.jar
commons-configuration-1.6.jar
commons-dbcp-1.4.jar
Expand Down Expand Up @@ -73,6 +73,12 @@ hadoop-yarn-client-2.4.0.jar
hadoop-yarn-common-2.4.0.jar
hadoop-yarn-server-common-2.4.0.jar
hadoop-yarn-server-web-proxy-2.4.0.jar
hive-beeline-1.2.1.spark.jar
hive-cli-1.2.1.spark.jar
hive-exec-1.2.1.spark.jar
hive-jdbc-1.2.1.spark.jar
hive-metastore-1.2.1.spark.jar
hive-service-1.2.1.spark.jar
httpclient-4.3.2.jar
httpcore-4.3.2.jar
ivy-2.4.0.jar
Expand Down
8 changes: 7 additions & 1 deletion dev/deps/spark-deps-hadoop-2.6
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ commons-beanutils-core-1.8.0.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
commons-compiler-2.7.6.jar
commons-compiler-2.7.8.jar
commons-compress-1.4.1.jar
commons-configuration-1.6.jar
commons-dbcp-1.4.jar
Expand Down Expand Up @@ -78,6 +78,12 @@ hadoop-yarn-client-2.6.0.jar
hadoop-yarn-common-2.6.0.jar
hadoop-yarn-server-common-2.6.0.jar
hadoop-yarn-server-web-proxy-2.6.0.jar
hive-beeline-1.2.1.spark.jar
hive-cli-1.2.1.spark.jar
hive-exec-1.2.1.spark.jar
hive-jdbc-1.2.1.spark.jar
hive-metastore-1.2.1.spark.jar
hive-service-1.2.1.spark.jar
htrace-core-3.0.4.jar
httpclient-4.3.2.jar
httpcore-4.3.2.jar
Expand Down
8 changes: 7 additions & 1 deletion dev/deps/spark-deps-hadoop-2.7
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ commons-beanutils-core-1.8.0.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
commons-compiler-2.7.6.jar
commons-compiler-2.7.8.jar
commons-compress-1.4.1.jar
commons-configuration-1.6.jar
commons-dbcp-1.4.jar
Expand Down Expand Up @@ -78,6 +78,12 @@ hadoop-yarn-client-2.7.0.jar
hadoop-yarn-common-2.7.0.jar
hadoop-yarn-server-common-2.7.0.jar
hadoop-yarn-server-web-proxy-2.7.0.jar
hive-beeline-1.2.1.spark.jar
hive-cli-1.2.1.spark.jar
hive-exec-1.2.1.spark.jar
hive-jdbc-1.2.1.spark.jar
hive-metastore-1.2.1.spark.jar
hive-service-1.2.1.spark.jar
htrace-core-3.1.0-incubating.jar
httpclient-4.3.2.jar
httpcore-4.3.2.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/test-dependencies.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ for HADOOP_PROFILE in "${HADOOP_PROFILES[@]}"; do
$MVN $HADOOP2_MODULE_PROFILES -P$HADOOP_PROFILE dependency:build-classpath -pl assembly \
| grep "Building Spark Project Assembly" -A 5 \
| tail -n 1 | tr ":" "\n" | rev | cut -d "/" -f 1 | rev | sort \
| grep -v spark > dev/pr-deps/spark-deps-$HADOOP_PROFILE
| grep -v ^spark > dev/pr-deps/spark-deps-$HADOOP_PROFILE
done

if [[ $@ == **replace-manifest** ]]; then
Expand Down
1 change: 0 additions & 1 deletion external/flume-assembly/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,6 @@
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro-mapred</artifactId>
<classifier>${avro.mapred.classifier}</classifier>
<scope>provided</scope>
</dependency>
<dependency>
Expand Down
1 change: 0 additions & 1 deletion external/kafka-assembly/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,6 @@
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro-mapred</artifactId>
<classifier>${avro.mapred.classifier}</classifier>
<scope>provided</scope>
</dependency>
<dependency>
Expand Down
1 change: 0 additions & 1 deletion external/kinesis-asl-assembly/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,6 @@
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro-mapred</artifactId>
<classifier>${avro.mapred.classifier}</classifier>
<scope>provided</scope>
</dependency>
<dependency>
Expand Down
Loading