Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-3812] [BUILD] Adapt maven build to publish effective pom. #2673

Closed
wants to merge 4 commits into from
Closed

[SPARK-3812] [BUILD] Adapt maven build to publish effective pom. #2673

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
553d96b
Shaded some unused class of an unused dep, to generate effective pom(s)
ScrapCodes Oct 6, 2014
28f891e
Added a useless dependency, so that we can shade it. And realized fak…
ScrapCodes Oct 21, 2014
0300dac
improved comment messages..
ScrapCodes Oct 21, 2014
aa7b91d
used an unused dep.
ScrapCodes Oct 22, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,17 @@
</snapshots>
</pluginRepository>
</pluginRepositories>

<!--
This is a dummy dependency that is used along with the shading plug-in
to create effective poms on publishing (see SPARK-3812).
-->
<dependencies>
<dependency>
<groupId>org.spark-project.spark</groupId>
<artifactId>unused</artifactId>
<version>1.0.0</version>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
Expand Down Expand Up @@ -992,6 +1002,27 @@
</pluginManagement>

<plugins>
<!-- The shade plug-in is used here to create effective pom's (see SPARK-3812). -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I would add a comment that says:

The shade plug-in is used here to create effective pom's (see SPARK-3812).

<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<shadedArtifactAttached>false</shadedArtifactAttached>
<artifactSet>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to include only the <artifactSet> section and not the <filters> section? Or what about the opposite? If we could make this shorter that would be great.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then the unwanted content of the jar will be put inside all of our spark jars.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if you remove the filters?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: you could publish an empty jar in whatever repo hosts the "org.spark-project" dependencies.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @vanzin - does this jar need to exist at all? We never actually list it as a dependency in any sub project.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If maven-shade-plugin does not think it "shaded" anything, it won't touch the pom and will leave the original one to be published. So you do need to provide an artifactSet with a dependency to shade.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Also note that Prashant added the dependency to all modules by declaring it in the dependencies section, not dependencyManagement.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see - so I should just publish an empty un-used dependency right? A bit clunky but a once time cost.

<includes>
<include>org.spark-project.spark:unused</include>
</includes>
</artifactSet>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
Expand Down