Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0 #32415

Closed
wants to merge 16 commits into from
Closed

[SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0 #32415

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
d9aa654
Reduce code duplication for BLASBenchmark
luhenry Apr 26, 2021
034bef6
Bump to dev.ludovic.netlib:2-SNAPSHOT
luhenry Apr 27, 2021
2dbf90b
Try to make it build on CI
luhenry Apr 27, 2021
6909f0a
Try to make it build on CI (take 2)
luhenry Apr 27, 2021
61b95f7
Try to make it build on CI (take 3)
luhenry Apr 27, 2021
a53a556
Fix naming
luhenry Apr 30, 2021
bc41add
Remove dependency on com.github.fommil.netlib
luhenry May 2, 2021
9dab1f2
Disable caching on CI because of -SNAPSHOT repository
luhenry May 2, 2021
8961a6f
Update docs
luhenry May 3, 2021
46c3241
fixup! Update docs
luhenry May 3, 2021
68b5425
Bump to dev.ludovic.netlib:2.0.0
luhenry May 3, 2021
48acb9f
Correct versions in dev/deps
luhenry May 3, 2021
37ea990
Bring back dependency on com.github.fommil.netlib:all for breeze
luhenry May 5, 2021
9ee1d2d
Merge branch 'master' of github.com:apache/spark
luhenry May 7, 2021
cd9b5c8
Bump to dev.ludovic.netlib:2.1.0
luhenry May 9, 2021
420802e
Bump to dev.ludovic.netlib:2.2.0
luhenry May 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion LICENSE-binary
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -456,7 +456,6 @@ org.antlr:ST4
org.antlr:stringtemplate
org.antlr:antlr4-runtime
antlr:antlr
com.github.fommil.netlib:core
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, this wasn't in the dev/deps files to begin with. Not totally sure why, but, yes then it only needs to come out here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You bring up a good point: com.github.fommil.netlib is still a transient dependency of Spark through org.scalanlp:breeze. I already submitted and merged a PR there to replace the dependency (scalanlp/breeze#811), but it hasn't been released yet.

For the sake of not regressing performance, let me re-add the dependency on com.github.fommil.netlib:all in pom.xml when enabling-Pnetlib-lgpl and leave a comment to explain when we can remove it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, breeze uses it. OK yes let's keep this aspect then.

com.google.re2j:re2j
com.thoughtworks.paranamer:paranamer
org.scala-lang:scala-compiler
Expand Down Expand Up @@ -498,6 +497,9 @@ org.slf4j:jul-to-slf4j
org.slf4j:slf4j-api
org.slf4j:slf4j-log4j12
com.github.scopt:scopt_2.12
dev.ludovic.netlib:blas
dev.ludovic.netlib:arpack
dev.ludovic.netlib:lapack

core/src/main/resources/org/apache/spark/ui/static/dagre-d3.min.js
core/src/main/resources/org/apache/spark/ui/static/*dataTables*
Expand Down
13 changes: 0 additions & 13 deletions mllib-local/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,19 +81,6 @@
<artifactId>blas</artifactId>
</dependency>
</dependencies>
<profiles>
<profile>
<id>netlib-lgpl</id>
<dependencies>
<dependency>
<groupId>com.github.fommil.netlib</groupId>
<artifactId>all</artifactId>
<version>${netlib.java.version}</version>
<type>pom</type>
</dependency>
</dependencies>
</profile>
</profiles>
<build>
<outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory>
<testOutputDirectory>target/scala-${scala.binary.version}/test-classes</testOutputDirectory>
Expand Down
Loading