Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-24715][Build] Override jline version as 2.14.3 in SBT #21692

Closed
wants to merge 1 commit into from

Conversation

viirya
Copy link
Member

@viirya viirya commented Jul 2, 2018

What changes were proposed in this pull request?

During SPARK-24418 (Upgrade Scala to 2.11.12 and 2.12.6), we upgrade jline version together. So, mvn works correctly. However, sbt brings old jline library and is hitting NoSuchMethodError in master branch, see #21495 (comment). This overrides jline version in SBT to make sbt build work.

How was this patch tested?

Manually test.

@viirya
Copy link
Member Author

viirya commented Jul 2, 2018

@HyukjinKwon
Copy link
Member

Nice! Mind if I ask to show the diff between sbt dependency-tree vs mvn dependency:tree? Looks fine to me.

@viirya
Copy link
Member Author

viirya commented Jul 2, 2018

Hmm, it seems hard to compare the diff between sbt dependency-tree and mvn dependency:tree. Any suggestion?

@maropu
Copy link
Member

maropu commented Jul 2, 2018

If we don't override this, which jline version does sbt use? 2.12.1? (just a question...)

@viirya
Copy link
Member Author

viirya commented Jul 2, 2018

I've seen jline:jline:0.9.94 in sbt dependency-tree before this change.

@SparkQA
Copy link

SparkQA commented Jul 2, 2018

Test build #92521 has finished for PR 21692 at commit 91ff758.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@maropu
Copy link
Member

maropu commented Jul 2, 2018

Thanks, I feel it'd be better to put concrete version numbers in the description.

@viirya viirya changed the title [SPARK-24715][Build] Override jline version in SBT [SPARK-24715][Build] Override jline version as 2.14.3 in SBT Jul 2, 2018
@viirya
Copy link
Member Author

viirya commented Jul 2, 2018

@maropu Thanks. Version number is in the description now.

@HyukjinKwon
Copy link
Member

Merged to master.

@asfgit asfgit closed this in 8008f9c Jul 2, 2018
@dbtsai
Copy link
Member

dbtsai commented Jul 2, 2018

@viirya thanks for this PR. I thought SBT always uses pom for dependencies, and I wonder why there is a discrepancy so we need to manually override it?

@srowen
Copy link
Member

srowen commented Jul 2, 2018

I haven't looked into this particular issue thoroughly, but, I'm aware that SBT and Maven don't actually resolve dependencies in quite the same way. I think the resolve conflicts with different rules -- most recent wins vs 'nearest' wins. Could be it.

@dongjoon-hyun
Copy link
Member

Thank you all!

@viirya
Copy link
Member Author

viirya commented Jul 3, 2018

Thanks @srowen for explanation. As I can tell, -verbose:class shows jline classes come from jline-2.12.jar, though sbt dependency-tree shows jline:jline:0.9.94. After this overriding, jline classes come from jline-2.14.3.jar.

@viirya viirya deleted the SPARK-24715 branch December 27, 2023 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants