Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove ammonite imports support #1787

Merged
merged 3 commits into from
Jan 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 3 additions & 16 deletions INTERNALS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ The other modules are either:
These are:
- `bloop-rifle`: starts a Bloop server if needed, connects to it via nailgun, opens a BSP server to it, …
- `runner`: simple app that starts a main class, catches any exception it throws and pretty-prints it.
- `stubs`: empty classes, so that lines such as `import $ivy.$`, left after full blown `import $ivy`s are processed, compile fine (about to be removed?)
- `test-runner`: finds test frameworks, test suites, and runs them
- `tasty-lib`: edits file names in `.tasty` files

Expand Down Expand Up @@ -56,29 +55,17 @@ We roughly go from user inputs to byte code through 3 classes:
Most commands
- take the arguments passed on the command-line: we have an `Array[String]`
- check whether each of them is a `.scala` file, an `.sc` file, a directory, …: we get an `Inputs` instance
- reads the directories, the `.scala` / `.sc` files, processes `import $ivy` in them: we get a `Sources` instance
- reads the directories, the `.scala` / `.sc` files: we get a `Sources` instance
- compile those sources: we get a `Build` instance
- do something with the build output (run it, run tests, package it, …)

In watch mode, we loop over the last 3 steps (`Inputs` is computed only once, the rest is re-computed upon file change).

## Source pre-processing

Some input files cannot be passed as is to scalac, because:
- they contain `import $ivy`s
- they are scripts (`.sc` files), which contain top-level statements
Some input files cannot be passed as is to scalac, if they are scripts (`.sc` files), which contain top-level statements

The `import $ivy` gets replaced like
```scala
import $ivy.`org:name:ver`, something.else._
```
becomes
```scala
import $ivy.$ , something.else._
```
(We just do the same as Ammonite.)

Scripts gets wrapped. If the script `a/b/foo.sc` contains
Scripts get wrapped. If the script `a/b/foo.sc` contains
```scala
val n = 2
```
Expand Down
18 changes: 2 additions & 16 deletions build.sc
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,6 @@ object `scala3-graal` extends Cross[Scala3Graal](Scala.mainVersions: _*)
// Main app used to process classpath within build itself
object `scala3-graal-processor` extends Scala3GraalProcessor

object stubs extends JavaModule with ScalaCliPublishModule {
def javacOptions = T {
super.javacOptions() ++ Seq("-target", "8", "-source", "8")
}
}
object `scala-cli-bsp` extends JavaModule with ScalaCliPublishModule {
def ivyDeps = super.ivyDeps() ++ Seq(
Deps.bsp4j
Expand Down Expand Up @@ -367,10 +362,6 @@ trait Core extends ScalaCliSbtModule with ScalaCliPublishModule with HasTests
|
| def scalaJsCliVersion = "${InternalDeps.Versions.scalaJsCli}"
|
| def stubsOrganization = "${stubs.pomSettings().organization}"
| def stubsModuleName = "${stubs.artifactName()}"
| def stubsVersion = "${stubs.publishVersion()}"
|
| def testRunnerOrganization = "$testRunnerOrganization"
| def testRunnerModuleName = "${`test-runner`(Scala.runnerScala3).artifactName()}"
| def testRunnerVersion = "${`test-runner`(Scala.runnerScala3).publishVersion()}"
Expand Down Expand Up @@ -1171,16 +1162,11 @@ object `local-repo` extends LocalRepo {
*/
def developingOnStubModules = false

def stubsModules = {
val javaModules = Seq(
stubs
)
val crossModules = for {
def stubsModules =
for {
sv <- Scala.runnerScalaVersions
proj <- Seq(runner, `test-runner`)
} yield proj(sv)
javaModules ++ crossModules
}
def version = runner(Scala.runnerScala3).publishVersion()
}

Expand Down
2 changes: 1 addition & 1 deletion examples/junit/MyTests.scala
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import $ivy.`com.github.sbt:junit-interface:0.13.2`
//> using lib "com.github.sbt:junit-interface:0.13.2"
import org.junit.Test

class MyTests {
Expand Down
2 changes: 1 addition & 1 deletion examples/scala-files/simple.scala
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import $ivy.`com.lihaoyi::pprint:0.8.1`
//> using lib "com.lihaoyi::pprint:0.8.1"

object Test {
def something[F[_]] = ()
Expand Down
2 changes: 1 addition & 1 deletion examples/tests/MyTests.scala
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import $ivy.`org.scalameta::munit::0.7.29`
//> using lib "org.scalameta::munit::0.7.29"

class MyTests extends munit.FunSuite {
test("foo") {
Expand Down
2 changes: 1 addition & 1 deletion gifs/create_missing.sc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env scala-cli

import $ivy.`com.lihaoyi::os-lib:0.7.8`
//> using lib "com.lihaoyi::os-lib:0.7.8"

/** Small and handy script to generate stubs for .svg files with nice TODO
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -234,23 +234,20 @@ case object ScalaPreprocessor extends Preprocessor {
allowRestrictedFeatures
))

val afterProcessImports: Option[SpecialImportsProcessingOutput] = value {
processSpecialImports(
value {
checkForAmmoniteImports(
afterStrictUsing.strippedContent.getOrElse(content0),
path
)
}

if (afterStrictUsing.isEmpty && afterProcessImports.isEmpty) None
if (afterStrictUsing.isEmpty) None
else {
val allRequirements = afterProcessImports.map(_.reqs).toSeq :+ afterStrictUsing.globalReqs
val allRequirements = Seq(afterStrictUsing.globalReqs)
val summedRequirements = allRequirements.foldLeft(BuildRequirements())(_ orElse _)
val allOptions = afterStrictUsing.globalUsings +:
afterProcessImports.map(_.opts).toSeq
val summedOptions = allOptions.foldLeft(BuildOptions())(_ orElse _)
val lastContentOpt = afterProcessImports
.map(_.content)
.orElse(afterStrictUsing.strippedContent)
val allOptions = Seq(afterStrictUsing.globalUsings)
val summedOptions = allOptions.foldLeft(BuildOptions())(_ orElse _)
val lastContentOpt = afterStrictUsing.strippedContent
.orElse(if (isSheBang) Some(content0) else None)
val directivesPositions = afterStrictUsing.directivesPositions

Expand All @@ -265,10 +262,10 @@ case object ScalaPreprocessor extends Preprocessor {
}
}

private def processSpecialImports(
private def checkForAmmoniteImports(
content: String,
path: Either[String, os.Path]
): Either[BuildException, Option[SpecialImportsProcessingOutput]] = either {
): Either[BuildException, Unit] = {

import fastparse.*

Expand Down Expand Up @@ -304,51 +301,29 @@ case object ScalaPreprocessor extends Preprocessor {
val code = content.substring(start, end) // .trim // meh
val importRes = parse(code, ImportSplitter(_))
importRes.fold((_, _, _) => Iterator.empty, (trees, _) => trees.iterator).map { tree =>
tree.copy(start = start + tree.start, end = start + tree.end)
(start, tree.copy(start = start + tree.start, end = start + tree.end))
}
}
.toVector

val dependencyTrees = importTrees.filter { t =>
val dependencyTrees = importTrees.filter { case (_, t) =>
val firstSegmentOpt = t.prefix.headOption
(firstSegmentOpt.contains("$ivy") || firstSegmentOpt.contains("$dep")) &&
t.prefix.lengthCompare(1) > 0
}

if (dependencyTrees.isEmpty) None
else {
// replace statements like
// import $ivy.`foo`,
// by
// import $ivy.A ,
// Ideally, we should just wipe those statements, and take care of keeping 'import' and ','
// for standard imports.
val buf = content.toCharArray
for (t <- dependencyTrees) {
val substitute = (t.prefix.head + ".A").padTo(t.end - t.start, ' ')
assert(substitute.length == (t.end - t.start))
System.arraycopy(substitute.toArray, 0, buf, t.start, substitute.length)
}
val newCode = new String(buf)
if (dependencyTrees.nonEmpty) {
val toFilePos = Position.Raw.filePos(path, content)
val deps = value {
dependencyTrees
.map { t => /// skip ivy ($ivy.`) or dep syntax ($dep.`)
val pos = toFilePos(Position.Raw(t.start + "$ivy.`".length, t.end - 1))
val strDep = t.prefix.drop(1).mkString(".")
val maybeDep = parseDependency(strDep, pos)
maybeDep.map(dep => Positioned(Seq(pos), dep))
}
.sequence
.left.map(CompositeBuildException(_))
}
val options = BuildOptions(
classPathOptions = ClassPathOptions(
extraDependencies = ShadowingSeq.from(deps)
)
)
Some(SpecialImportsProcessingOutput(BuildRequirements(), options, newCode))
val exceptions = for {
(importStart, t) <- dependencyTrees
pos = toFilePos(Position.Raw(importStart, t.end))
dep = t.prefix.drop(1).mkString(".")
newImportText = s"//> using lib \"$dep\""
} yield new UnsupportedAmmoniteImportError(Seq(pos), newImportText)

Left(CompositeBuildException(exceptions))
}
else Right(())
}

private def processStrictUsing(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
package scala.build.tests

import com.eed3si9n.expecty.Expecty.expect

import scala.build.options.{BuildOptions, InternalOptions}
import scala.build.Ops._
import scala.build.Ops.*
import scala.build.{BuildThreads, Directories, LocalRepo}
import scala.build.actionable.ActionablePreprocessor
import scala.build.actionable.ActionableDiagnostic._
import scala.build.actionable.ActionableDiagnostic.*
import scala.build.Position.File
import coursier.core.Version

import scala.build.errors.{BuildException, CompositeBuildException, UnsupportedAmmoniteImportError}

class ActionableDiagnosticTests extends munit.FunSuite {

val extraRepoTmpDir = os.temp.dir(prefix = "scala-cli-tests-actionable-diagnostic-")
Expand Down Expand Up @@ -47,11 +51,15 @@ class ActionableDiagnosticTests extends munit.FunSuite {
}
}

test("update ivy dependence upickle") {
val dependencyOsLib = "com.lihaoyi::upickle:1.4.0"
test("error on ammonite imports") {
val dependencyOsLib = "com.lihaoyi::os-lib:0.7.8"
val dependencyUpickleLib = "com.lihaoyi::upickle:1.4.0"
val ivyImport = s"import $$ivy.`$dependencyOsLib`"
val depImport = s"import $$dep.`$dependencyUpickleLib`"
val testInputs = TestInputs(
os.rel / "Foo.scala" ->
s"""import $$ivy.`$dependencyOsLib`
s"""$ivyImport
|$depImport
|
|object Hello extends App {
| println("Hello")
Expand All @@ -60,46 +68,26 @@ class ActionableDiagnosticTests extends munit.FunSuite {
)
testInputs.withBuild(baseOptions, buildThreads, None, actionableDiagnostics = true) {
(_, _, maybeBuild) =>
val build = maybeBuild.orThrow
val updateDiagnostics =
ActionablePreprocessor.generateActionableDiagnostics(build.options).orThrow

val osLibDiagnosticOpt = updateDiagnostics.collectFirst {
case diagnostic: ActionableDependencyUpdateDiagnostic => diagnostic
expect(maybeBuild.isLeft)
val exceptions = maybeBuild match {
case Left(c: CompositeBuildException) => c.exceptions
case _ => Seq()
}

expect(osLibDiagnosticOpt.nonEmpty)
val osLibDiagnostic = osLibDiagnosticOpt.get

expect(Version(osLibDiagnostic.newVersion) > Version(osLibDiagnostic.currentVersion))
}
}

test("update dep dependence upickle") {
val dependencyOsLib = "com.lihaoyi::upickle:1.4.0"
val testInputs = TestInputs(
os.rel / "Foo.scala" ->
s"""import $$dep.`$dependencyOsLib`
|
|object Hello extends App {
| println("Hello")
|}
|""".stripMargin
)
testInputs.withBuild(baseOptions, buildThreads, None, actionableDiagnostics = true) {
(_, _, maybeBuild) =>
val build = maybeBuild.orThrow
val updateDiagnostics =
ActionablePreprocessor.generateActionableDiagnostics(build.options).orThrow
expect(exceptions.length == 2)
expect(exceptions.forall(_.isInstanceOf[UnsupportedAmmoniteImportError]))

val osLibDiagnosticOpt = updateDiagnostics.collectFirst {
case diagnostic: ActionableDependencyUpdateDiagnostic => diagnostic
}
expect(exceptions.head.textEdit.get.newText == s"//> using lib \"$dependencyOsLib\"")
expect(
exceptions.tail.head.textEdit.get.newText == s"//> using lib \"$dependencyUpickleLib\""
)

expect(osLibDiagnosticOpt.nonEmpty)
val osLibDiagnostic = osLibDiagnosticOpt.get
val filePositions = exceptions.flatMap(_.positions.collect {
case File(_, startPos, endPos) => (startPos, endPos)
})

expect(Version(osLibDiagnostic.newVersion) > Version(osLibDiagnostic.currentVersion))
expect(filePositions.head == ((0, 0), (0, ivyImport.length)))
expect(filePositions.tail.head == ((1, 0), (1, depImport.length)))
}
}

Expand Down
Loading