Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Macro expansion regression (Java 11) #9769

Closed
michelou opened this issue Sep 10, 2020 · 10 comments · Fixed by #12847
Closed

Macro expansion regression (Java 11) #9769

michelou opened this issue Sep 10, 2020 · 10 comments · Fixed by #12847

Comments

@michelou
Copy link
Contributor

Minimized code

object Main {
  val lifeOfPi = 3.14159
  val fInterpolator = f"The approximate value of pi is $lifeOfPi%4.2f"
  
  def main(args: Array[String]): Unit = {
    println(fInterpolator)
  }

}

Output (click arrow to expand)

Release 0.27.0-RC1 compiles (and runs) fine:

$ dotc -version
Dotty compiler version 0.27.0-RC1 -- Copyright 2002-2020, LAMP/EPFL

$ dotc -deprecation -feature -classpath "W:\examples\Interpolators\target\classes" -d "W:\examples\Interpolators\target\classes" W:\examples\Interpolators\src\main\scala\Main.scala

$ dotr -classpath W:\examples\Interpolators\target\classes Main
The approximate value of pi is 3,14

Nightly build 0.28.0-bin-20200908-ce48f5a-NIGHTLY crashes:

$ dotc -version
Dotty compiler version 0.28.0-bin-20200908-ce48f5a-NIGHTLY-git-ce48f5a -- Copyright 2002-2020, LAMP/EPFL

$ dotc -deprecation -feature -classpath "W:\examples\Interpolators\target\classes" -d "W:\examples\Interpolators\target\classes" W:\examples\Interpolators\src\main\scala\Main.scala
-- Error: W:\examples\Interpolators\src\main\scala\Main.scala:52:22 ------------
52 |  val fInterpolator = f"The approximate value of pi is $lifeOfPi%4.2f"
   |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |Exception occurred while executing macro expansion.
   |java.lang.BootstrapMethodError: bootstrap method initialization exception
   |    at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:194)
   |    at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
   |    at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
   |    at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
   |    at dotty.internal.StringContextMacro$.interpolate(StringContextMacro.scala:709)
   |    at dotty.internal.StringContextMacro$.interpolate(StringContextMacro.scala:117)
   |    at dotty.internal.StringContextMacro$.inline$interpolate(StringContextMacro.scala:60)
   |Caused by: java.lang.invoke.LambdaConversionException: Type mismatch for instantiated parameter 0: int is not a subtype of class java.lang.Object
   |    at java.base/java.lang.invoke.AbstractValidatingLambdaMetafactory.checkDescriptor(AbstractValidatingLambdaMetafactory.java:308)
   |    at java.base/java.lang.invoke.AbstractValidatingLambdaMetafactory.validateMetafactoryArgs(AbstractValidatingLambdaMetafactory.java:294)
   |    at java.base/java.lang.invoke.LambdaMetafactory.altMetafactory(LambdaMetafactory.java:503)
   |    at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:138)
   |    at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
   |    at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
   |    at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
   |    at dotty.internal.StringContextMacro$.interpolate(StringContextMacro.scala:709)
   |    at dotty.internal.StringContextMacro$.interpolate(StringContextMacro.scala:117)
   |    at dotty.internal.StringContextMacro$.inline$interpolate(StringContextMacro.scala:60)
   |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   |    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   |    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   |    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   |    at dotty.tools.dotc.transform.Splicer$Interpreter.interpretedStaticMethodCall$$anonfun$2$$anonfun$1(Splicer.scala:328)

   |    at dotty.tools.dotc.transform.Splicer$Interpreter.stopIfRuntimeException(Splicer.scala:388)
   |    at dotty.tools.dotc.transform.Splicer$Interpreter.interpretedStaticMethodCall$$anonfun$1(Splicer.scala:328)
   |    at dotty.tools.dotc.transform.Splicer$Interpreter.interpretTree(Splicer.scala:253)
   |    at dotty.tools.dotc.transform.Splicer$Interpreter.interpretTree$$anonfun$4(Splicer.scala:273)
   |    at dotty.tools.dotc.transform.Splicer$.$anonfun$2(Splicer.scala:53)
   |    at scala.Option.fold(Option.scala:263)
   |    at dotty.tools.dotc.transform.Splicer$.splice(Splicer.scala:53)
   |    at dotty.tools.dotc.typer.Inliner.dotty$tools$dotc$typer$Inliner$$expandMacro(Inliner.scala:1415)
   |    at dotty.tools.dotc.typer.Inliner$InlineTyper.typedApply(Inliner.scala:1242)
   |    at dotty.tools.dotc.typer.Typer.typedUnnamed$1(Typer.scala:2456)
   |    at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2517)
   |    at dotty.tools.dotc.typer.ReTyper.typedUnadapted(ReTyper.scala:122)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2586)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2590)
   |    at dotty.tools.dotc.typer.ReTyper.typedTyped(ReTyper.scala:60)
   |    at dotty.tools.dotc.typer.Typer.typedUnnamed$1(Typer.scala:2461)
   |    at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2517)
   |    at dotty.tools.dotc.typer.ReTyper.typedUnadapted(ReTyper.scala:122)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2586)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2583)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2590)
   |    at dotty.tools.dotc.typer.Inliner.inlined(Inliner.scala:738)
   |    at dotty.tools.dotc.typer.Inliner$.inlineCall(Inliner.scala:127)
   |    at dotty.tools.dotc.typer.Typer.adaptNoArgsOther$4(Typer.scala:3269)
   |    at dotty.tools.dotc.typer.Typer.adaptNoArgs$1(Typer.scala:3362)
   |    at dotty.tools.dotc.typer.Typer.adapt1(Typer.scala:3586)
   |    at dotty.tools.dotc.typer.Typer.adapt(Typer.scala:2917)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2586)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2590)
   |    at dotty.tools.dotc.typer.Namer.typedAheadExpr$$anonfun$1(Namer.scala:1228)
   |    at dotty.tools.dotc.typer.Namer.typedAhead(Namer.scala:1215)
   |    at dotty.tools.dotc.typer.Namer.typedAheadExpr(Namer.scala:1228)
   |    at dotty.tools.dotc.typer.Namer.rhsType$1$$anonfun$1(Namer.scala:1362)
   |    at dotty.tools.dotc.typer.PrepareInlineable$.dropInlineIfError(PrepareInlineable.scala:217)
   |    at dotty.tools.dotc.typer.Namer.rhsType$2(Namer.scala:1362)
   |    at dotty.tools.dotc.typer.Namer.cookedRhsType$1(Namer.scala:1373)
   |    at dotty.tools.dotc.typer.Namer.lhsType$1(Namer.scala:1374)
   |    at dotty.tools.dotc.typer.Namer.inferredType$1(Namer.scala:1385)
   |    at dotty.tools.dotc.typer.Namer.valOrDefDefSig(Namer.scala:1393)
   |    at dotty.tools.dotc.typer.Namer$Completer.typeSig(Namer.scala:680)
   |    at dotty.tools.dotc.typer.Namer$Completer.completeInCreationContext(Namer.scala:801)
   |    at dotty.tools.dotc.typer.Namer$Completer.complete(Namer.scala:712)
   |    at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:166)
   |    at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:188)
   |    at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:190)
   |    at dotty.tools.dotc.core.SymDenotations$SymDenotation.ensureCompleted(SymDenotations.scala:364)
   |    at dotty.tools.dotc.typer.Typer.retrieveSym(Typer.scala:2398)
   |    at dotty.tools.dotc.typer.Typer.typedNamed$1(Typer.scala:2423)
   |    at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2516)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2586)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2590)
   |    at dotty.tools.dotc.typer.Typer.traverse$1(Typer.scala:2612)
   |    at dotty.tools.dotc.typer.Typer.typedStats(Typer.scala:2661)
   |    at dotty.tools.dotc.typer.Typer.typedClassDef(Typer.scala:2118)
   |    at dotty.tools.dotc.typer.Typer.typedTypeOrClassDef$2(Typer.scala:2444)
   |    at dotty.tools.dotc.typer.Typer.typedNamed$1(Typer.scala:2448)
   |    at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2516)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2586)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2590)
   |    at dotty.tools.dotc.typer.Typer.traverse$1(Typer.scala:2612)
   |    at dotty.tools.dotc.typer.Typer.typedStats(Typer.scala:2661)
   |    at dotty.tools.dotc.typer.Typer.typedPackageDef(Typer.scala:2242)
   |    at dotty.tools.dotc.typer.Typer.typedUnnamed$1(Typer.scala:2488)
   |    at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2517)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2586)
   |    at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2590)
   |    at dotty.tools.dotc.typer.Typer.typedExpr(Typer.scala:2705)
   |    at dotty.tools.dotc.typer.FrontEnd.liftedTree1$1(FrontEnd.scala:79)
   |    at dotty.tools.dotc.typer.FrontEnd.typeCheck$$anonfun$1(FrontEnd.scala:84)
   |    at dotty.runtime.function.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
   |    at dotty.tools.dotc.typer.FrontEnd.monitor(FrontEnd.scala:43)
   |    at dotty.tools.dotc.typer.FrontEnd.typeCheck(FrontEnd.scala:85)
   |    at dotty.tools.dotc.typer.FrontEnd.runOn$$anonfun$3(FrontEnd.scala:120)
   |    at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
   |    at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
   |    at scala.collection.immutable.List.foreach(List.scala:333)
   |    at dotty.tools.dotc.typer.FrontEnd.runOn(FrontEnd.scala:120)
   |    at dotty.tools.dotc.Run.runPhases$4$$anonfun$4(Run.scala:185)
   |    at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
   |    at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
   |    at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
   |    at dotty.tools.dotc.Run.runPhases$5(Run.scala:195)
   |    at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:203)
   |    at dotty.runtime.function.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
   |    at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:63)
   |    at dotty.tools.dotc.Run.compileUnits(Run.scala:210)
   |    at dotty.tools.dotc.Run.compileSources(Run.scala:147)
   |    at dotty.tools.dotc.Run.compile(Run.scala:129)
   |    at dotty.tools.dotc.Driver.doCompile(Driver.scala:38)
   |    at dotty.tools.dotc.Driver.process(Driver.scala:195)
   |    at dotty.tools.dotc.Driver.process(Driver.scala:164)
   |    at dotty.tools.dotc.Driver.process(Driver.scala:176)
   |    at dotty.tools.dotc.Driver.main(Driver.scala:203)
   |    at dotty.tools.dotc.Main.main(Main.scala)
   |
   | This location contains code that was inlined from Main.scala:52
1 error found
@smarter
Copy link
Member

smarter commented Sep 10, 2020

What's the output of java -version?

@nicolasstucki
Copy link
Contributor

nicolasstucki commented Sep 10, 2020

@michelou
Copy link
Contributor Author

michelou commented Sep 10, 2020

@smarter Good point. I forgot to mention that I'm using Java 11.0.8+10.

$ java -version
openjdk version "11.0.8" 2020-07-14
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.8+10)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.8+10, mixed mode, sharing)

The code example works fine with Java 1.8.0_262-b10 (i.e. JAVA_HOME=c:\opt\jdk-1.8.0_262-b10).

@michelou michelou changed the title Macro expansion regression Macro expansion regression (Java 11) Sep 10, 2020
@smarter
Copy link
Member

smarter commented Sep 10, 2020

Caused by: java.lang.invoke.LambdaConversionException: Type mismatch for instantiated parameter 0: int is not a subtype of class java.lang.Object

Java 9+ is stricter about the type signature of methods used to implement lambdas which means we sometimes have to create a forwarder method with a different signature, this is supposed to be handled by Erasure when retyping Closures: https://github.com/lampepfl/dotty/blob/760e50310e3913f704af369b8c0aaf1f254f0b14/compiler/src/dotty/tools/dotc/transform/Erasure.scala#L372-L421, I'm not sure why it's not working here.

I think the first thing to do would be to try to reproduce the problem outside of a macro to simplify things, @nicolasstucki is this possible?

@smarter
Copy link
Member

smarter commented Sep 10, 2020

Also we should try to bisect this, @michelou can you check older nightlies of dotty 0.28 (https://repo1.maven.org/maven2/ch/epfl/lamp/dotty_0.28/) to find the first one where this problem occurs?

@michelou
Copy link
Contributor Author

@smarter Ok, I'll try to bisect this from the Maven repository (between 20200827 and 20200908).

@michelou
Copy link
Contributor Author

@smarter That was fast.
The issue appears between 0.27.0-RC1 and 0.28.0-bin-20200827-757e431-NIGHTLY, the first 0.28 nightly build.

$ dotc -version
Dotty compiler version 0.28.0-bin-20200827-757e431-NIGHTLY-git-757e431 -- Copyright 2002-2020, LAMP/EPFL

$ dotc -deprecation -feature -classpath "W:\examples\Interpolators\target\classes" -d "W:\examples\Interpolators\target\classes" W:\examples\Interpolators\src\main\scala\Main.scala
-- Error: W:\examples\Interpolators\src\main\scala\Main.scala:13:22 ------------
13 |  val fInterpolator = f"The approximate value of pi is $lifeOfPi%4.2f"
   |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |Exception occurred while executing macro expansion.
   |java.lang.BootstrapMethodError: bootstrap method initialization exception
   ...
   |    at dotty.tools.dotc.Driver.main(Driver.scala:203)
   |    at dotty.tools.dotc.Main.main(Main.scala)
   |
   | This location contains code that was inlined from Main.scala:13
1 error found

@smarter
Copy link
Member

smarter commented Sep 11, 2020

@smarter
Copy link
Member

smarter commented Sep 11, 2020

Are you cleaning the content of W:\examples\Interpolators\target\classes before every test?

@michelou
Copy link
Contributor Author

Yes. In my initial post I did only retranscript the two comands dotc and dotr.

Here is the full output (using option -debug) of my batch file build.bat (subcommand clean --> rmdir on output line 5):

$ build -debug clean compile
[build] Options    : _DOTTY=1 _TASTY=0 _TIMER=0 _VERBOSE=0
[build] Subcommands: _CLEAN=1 _COMPILE=1 _DECOMPILE=0 _DOC=0 _LINT=0 _RUN=0 _TEST=0
[build] Variables  : JAVA_HOME="C:\opt\jdk-11.0.8+10" DOTTY_HOME="C:\opt\dotty-0.27.0-RC1"
[build] Variables  : _MAIN_CLASS=Main _MAIN_ARGS=
[build] rmdir /s /q "W:\examples\Interpolators\target"
[build] 00000000000000 'W:\examples\Interpolators\target\classes\.latest-build'
[build] 00000000000000 'W:\examples\Interpolators\src\main\java\*.java'
[build] _COMPILE_REQUIRED=0
[build] 00000000000000 'W:\examples\Interpolators\target\classes\.latest-build'
[build] 20200910202506 'W:\examples\Interpolators\src\main\scala\*.scala'
[build] _COMPILE_REQUIRED=1
[build] "C:\opt\dotty-0.27.0-RC1\bin\dotc.bat" "@W:\examples\Interpolators\target\scalac_opts.txt" "@W:\examples\Interpolators\target\scalac_sources.txt"
-- Error: W:\examples\Interpolators\src\main\scala\Main.scala:13:22 ------------
13 |  val fInterpolator = f"The approximate value of pi is $lifeOfPi%4.2f"
   |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |Exception occurred while executing macro expansion.
   |java.lang.BootstrapMethodError: bootstrap method initialization exception
   ...
   |    at dotty.tools.dotc.Driver.main(Driver.scala:203)
   |    at dotty.tools.dotc.Main.main(Main.scala)
   |
   | This location contains code that was inlined from Main.scala:13
1 error found
Error: Compilation of 1 Scala source files failed
[build] _EXITCODE=1

$ C:\opt\dotty-0.27.0-RC1\bin\dotc.bat -version
Dotty compiler version 0.28.0-bin-20200827-757e431-NIGHTLY-git-757e431 -- Copyright 2002-2020, LAMP/EPFL

PS. In order to compare the behavior of the two versions 0.27.0-RC1 and 0.28.0-bin-20200827-757e431-NIGHTLY I simply switch their respective libraries. Do you have any concerns about that operation ?

$ dir /b c:\opt\dotty-0.27.0-RC1\lib\0.27.0-RC1
dist_0.27-0.27.0-RC1.jar
dotty-compiler_0.27-0.27.0-RC1.jar
dotty-doc_0.27-0.27.0-RC1.jar
dotty-interfaces-0.27.0-RC1.jar
dotty-library_0.27-0.27.0-RC1.jar
dotty-staging_0.27-0.27.0-RC1.jar
dotty-tasty-inspector_0.27-0.27.0-RC1.jar
tasty-core_0.27-0.27.0-RC1.jar

$ dir /b c:\opt\dotty-0.27.0-RC1\lib\0.28.0-bin-20200827-757e431-NIGHTLY
dotty-compiler_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty-doc_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty-interfaces-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty-language-server_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty-library_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty-library_sjs1_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty-sbt-bridge-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty-staging_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty-tasty-inspector_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty-tastydoc-input_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty-tastydoc_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar
dotty_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar
tasty-core_0.28-0.28.0-bin-20200827-757e431-NIGHTLY.jar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants