Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dotty with explicit nulls (and flow typing) #7546

Merged
merged 54 commits into from
Dec 14, 2019

Conversation

abeln
Copy link
Contributor

@abeln abeln commented Nov 12, 2019

This PR adds a -Yexplicit-nulls experimental flag which, if enabled, modifies the type system so that reference types are non-nullable. Nullability needs then to be expressed explicitly via unions (e.g. String|Null).

At a high level, the changes to the compiler are:

  • new type hierarchy for Null, as described
  • "translation layer" from Java types to Scala types, which tries to balance soundness and usability

Copy link
Member

@dottybot dottybot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, and thank you for opening this PR! 🎉

All contributors have signed the CLA, thank you! ❤️

Have an awesome day! ☀️

@abeln abeln requested a review from odersky November 12, 2019 19:44
@abeln
Copy link
Contributor Author

abeln commented Nov 12, 2019

@odersky this is a version of #6344 without the flow typing. PTAL.

@odersky
Copy link
Contributor

odersky commented Nov 13, 2019

I'll get to this ASAP

Copy link
Contributor

@odersky odersky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be rebased on top of #7520, or on master once that is in.

I have more to add shortly.

compiler/src/dotty/tools/dotc/core/NullOpsDecorator.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/dotc/core/NullOpsDecorator.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/dotc/core/Flags.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/dotc/core/Symbols.scala Outdated Show resolved Hide resolved
Copy link
Contributor

@odersky odersky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code LGTM. I still want pass over the doc pages.

compiler/src/dotty/tools/dotc/core/TypeComparer.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/dotc/core/Types.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/dotc/core/Types.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/dotc/transform/LazyVals.scala Outdated Show resolved Hide resolved
@odersky
Copy link
Contributor

odersky commented Nov 14, 2019

@liufengyun Could you do a review of the changes to Space.scala? Thanks!

Copy link
Contributor

@liufengyun liufengyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes to exhaustivity check LGTM

Copy link
Contributor

@odersky odersky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything I see looks good.

The only remaining gap is the link with the flow analysis. So, this PR should be rebased on top of #7520 (unless #7520 is merged first) and the $nn insertion (or equivalent) needs to be implemented. Also, there should be more tests that exercise the flow analysis / null typing interplay.

Changes to the docs: The interplay between flow analysis and null typing needs to be documented. The other proposed changes to make this more a user documentation and less a proposal can be deferred into a separate PR.

Two more things: We should test everything once with -Xexplicit-nulls always on, just to see there are no bad surprises lurking there. And we should do a performance benchmark test at the same time.

title: "Explicit Nulls"
---

This proposal describes a modification to the Scala type system that makes reference types
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's merged it's no longer a proposal, so this needs another introduction.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, this should be slightly revised so that it'a doc page for a Scala developer. This means providing more examples and simple explanations. There should be another page with more details that is linked to at the end of the doc page. I would defer all soundness discussions to this second page.

So the first page should be and example driven use-level description, and the second page can then fill in spec, meta-properties such as soundness, and discussion of design tradeoffs.

The docs should also cover flow analysis and how flow analysis links up with typing. See my comment #6344 (comment).

docs/docs/reference/other-new-features/explicit-nulls.md Outdated Show resolved Hide resolved
docs/docs/reference/other-new-features/explicit-nulls.md Outdated Show resolved Hide resolved
docs/docs/reference/other-new-features/explicit-nulls.md Outdated Show resolved Hide resolved
library/src/dotty/DottyPredef.scala Outdated Show resolved Hide resolved
library/src/dotty/DottyPredef.scala Outdated Show resolved Hide resolved
docs/docs/reference/other-new-features/explicit-nulls.md Outdated Show resolved Hide resolved
@odersky
Copy link
Contributor

odersky commented Nov 15, 2019

Since we will no longer pursue NotNull for now, I believe it's better to make Null a value class. But this could be done in a separate issue.

So, things to do:

inline def $notNull[A](x: A | Null): x.type & A =
    x.asInstanceOf

One needs to verify that $notNull calls are indeed erased completely. I propose to put the method in the scala.compiletime package object.

@odersky odersky assigned abeln and noti0na1 and unassigned odersky and liufengyun Nov 15, 2019
@abeln
Copy link
Contributor Author

abeln commented Nov 15, 2019

We're working on addressing the review comments. We'll implement the insertion of notNull in a separate PR for readability (but we'll wait until both PRs are accepted before committing either of them).

Copy link
Member

@lrytz lrytz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is JavaNull treated specially in subtyping? Would def foo(s: String): String = s.trim give an error (expected: String, found: String|JavaNull)?

Once the dust has settled, some of the null checking could maybe find its way into typechecking even without -Yexplicit-nulls

  • warning / suggestion to change T to T|Null when assigning / returning null
  • treat T|Null like T|JavaNull for selections?

docs/docs/reference/other-new-features/explicit-nulls.md Outdated Show resolved Hide resolved
@abeln
Copy link
Contributor Author

abeln commented Nov 18, 2019

@odersky can you clarify your comment in #7546 (comment) that notNull should be inline, given that we will remove it later during/after erasure? Thanks

@odersky
Copy link
Contributor

odersky commented Dec 12, 2019

Do you think we can get this ready by Monday? In that case we could still merge as part of the next Dotty release. That would be nice, since we are supposed to be in feature freeze afterwards. But if it does not work out, we'll just carve out an exception for it.

odersky added a commit to dotty-staging/dotty that referenced this pull request Dec 12, 2019
Implements the scheme described in my comment to scala#7546.

 - We type check an argument without or with flow info,
   depending on whether the function part of the application
   is known to take a call-by-name parameter. If we know nothing
   about the function part, we assume call-by-value.
 - At the end of type checking an application, if the argument is
   known to be by-name, remove all x.$asInstanceOf$[T] casts in the
   argument where x is a mutable variable, and run the type assigner
   (not the type checker!) again on the result. If this succeeds and
   gives a type that is still compatible with the formal parameter type,
   we are done.
 - Otherwise, issue an error saying that the argument cannot be treated
   as call-by-name since it contains flow-assumptions about mutable variables.
   As a remedy, suggest to wrap the argument in a `scala.compiletime.byName(...)`
   call. Here, `byName` is defined as follws:
   ```
   inline def byName[T](x: => T): T
   ```
   Wrapping an argument with byName means that we know statically that
   it is passed to a by-name parameter, so it will be typechecked without
   flow info.
@odersky
Copy link
Contributor

odersky commented Dec 12, 2019

See #7727 for the by-name part. It's not pretty, but I don't know of anything better. For testing it, we should base it on top of this PR.

@noti0na1
Copy link
Member

Hi @odersky , I just pushed my latest changes which solve the comments.

@bishabosha bishabosha requested a review from odersky December 13, 2019 14:13
compiler/src/dotty/tools/dotc/typer/Nullables.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/dotc/typer/Nullables.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/dotc/core/NullOpsDecorator.scala Outdated Show resolved Hide resolved
compiler/src/dotty/tools/dotc/typer/Typer.scala Outdated Show resolved Hide resolved
@odersky
Copy link
Contributor

odersky commented Dec 13, 2019

Except for these minor details, looks good now!

@noti0na1
Copy link
Member

@odersky Thanks for all the comments. I have solved the comments.

Copy link
Contributor

@odersky odersky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks all good now. Congratulations for getting this over the finish line!

@odersky odersky merged commit 8101aeb into scala:master Dec 14, 2019
odersky added a commit to dotty-staging/dotty that referenced this pull request Dec 14, 2019
Implements the scheme described in my comment to scala#7546.

 - We type check an argument without or with flow info,
   depending on whether the function part of the application
   is known to take a call-by-name parameter. If we know nothing
   about the function part, we assume call-by-value.
 - At the end of type checking an application, if the argument is
   known to be by-name, remove all x.$asInstanceOf$[T] casts in the
   argument where x is a mutable variable, and run the type assigner
   (not the type checker!) again on the result. If this succeeds and
   gives a type that is still compatible with the formal parameter type,
   we are done.
 - Otherwise, issue an error saying that the argument cannot be treated
   as call-by-name since it contains flow-assumptions about mutable variables.
   As a remedy, suggest to wrap the argument in a `scala.compiletime.byName(...)`
   call. Here, `byName` is defined as follws:
   ```
   inline def byName[T](x: => T): T
   ```
   Wrapping an argument with byName means that we know statically that
   it is passed to a by-name parameter, so it will be typechecked without
   flow info.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants