Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump Build Dependencies #118

Open
wants to merge 7,159 commits into
base: 2.12.x
Choose a base branch
from
Open

Bump Build Dependencies #118

wants to merge 7,159 commits into from

Conversation

retronym
Copy link
Owner

No description provided.

SethTisue and others added 30 commits March 17, 2021 11:33
The issue here was that in order to replace a
Scala 3 macro with a matching Scala 2 macro we have
to wait until we have seen all definitions in the
scope - before this PR, only Scala 3 macros were
considered for eviction, now all inline methods are.
…rApply

clarify what it means to 'usually' evaluate in that order
Recognize calls to generic Array.apply with primitive arguments in CleanUp
fix #12357: all inline defs from tasty are treated as macros
Just like in Scala 3.0, adding this keyword doesn't change anything, but
it will be required in future versions of Scala 3 for non-exhaustive
patterns in a for comprehension. We would like to start issuing warnings
by default in Scala 3 for code which does not use `case` in those
situations, but to not hamper cross-compilation we need Scala 2 to also
support that keyword.

For details, see:
https://dotty.epfl.ch/docs/reference/changed-features/pattern-bindings.html
in user-facing contexts, call it REPL not interpreter
Like in Scala 3.0, this allows `?` to be used as a type argument in all
situations where `_` could be used as a wildcard previously. This should
allow us to deprecate the use of `_` as a wildcard in Scala 3 to be able
to eventually repurpose it as explained in
http://dotty.epfl.ch/docs/reference/changed-features/wildcards.html

This is a source-breaking change since a type named `?` is legal in
Scala 2 (but not in Scala 3 unless -source 3.0-migration is used).
`?` also has a special meaning when the kind-projector plugin is used,
but that usage has been deprecated in favor of `*` for a while now.
spec: include pattern matching function in block expression
Spec: add block expression to function application
Support `?` as wildcard marker under -Xsource:3
lrytz and others added 30 commits May 31, 2021 15:01
Regression test for varargs and seq override
Substitute both type and value parameter symbols in return type
Fix specialization of methods with dependent return types
Fix `ArrayOps` bugs (by avoiding using `ArraySeq#array`, which does not guarantee element type)
Include the referenced symbol and the line where it's defined.
More details to forward reference error messages
In 41e376a, the build was updated to support publishing from
Travis CI.

However, on Jenkins, the old means of supplying the publish
credentials to pr-validation snapshots was retained, it has
a ~/.credentials file. So we provided two credentials for the
same host/realm to SBT, and on Jenkins the DirectCredentials
contains an empty password.

Which one of these would SBT pick?

```
sbt 'setupPublishCore https://scala-ci.typesafe.com/artifactory/scala-pr-validation-snapshots/' 'show credentials'
[info] compiler / credentials
[info] 	List(DirectCredentials("Artifactory Realm", "scala-ci.typesafe.com", "scala-ci", ****), FileCredentials("/Users/jz/.credentials"))
[info] scalap / credentials
...    <10 more like this>
[info] credentials
[info] 	List(DirectCredentials("Artifactory Realm", "scala-ci.typesafe.com", "scala-ci", ****))
```

The `ivySbt` task in SBT registers the credentials in order in a global map
in Ivy (`CredentialStore`). So on Jenkins, the invalid `DirectCredentials`
would be overwritten in the map by he `FileCredentials`.

But the fact that this is global state in Ivy appears to be a source of cross
talk between the configured credentials for different modules in the build.
Even though the publish task is serialized through the ivy lock, this lock
does not enclose the previous execution of the `ivySbt` which sets up the
credentials in `CredentialStore`.

In our build, notice that the root project does _not_ have the
`FileCredentials` set. So if the `ivySBT` task for this project runs last,
the global map will have the incorrect `DirectCredentials`.

The fix in our build is easy, avoid configuring the `DirectCredentials` if the
environment variables are absent. We can also standardize on using
`Global/credentials := `.

The principled fix in SBT would be to thread the credentials down to the HTTP
client without using global state. It could also emit a warning if conflicting
credentials are configured for a given host/realm.
Fix intermittent auth failure to artifactory from Jenkins
Fix prefixAligns, avoid spurious outer test warnings on patdep types
…nce} by making them fail fast regardless of ordering

Currently, given the following setup:

```scala
val f1 = Future{Thread.sleep(10000)}
val f2 = Future{Thread.sleep(2000); throw new Exception("Boom")}
```

The following two snippets exhibit different failure behavior:

```scala
val fa = Await.result(f1.zip(f2). Duration.Inf)
```

```scala
val fb = Await.result(f2.zip(f1). Duration.Inf)
```

`fa` fails after 10000ms, while `fb` fails after 2000ms. Both fail with `java.lang.Exception: boom`.

When zipping two `Future`s together, if the left `Future` fails early, the zipped `Future` fails early. But if the right `Future` fails early, the zipped `Future` waits until the right `Future` completes before failing.

`traverse` and `sequence` are similarly implemented with `zipWith` and should exhibit the same behavior. This all arises because `zipWith` is implemented using `flatMap`, which by definition asymmetric due to waiting fo the left `Future` to complete before even considering the right `Future`.

The current behavior makes the failure behavior of `Future`s most unpredictable; in general nobody pays attention to the order of `Future`s when zipping them together, and thus whether a `zipWith`ed/`zip`ed/`traverse`d/`sequence`d `Future` fails early or not is entirely arbitrary.

This PR replaces the implementation of `zipWith`, turning it from `flatMap`-based to `Promise`-based, so that when a `Future` fails early, regardless of whether it's the left or right `Future`, the resultant `Future` will fail immediately.

Implementation-wise I'm using an `AtomicReference` and `compareAndSet`, which should give us the behavior we want without any locking. It may well be possible to achieve with even less overhead, e.g. using only `volatile`s or even using no concurrency controls at all, but I couldn't come up with anything better. If anyone has a better solution I'll happily include it.

This fix would apply to all of `zip`/`zipWith`/`traverse`/`sequence`, since they all are implemented on top of `zipWith`

While it is possible that someone could be relying on the left-biased nature of current `zip`/`zipWith`/`traverse`/`sequence` implementation, but it seems like something that's unlikely to be reliable enough to depend upon. In my experience people generally aren't aware that `zipWith`/`zip`/`traverse`/`sequence`, and they don't generally know the total ordering of how long their Futures take to run. That means status quo behavior would just result in some `Future` fails mysterious taking longer to report for no clear reason.

Notably, the biased nature of these operators is not documented in any of their scaladoc comments.

While there is a non-zero chance that somebody could be intentionally or unintentionally depending on the biased nature of these combinators, there is a much greater chance that someone unaware of the current bias would be puzzled why their highly-concurrent system seems to be taking longer than expected in certain scenarios. It seems likely that this PR would fix more bugs than it would introduce

Note that this does not fix the left-biased fail-fast behavior of `flatMap` chains, or their equivalent `for`-comprehensions, as `flatMap`'s API is inherently left-biased. But anyone who wants fail-fast behavior can convert sections of their `flatMap` chains into `.zip`s where possible, and where not possible that's generally because there is some true data dependency between the `flatMap`s
Fix asymmetric failure behavior of Future#{zip,zipWith,traverse,sequence} by making them fail fast regardless of ordering
If a temp val has been created to hold varargs,
always use it. A single constant arg would induce
inlining and creation of a fresh varargs, ignoring
the unused temp val.
Never inline varargs after named arg rewrite
…mentName

Limit productElementName to productArity
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.