Skip to content

Commit

Permalink
Disable some flaky scalatest's tests
Browse files Browse the repository at this point in the history
We usually don't notice these failures because the community build
retries a project up to three times in case of failure, but sometimes
all three of them fail, and even when it succeeds it means that
community_build_a sometimes takes much longer than it needs to (2 hours
instead of 1 hour).

I'v reported this upstream at
scalatest/scalatest#2049.
  • Loading branch information
smarter committed Jun 25, 2021
1 parent 0722bf4 commit 7556fd5
Showing 1 changed file with 13 additions and 1 deletion.
14 changes: 13 additions & 1 deletion community-build/src/scala/dotty/communitybuild/projects.scala
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,19 @@ object projects:

lazy val scalatest: SbtCommunityProject = SbtCommunityProject(
project = "scalatest",
sbtTestCommand = "scalacticDotty/clean; scalacticDottyJS/clean; scalacticTestDotty/test; scalatestTestDotty/test; scalacticDottyJS/compile; scalatestDottyJS/compile",
sbtTestCommand =
List(
"scalacticDotty/clean; scalacticDottyJS/clean",

// Some scalatest's tests are flaky (https://github.com/scalatest/scalatest/issues/2049)
// so we disable them, this list is based on the one used in the Scala 2 community build
// (https://github.com/scala/community-build/blob/2.13.x/proj/scalatest.conf).
"""set scalatestTest / Test / unmanagedSources / excludeFilter := HiddenFileFilter || "GeneratorSpec.scala" || "FrameworkSuite.scala" || "WaitersSpec.scala" || "TestSortingReporterSpec.scala" || "JavaFuturesSpec.scala" || "ParallelTestExecutionSpec.scala" || "TimeLimitsSpec.scala" || "TestThreadsStartingCounterSpec.scala" || "SuiteSortingReporterSpec.scala" || "CommonGeneratorsSpec.scala" || "PropCheckerAssertingSpec.scala" || "ConductorMethodsSuite.scala"""",

This comment has been minimized.

Copy link
@smarter

smarter Jul 6, 2021

Author Member

This might actually need to be scalatestTestDotty, or we could just get scalatest/scalatest#2051 in our submodule

"""set scalacticTest / Test / unmanagedSources / excludeFilter := HiddenFileFilter || "NonEmptyArraySpec.scala"""",
"""set genRegularTests4 / Test / managedSources ~= (_.filterNot(_.getName == "FrameworkSuite.scala").filterNot(_.getName == "GeneratorSpec.scala").filterNot(_.getName == "CommonGeneratorsSpec.scala").filterNot(_.getName == "ParallelTestExecutionSpec.scala").filterNot(_.getName == "DispatchReporterSpec.scala").filterNot(_.getName == "TestThreadsStartingCounterSpec.scala").filterNot(_.getName == "EventuallySpec.scala"))""",

"scalacticTestDotty/test; scalatestTestDotty/test; scalacticDottyJS/compile; scalatestDottyJS/compile"
).mkString("; "),
sbtPublishCommand = "scalacticDotty/publishLocal; scalatestDotty/publishLocal; scalacticDottyJS/publishLocal; scalatestDottyJS/publishLocal",
sbtDocCommand = ";scalacticDotty/doc", // fails with missing type ;scalatestDotty/doc"
// cannot take signature of (test: org.scalatest.concurrent.ConductorFixture#OneArgTest):
Expand Down

0 comments on commit 7556fd5

Please sign in to comment.