Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Randomize test-ghc-env to avoid conflicts between parallel jobs #5757

Merged
merged 8 commits into from
Aug 16, 2022
Merged

Randomize test-ghc-env to avoid conflicts between parallel jobs #5757

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
fe761f9
Make test-ghc-env files unique across parallel jobs
zyla Oct 27, 2021
667f5c4
Merge remote-tracking branch 'upstream/master' into random-test-ghc-env
zyla May 27, 2022
4cce213
Add changelog entry
zyla May 27, 2022
a4f3c08
Merge branch 'master' into random-test-ghc-env
mpilgrem Aug 1, 2022
d4189fc
Merge branch 'master' into random-test-ghc-env
mpilgrem Aug 15, 2022
a96eb9e
Reflect ulidtko's suggestions
mpilgrem Aug 16, 2022
e1bda13
Extension needs to be valid
mpilgrem Aug 16, 2022
0c168c7
Accept Hlint's suggestions
mpilgrem Aug 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions ChangeLog.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ Bug fixes:
[#5714](https://github.com/commercialhaskell/stack/issues/5714)
* Fix an inconsistency in the pretty formatting of the output of
`stack build --coverage`
* Fix non-deterministic test failures when executing a test suite for a
multi-project repository with parallelism enabled. See
[#5024](https://github.com/commercialhaskell/stack/issues/5024)

## v2.7.5

Expand Down
1 change: 1 addition & 0 deletions package.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ dependencies:
- primitive
- process
- project-template
- random
- retry
- rio >= 0.1.21.0
- rio-prettyprint >= 0.1.1.0
Expand Down
4 changes: 3 additions & 1 deletion src/Stack/Build/Execute.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ import System.PosixCompat.Files (createLink, modificationTime, getFile
import RIO.PrettyPrint
import RIO.Process
import Pantry.Internal.Companion
import System.Random (randomIO)

-- | Has an executable been built or not?
data ExecutableBuildStatus
Expand Down Expand Up @@ -1928,10 +1929,11 @@ singleTest topts testsToRun ac ee task installedMap = do
-- see e.g. https://github.com/doctest/issues/119
-- also we set HASKELL_DIST_DIR to a package dist directory so
-- doctest will be able to load modules autogenerated by Cabal
ghcEnvRandomId <- liftIO (randomIO :: IO Int)
let setEnv f pc = modifyEnvVars pc $ \envVars ->
Map.insert "HASKELL_DIST_DIR" (T.pack $ toFilePath buildDir) $
Map.insert "GHC_ENVIRONMENT" (T.pack f) envVars
fp = toFilePath $ eeTempDir ee </> testGhcEnvRelFile
fp = toFilePath (eeTempDir ee </> testGhcEnvRelFile) <> show ghcEnvRandomId
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ghcEnvRandomId line interjects the preceding multiline comment explaining env variables and the next line, let setEnv ... — to which that comment is relevant. Please move somewhere else.

Perhaps like this?

                -- see e.g. https://github.com/doctest/issues/119
                -- also we set HASKELL_DIST_DIR to a package dist directory so
                -- doctest will be able to load modules autogenerated by Cabal
                let setEnv f pc = modifyEnvVars pc $ \envVars ->
                      Map.insert "HASKELL_DIST_DIR" (T.pack $ toFilePath buildDir) $
                      Map.insert "GHC_ENVIRONMENT" (T.pack f) envVars
+               ghcEnvRandomId <- liftIO (randomIO :: IO Int)
-                   fp = toFilePath $ eeTempDir ee </> testGhcEnvRelFile
+               let fp = toFilePath (eeTempDir ee </> testGhcEnvRelFile) <> show ghcEnvRandomId

Further, it's not so hard to stay within typed Path, just import addExtension:

-               ghcEnvRandomId <- liftIO (randomIO :: IO Int)
-               let fp = toFilePath $ eeTempDir ee </> testGhcEnvRelFile
+               randomSuffix <- ("-"<>) . show <$> liftIO (randomIO :: IO Int)
+               fp <- toFilePath <$> addExtension randomSuffix (eeTempDir ee </> testGhcEnvRelFile)

snapDBPath = toFilePathNoTrailingSep (bcoSnapDB $ eeBaseConfigOpts ee)
localDBPath = toFilePathNoTrailingSep (bcoLocalDB $ eeBaseConfigOpts ee)
ghcEnv =
Expand Down
4 changes: 4 additions & 0 deletions stack.cabal
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,7 @@ library
, primitive
, process
, project-template
, random
, retry
, rio >=0.1.21.0
, rio-prettyprint >=0.1.1.0
Expand Down Expand Up @@ -404,6 +405,7 @@ executable stack
, primitive
, process
, project-template
, random
, retry
, rio >=0.1.21.0
, rio-prettyprint >=0.1.1.0
Expand Down Expand Up @@ -528,6 +530,7 @@ executable stack-integration-test
, primitive
, process
, project-template
, random
, retry
, rio >=0.1.21.0
, rio-prettyprint >=0.1.1.0
Expand Down Expand Up @@ -657,6 +660,7 @@ test-suite stack-test
, primitive
, process
, project-template
, random
, raw-strings-qq
, retry
, rio >=0.1.21.0
Expand Down