Convert most WorkspaceTests to Swift Testing

[bkhouri]

Convert most of the WorkspaceTests from XCTests to Swift Testing to make use of parallelism and, in some cases, test parameterization.

Motivation:

The XCTest run, by default, sequentially. Convert most of the WorkspaceTests from XCTests to Swift Testing to make use of parallelism, and in some cases, parameterized test cases.

Not all Test Suites in WorkspaceTests have been converted as some use helpers in swift-tools-core-support, which don't have a matching Swift Testing helper.

Modifications:

Convert XCTests to Swift Testing

Result:

Ran the equivalent of

for _ in $(seq 0 100); 
do
    swift test --enable-swift-testing --disable-xctest
done

and ensured there were no test-related failures.

Blocked by #8137
Requires swiftlang/swift#78300

[bkhouri]

Ready for review, though it may be blocked as not all pipeline builds run with Swift 6.0!

[bkhouri]

Ready for review, though it may be blocked as not all pipeline builds run with Swift 6.0!

Looks like the Selft-hosted macOS pipeline is now running the nightly build
https://ci.swift.org/job/swift-package-manager-with-xcode-self-hosted-PR-osx/4511/execution/node/49/log/

[grynspan]

As above

[bkhouri]

I changed this to

        #expect(throws: ToolsVersionSpecificationWriter.ManifestAccessError(.isADirectory, at: manifestFilePath.parentDirectory)) {
            try ToolsVersionSpecificationWriter.rewriteSpecification(
                manifestDirectory: manifestFilePath.parentDirectory.parentDirectory, // /pkg/
                toolsVersion: toolsVersion,
                fileSystem: inMemoryFileSystem
            )
        }

and got the following compilation error

/Users/bkhouri/Documents/git/public/swiftlang/swift-package-manager/Tests/WorkspaceTests/ToolsVersionSpecificationRewriterTests.swift:153:9: error: macro 'expect(throws:_:sourceLocation:performing:)' requires that 'ToolsVersionSpecificationWriter.ManifestAccessError' conform to 'Equatable'
151 |             return isExpectedKind && isExpectedDescription
152 |         }
153 |         #expect(throws: ToolsVersionSpecificationWriter.ManifestAccessError(.isADirectory, at: manifestFilePath.parentDirectory)) {
    |         `- error: macro 'expect(throws:_:sourceLocation:performing:)' requires that 'ToolsVersionSpecificationWriter.ManifestAccessError' conform to 'Equatable'
154 |             try ToolsVersionSpecificationWriter.rewriteSpecification(
155 |                 manifestDirectory: manifestFilePath.parentDirectory.parentDirectory, // /pkg/

Testing.expect:1:40: note: where 'E' = 'ToolsVersionSpecificationWriter.ManifestAccessError'
1 | @freestanding(expression) public macro expect<E, R>(throws error: E, _ comment: @autoclosure () -> Testing.Comment? = nil, sourceLocation: Testing.SourceLocation = #_sourceLocation, performing expression: () async throws -> R) = #externalMacro(module: "TestingMacros", type: "ExpectMacro") where E : Equatable, E : Error
  |                                        `- note: where 'E' = 'ToolsVersionSpecificationWriter.ManifestAccessError'
[5/6] Compiling WorkspaceTests ToolsVersionSpecificationRewriterTests.swift

Am I doing something wrong here? If feels odd modifying the production code just to make the test compile. is this the norm in Swift?

[grynspan]

It means this Error type doesn't conform to Equatable, so we can't just compare instances of it. You have three choices:

1. Modify the error type to be Equatable (may not be possible depending on the cases),
2. Use #expect(throws: ToolsVersionSpecificationWriter.ManifestAccessError.self) so it only filters on type rather than exact error (which is often sufficient for test purposes), or
3. Continue using #expect {} throws: {}, except I'm deprecating it. :)

[bkhouri]

I would like to validate the exact error as it's more precise test. So I'll leave it as is for now and update once it's deprecated .

[plemarquand]

@swift-ci test

[shahmishal]

@swift-ci test self hosted

[shahmishal]

@swift-ci test self hosted

[shahmishal]

@swift-ci test self hosted

[bkhouri]

This PR is blocked by #8137

[bkhouri]

@swift-ci please test

[bkhouri]

Please test with following pull request:
swiftlang/swift#78300

@swift-ci Please smoke test

[bkhouri]

Please test with following pull request:
swiftlang/swift#78300

@swift-ci Please test

[bkhouri]

Please test with following pull request:
swiftlang/swift#78300

@swift-ci Please test

[bkhouri]

Please test with following pull request:
swiftlang/swift#78300

@swift-ci Please test

[bkhouri]

@swift-ci please test windows

[bkhouri]

Please test with following pull request:
swiftlang/swift#78300

@swift-ci Please smoke test Linux platform

[bkhouri]

Please test with following pull request:
swiftlang/swift#78300

@swift-ci Please test Linux