Feature request: please support target names

[paulz]

Thank you for useful GitHub action!

We used to use codecov GitHub action to extract xcodebuild coverage and upload.
Since new codecov uploader does not process Xcode coverage, see codecov/uploader#223 we are looking to use this action to extract coverage for upload

One useful feature it had was ability to specify target names:
https://github.com/codecov/codecov-bash/blob/ae76691c8554334bc1ddc355f6667fd36d6890f5/codecov#L165-L169

    -J           Specify packages to build coverage. Uploader will only build these packages.
                 This can significantly reduces time to build coverage reports.
                 -J 'MyAppName'      Will match "MyAppName" and "MyAppNameTests"
                 -J '^ExampleApp$'   Will match only "ExampleApp" not "ExampleAppTests"

it allowed us:

1. to run multiple test targets (unit tests and ui tests) and upload multiple packages separately.
2. separate coverage for our framework and app from other libraries and Pods

[ffried]

#6 will add a RegEx filter to do exactly that. 👍

[paulz]

that would we awesome!

I've tried to put the name of our framework as name filter:

    - uses: sersoft-gmbh/xcode-coverage-action@feature/additional-inputs
      id: xcode-cov
      with:
        derived-data: /tmp/ci-derived-data
        name-filter: Testable
    - uses: codecov/[email protected]
      with:
        token: ${{secrets.CODECOV_TOKEN}}
        files: ${{join(fromJSON(steps.xcode-cov.outputs.files), ',')}}
        flags: unit-specs
        name: self-hosted

got xcrun failure:

Run sersoft-gmbh/xcode-coverage-action@feature/additional-inputs
2
  with:
3
    derived-data: /tmp/ci-derived-data
4
    name-filter: Testable
5
    output: ./.xcodecov
6
    format: txt
7
    fail-on-empty-output: false
8
Validating input
9
Setting up paths
10
Finding coverage files
11
Converting files
8277
Error: The process '/usr/bin/xcrun' failed with exit code 1

also tried Testable*.* to match Testable.framework

[ffried]

@paulz
I've just refined the debug output a bit.
Can you please add a secret named ACTIONS_STEP_DEBUG with its value set to true? This should print a bit more information on what fails.

Also note that I've renamed the input name-filter to target-name-filter to better describe its meaning.

[ffried]

Btw. the target-name-filter expects a full regular expression (in contrast to the bash uploader of Codecov). So in your case this would be either ^Testable$ for an exact match on Testable or ^Testable.*$ to match Testable followed by any (or no) characters.

[paulz]

tried ^Testable.*$ with target-name-filter:

Run sersoft-gmbh/xcode-coverage-action@feature/additional-inputs
2
  with:
3
    derived-data: /tmp/ci-derived-data
4
    target-name-filter: ^Testable.*$
5
    output: ./.xcodecov
6
    format: txt
7
    fail-on-empty-output: false
8
Validating input
9
Setting up paths
10
Finding coverage files
14
Converting files
15
  /usr/bin/xcrun llvm-cov show -instr-profile /tmp/ci-derived-data/Build/ProfileData/6DEA7331-2DDF-5104-9173-BB09B5DC0E26/Coverage.profdata /tmp/ci-derived-data/Build/Products/Debug/Legible/Legible.framework/Legible
16
  /Users/sparta/actions-runner/_work/mariposa/mariposa/Pods/Legible/Legible/Classes/BehavesLikeCombine.swift:
17
...
...
     16|       |
8274
     17|       |@end
8275
  
8276
  /usr/bin/xcrun llvm-cov show -instr-profile /tmp/ci-derived-data/Build/ProfileData/6DEA7331-2DDF-5104-9173-BB09B5DC0E26/Coverage.profdata /tmp/ci-derived-data/Build/Products/Debug/Pods_UnitSpecs.framework/Pods_UnitSpecs
8277
Error: The process '/usr/bin/xcrun' failed with exit code 1

is it possible to add ACTIONS_STEP_DEBUG to one workflow only? we are on experimental branch here

[paulz]

would be nice if there would be some output from these sub steps:

Validating input
Setting up paths
Finding coverage files

[ffried]

The first two of those sub-steps don't do much worth printing.
The "Finding coverage files" and "Converting coverage files" are the interesting ones. But given that the failure occurs in "Converting coverage files", the action already found some files.

You can try to add ACTIONS_STEP_DEBUG via env. Not sure if that works, though.

[ffried]

I'm a bit surprised that you see those failures when your target-name-filter shouldn't match the files listed in your output. 🤔

Either way, I've just added another input ignore-conversion-failures to the action. Setting it to true will ignore failures of the llvm-cov call. I'd recommend setting fail-on-empty-output to true as well, to ensure that the action still fails if none of the files could be successfully converted.
The action now also attempts to convert all found files before failing if the conversion of one file failed.

[ffried]

(Note that the Codecov bash uploader ignored failures as well: https://github.com/codecov/codecov-bash/blob/ae76691c8554334bc1ddc355f6667fd36d6890f5/codecov#L235)

[ffried]

I've merged the PR for now. I'll keep this issue open, though, until it works for you as well. You just need to change to the main branch for now.

Note that I also renamed the action from xcode-coverage-action to swift-coverage-action since I'm gonna add Linux support next.
GitHub redirects the old name, so you don't have to adjust it, but it's sure safer if you do 😉

[ffried]

@paulz
Just a little heads-up: Linux support has landed on main. The derived-data input was renamed to search-paths and can now accept a list of (newline separated) paths which will be searched.

Any news from your end? I think I'm gonna release a version 2 with the changes in main now. Let me know if you still have issues.

[paulz]

we got successful run with first target!

using:

    - uses: sersoft-gmbh/swift-coverage-action@main
      id: unit-specs-coverage
      with:
        search-paths: /tmp/ci-derived-data
        target-name-filter: ^Testable$
    - uses: codecov/[email protected]
      with:
        token: ${{secrets.CODECOV_TOKEN}}
        files: ${{join(fromJSON(steps.unit-specs-coverage.outputs.files), ',')}}
        flags: unit-specs
        name: self-hosted

here is debug output;"

##[debug]Evaluating condition for step: 'Run sersoft-gmbh/swift-coverage-action@main'
2
##[debug]Evaluating: success()
3
##[debug]Evaluating success:
4
##[debug]=> true
5
##[debug]Result: true
6
##[debug]Starting: Run sersoft-gmbh/swift-coverage-action@main
7
##[debug]Loading inputs
8
##[debug]Loading env
9
Run sersoft-gmbh/swift-coverage-action@main
17
::group::Validating input
18
Validating input
20
::group::Setting up paths
21
Setting up paths
23
::group::Finding coverage files
24
Finding coverage files
25
  ##[debug]Found profdata file: /tmp/ci-derived-data/Build/ProfileData/6DEA7331-2DDF-5104-9173-BB09B5DC0E26/Coverage.profdata
26
  Found 1 profiling data file(s)...
27
  ::endgroup::
28
::group::Converting files
29
Converting files
30
  ##[debug]Checking contents of build dir /tmp/ci-derived-data/Build of prof data file /tmp/ci-derived-data/Build/ProfileData/6DEA7331-2DDF-5104-9173-BB09B5DC0E26/Coverage.profdata
31
  ##[debug]Found match of type framework: /tmp/ci-derived-data/Build/Products/Debug/Legible/Legible.framework
32
  ##[debug]Project name: Legible
33
  Skipping Legible due to target name filter...
34
  ##[debug]Found match of type framework: /tmp/ci-derived-data/Build/Products/Debug/Nimble/Nimble.framework
35
  ##[debug]Project name: Nimble
36
  Skipping Nimble due to target name filter...
37
  ##[debug]Found match of type framework: /tmp/ci-derived-data/Build/Products/Debug/Pods_UnitSpecs.framework
38
  ##[debug]Project name: Pods_UnitSpecs
39
  Skipping Pods_UnitSpecs due to target name filter...
40
  ##[debug]Found match of type framework: /tmp/ci-derived-data/Build/Products/Debug/Quick/Quick.framework
41
  ##[debug]Project name: Quick
42
  Skipping Quick due to target name filter...
43
  ##[debug]Found match of type framework: /tmp/ci-derived-data/Build/Products/Debug/Testable.framework
44
  ##[debug]Project name: Testable
45
  /usr/bin/xcrun llvm-cov show -instr-profile /tmp/ci-derived-data/Build/ProfileData/6DEA7331-2DDF-5104-9173-BB09B5DC0E26/Coverage.profdata /tmp/ci-derived-data/Build/Products/Debug/Testable.framework/Testable

[paulz]

works great, thank you @ffried !