Does not appear to do anything

[ensonic]

I get this output:

Run pcolby/tap-summary@v1
  with:
    path: /tmp/*.tap
  
    summary-file: tap_summary.md
Run set -o errexit -o noclobber -o nounset -o pipefail
  set -o errexit -o noclobber -o nounset -o pipefail
  [[ -z "$(shopt globstar [2](https://github.com/Buzztrax/buzztrax/actions/runs/7856445010/job/21439210478#step:9:2)> /dev/null)" ]] && echo 'Shell does not support globstar' || shopt -s globstar
  : "${SUMMARY_FILE:=$GITHUB_STEP_SUMMARY}"
  gawk --file "$GITHUB_ACTION_PATH/summary.gawk" --sandbox -- /tmp/*.tap
   >> "$SUMMARY_FILE"
  echo "summary-file=$SUMMARY_FILE" >> "$GITHUB_OUTPUT"
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
  env:
    SUMMARY_FILE: tap_summary.md
|       Test result       | Passed | Failed | Skipped | Test name |
|:------------------------|-------:|-------:|--------:|:---|
| :heavy_check_mark: pass |      0 |      0 |       0 | 2  |
| :heavy_check_mark: pass |      0 |      0 |       0 | ok |

example tap file from /tmp/*.tap (you can download the artifact.zip from the linked build too):

ok 1 - tests/lib/bml/e-core.c:BmlCoreExamples:test_bml_setup: Passed
ok 2 - tests/lib/bml/e-core.c:BmlCoreExamples:test_bml_finalize: Passed
ok 3 - tests/lib/bml/e-core.c:BmlCoreExamples:test_bmln_master_info: Passed
ok 4 - tests/lib/bml/e-core.c:BmlCoreExamples:test_bmln_open: Passed
ok 5 - tests/lib/bml/e-class.c:BmlClassExamples:test_bmln_get_machine_info: Passed
1..5

[pcolby]

Thanks @ensonic, I'll take a look 🙂

[pcolby]

Hey @ensonic, I've just released v1.1.0 which has better support for vanilla TAP files (as opposed to those created by QtTest).

You can see in this build (I forked the commit from the build you linked above), the output is now:

|       Test result       | Passed | Failed | Skipped | Test name   |
|:------------------------|-------:|-------:|--------:|:------------|
| :heavy_check_mark: pass |      5 |      0 |       0 | bml.log     |
| :heavy_check_mark: pass |      6 |      0 |       0 | bt_cmd.log  |
| :x:                fail |    320 |      5 |       0 | bt_core.log |
| :x:                fail |      3 |    110 |       0 | bt_edit.log |
| :heavy_check_mark: pass |     95 |      0 |       0 | bt_gst.log  |
| :heavy_check_mark: pass |     15 |      0 |       0 | bt_ic.log   |

Which renders in the Workflow run summary like:

Test result	Passed	Failed	Skipped	Test name
✔️ pass	5	0	0	bml.log
✔️ pass	6	0	0	bt_cmd.log
❌ fail	320	5	0	bt_core.log
❌ fail	3	110	0	bt_edit.log
✔️ pass	95	0	0	bt_gst.log
✔️ pass	15	0	0	bt_ic.log

You will need to use the v1.1.0 or later tag. v1.1 and v1 have been updated to point to this version too, so any of those will work.

PS Thanks for a great bug report including the build with downloadable artifacts - make it much easier. I hope you don't mind, but I've included your build's TAP files in my automated tests.

Cheers.

[ensonic]

Thanks for the quick fix. Just switching back to your reporter (in Buzztrax/buzztrax@5de1028).
It works! I'll need to figure out how to show the generated summary at the end of the run like you do (https://github.com/pcolby/tap-summary/actions/runs/7861129078) next.

FYI I am using libcheck for the unit tests - in case you want to track the producers.

[pcolby]

Hi Stefan, No idea at this stage. I did notice that my fork of your repo also didn't get the summary in the GitHub web UI, but saw no obvious reason why. I'll have a play with it over the weekend. By the way, the build step typically takes just under 40 minutes... I'm not too familiar with your test framework, but is there a tweak I can make to just have a subset of tests built? I still want one or more tap files produced, but don't want the full test suite run - just to speed up the my experiment cycles :) Cheers, pc ---- http://colby.id.au

…

On Fri, 16 Feb 2024 at 07:53, Stefan Sauer ***@***.***> wrote: Do you have any idea, why I am not seeing the summary like in your automation: - yours: https://github.com/pcolby/tap-summary/actions/runs/7861129078 - mine: https://github.com/Buzztrax/buzztrax/actions/runs/7892774411 — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABHULVUWV2LPQ2ADFMVJFFLYTZYULAVCNFSM6AAAAABDC4WBJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBXGMZDEOBTGU> . You are receiving this because you were assigned.Message ID: ***@***.***>

[pcolby]

Hey @ensonic, I see now why the table appears in our workflow log, and not the summary. It's because you've used folder block scalar, and not chomped the trailing newline in your path property.

That is, as per the example in the README.md, instead of:

      with:
        path: |
          /tmp/*.tap
          /tmp/*.tap

You should:

      with:
        path: >-
          /tmp/*.tap
          /tmp/*.tap

I'll see if I can add some logic to catch that mistake, but might be a bit tricky without affecting Bash's globstar support 🤔 In the meantime, just replace | with >-. Cheers.

[pcolby]

@ensonic, the new 1.1.1 release should now handle any block scalar types. Since you're using the @v1 tag, you should pick up the fix in your next (re-)build. Cheers :D

Edit: You can see an example of the output from my fork of your repo, after rebuilding with no changes (because the tap-summary@v1 tag has moved forward) here: https://github.com/pcolby/buzztrax/actions/runs/7861379079#summary-21637346108

Cheers.

[ensonic]

Thanks for the help. I yesterday added a step to cat the summary to $GITHUB_STEP_SUMMARY and that was what was missing to show the summary. Would it be a useful addition to you README? e.g. suggesting this in a minimal example:

    - name: Output test results
      if: always()
      run: cat test_summary.md >> $GITHUB_STEP_SUMMARY   

[ensonic]

By the way, the build step typically takes just under 40 minutes... I'm not too familiar with your test framework, but is there a tweak I can make to just have a subset of tests built? I still want one or more tap files produced, but don't want the full test suite run - just to speed up the my experiment cycles :)

It is normally much faster, we just ported from travis to github actions and the UI tests time out one by one. I'll try to disable them for the time being, when it runs on github.