[LIT] Print discovered tests and percentages

[madhur13490]

This patch adds "nice-to-have" feature in lit.
it prints the total number of discovered tests at the beginning. It is covenient to see the total number of tests and avoid scrolling up to the beginning of log.

Further, this patch also prints %ge of tests.

Differential Revision: https://reviews.llvm.org/D159081

[llvmbot]

@llvm/pr-subscribers-testing-tools

Changes

This patch adds "nice-to-have" feature in lit.
it prints the total number of discovered tests at the beginning. It is covenient to see the total number of tests and avoid scrolling up to the beginning of log.

Further, this patch also prints %ge of tests.

Differential Revision: https://reviews.llvm.org/D159081

Full diff: https://github.com/llvm/llvm-project/pull/66057.diff

1 Files Affected:

• (modified) llvm/utils/lit/lit/main.py (+5-3)

diff --git a/llvm/utils/lit/lit/main.py b/llvm/utils/lit/lit/main.py
index 6858961752a66f1..70230eda48af86f 100755
--- a/llvm/utils/lit/lit/main.py
+++ b/llvm/utils/lit/lit/main.py
@@ -311,6 +311,7 @@ def print_histogram(tests):
 
 def print_results(tests, elapsed, opts):
     tests_by_code = {code: [] for code in lit.Test.ResultCode.all_codes()}
+    total_tests = len(tests)
     for test in tests:
         tests_by_code[test.result.code].append(test)
 
@@ -321,7 +322,7 @@ def print_results(tests, elapsed, opts):
             opts.shown_codes,
         )
 
-    print_summary(tests_by_code, opts.quiet, elapsed)
+    print_summary(total_tests, tests_by_code, opts.quiet, elapsed)
 
 
 def print_group(tests, code, shown_codes):
@@ -336,10 +337,11 @@ def print_group(tests, code, shown_codes):
     sys.stdout.write("\n")
 
 
-def print_summary(tests_by_code, quiet, elapsed):
+def print_summary(total_tests, tests_by_code, quiet, elapsed):
     if not quiet:
         print("\nTesting Time: %.2fs" % elapsed)
 
+    print("\nTotal Discovered Tests: %s" %(total_tests))
     codes = [c for c in lit.Test.ResultCode.all_codes() if not quiet or c.isFailure]
     groups = [(c.label, len(tests_by_code[c])) for c in codes]
     groups = [(label, count) for label, count in groups if count]
@@ -352,4 +354,4 @@ def print_summary(tests_by_code, quiet, elapsed):
     for (label, count) in groups:
         label = label.ljust(max_label_len)
         count = str(count).rjust(max_count_len)
-        print("  %s: %s" % (label, count))
+        print("  %s: %s (%.2f%%)" % (label, count, float(count) / total_tests * 100))

[RoboTux]

Nit: could you write percentage instead of shortening it to %ge? LGTM otherwise.

[jdenny-ornl]

Thanks for the nice feature. Please extend lit's test suite to help lit developers avoid breaking it in the future.

[madhur13490]

@RoboTux @jdenny-ornl Thanks for taking a look at it. I have addressed the review comments in the latest patch.

[RoboTux]

Suggested change

	print("\nTotal Discovered Tests: %s" % (total_tests))
	print(f"\nTotal Discovered Tests: {total_tests}")

[RoboTux]

Wishlist: this can probably also be converted to a f-string or if not to use .format() while you are at it. Since this pre-existing I don't mind if you leave it untouched.

[madhur13490]

Thanks. Will take it in a separate patch.

[madhur13490]

@jdenny-ornl @RoboTux Can I please get approval on this? Would like to land soon.

[RoboTux]

LGTM

[muiez]

Hi, I think the following tests need to be updated as well to avoid breakages:

ignore-fail.py
shtest-define.py
shtest-env.py
shtest-not.py
shtest-pushd-popd.py