Stop mixing test and promise_test usage in SAA tests

[chromium-wpt-export-bot]

It's not clear that the test harness supports mixing these two kinds of
tests safely, without concurrency issues.

Bug: 1427180
Change-Id: Ib9d060bb52477ce99697d559cfb1f1d6e30f468c
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4386517
Commit-Queue: Shuran Huang <[email protected]>
Auto-Submit: Chris Fredrickson <[email protected]>
Reviewed-by: Shuran Huang <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1124445}

[wpt-pr-bot]

The review process for this patch is being conducted in the Chromium project.

[WeizhongX]

python3 ./wpt run --channel=nightly --verify --verify-no-chaos-mode --verify-repeat-loop=0 --verify-repeat-restart=10 --github-checks-text-file=/home/test/artifacts/checkrun.md --affected base_head --log-mach-level=info --log-mach=- -y --no-pause --no-restart-on-unexpected --install-fonts --no-headless --verify-log-full --binary=/home/test/build/firefox/firefox firefox

Some affected tests had inconsistent (flaky) results:

Unstable results

Test	Subtest	Results	Messages
/top-level-storage-access-api/tentative/requestStorageAccessFor-insecure.sub.window.html	[top-level-context] document.requestStorageAccessFor() should be rejected when called in an insecure context	FAIL: 8/10, NOTRUN: 2/10	promise_test: Unhandled rejection with value: object "TypeError: document.requestStorageAccessFor is not a function"

These may be pre-existing or new flakes. Please try to reproduce (see the above WPT command, though some flags may not be needed when running locally) and determine if your change introduced the flake. If you are unable to reproduce the problem, please tag @web-platform-tests/wpt-core-team in a comment for help.

[WeizhongX]

@jcscottiii
created crbug.com/1429782, pls help admin merge, thanks!

[cfredric]

Can someone help me understand why 8 failures and 2 "not run" results mean this test is considered flaky? I would've thought that some mix of pass/fail would be required for a flaky determination. AFAICT, this test just doesn't pass on Firefox, since Firefox doesn't implement/expose the document.requestStorageAccessFor API.

[past]

@cfredric I would recommend pinging jgraham in the WPT matrix channel.

[WeizhongX]

@cfredric FAIL and NOTRUN are different results, so make sense to say it is flaky. My question is why that subtest is the only flaky one. Supposedly another subtest should also be flaky between Timeout and something else to cause this to happen.

Sorry for not looking closely to this earlier. Pls still check with jgraham@ as suggested.

[cfredric]

Thank you, @past!