Individual test doesn't seem to run after changing its name

[adamw]

Describe the bug

I encountered the following scenario:

As part of this Tapir PR, I added a new test case to VerifyYamlTest, at the bottom:

   test("xyz") {
     val e = endpoint.post.in(header[Option[String]]("X-test"))

     val expectedYaml = load("expected_string_body_response.yml")
     val actualYaml = OpenAPIDocsInterpreter().toOpenAPI(e, Info("Entities", "1.0")).toYaml

     println(expectedYaml)

     val actualYamlNoIndent = noIndentation(actualYaml)
     actualYamlNoIndent shouldBe expectedYaml
  }

After some time the green "play"button appeared and I was able to run this single test. As expected, it failed, and I could see the println result.

However, then I changed the title to should not include '400 invalid value for: header test' response if an optional string header is used, re-run the test, and it was reported as "green". Additionally, no test output was recorded (The test case did not report any output. ) - so I don't think it was ever run. Running from SBT, the test failed.

Expected behavior

The test should still fail.

Operating system

macOS

Editor/Extension

VS Code

Version of Metals

v1.4.2

Extra context or search terms

No response

[kasiaMarek]

This happens if you try to run the test too early before the compilation finishes and code lenses get updated. To avoid confusion we could remove test run code lenses from files that are not compiled and only add them back when it compiles.

[adamw]

I don't think so, I regularly have situations where tests "seem to run" but in fact nothing happens (the code is not run). Arbitrary amount of time after compilation.

[kasiaMarek]

That means that at times things don't update correctly. Similar issue: #6809
@adamw, is the problem strictly related to code lenses or is the information in the test explorer also outdated?

[adamw]

Not sure what's code lenses / test explorer :)

The problem was related to running tests by clicking on the green tick/play button the left of the code. Then the incorrect results where in the "test results" tab.