Tests fail for robot-simulator

[jiegillet]

I was trying to solve robot-simulator, but the tests keep failing like so

For the solution

turnRight : Robot -> Robot
turnRight robot = robot

turnLeft : Robot -> Robot
turnLeft robot = robot

advance : Robot -> Robot
advance robot = robot

simulate : String -> Robot -> Robot
simulate directions robot = robot

Or any code that compiles. When the code doesn't compile, the test runners shows the compiler errors like usual.

[mpizenberg]

Alright so I've run locally the test runner on this and it appears the issue comes from the fact that the automatic extractor of test code does not work with this tests/Tests.elm file. When adding some additional logging in the tests runner and displaying the code extracted I get the following:

Extracted test code:
[]
End of extracted test code

@ceddlyburge Could you have a look at this. My intuition is that either the test file does not follow a structure that your extractor can extract, or there is a bug somewhere.

Thanks for the report @jiegillet !

[ceddlyburge]

I'll take a look ...

[jiegillet]

I've also looked at things (and managed to run the tests in docker, yay!) and I think I've identified the issue. Your intuition was correct.

The tests for robot-simulator look like this:

tests : Test
tests =
    describe "RobotSimulator"
        [ describe "init"
            (let
                robot =
                    defaultRobot
             in  
             [ test "coordinates" <|
                \() -> Expect.equal { x = 0, y = 0 } robot.coordinates
             , test "bearing" <|
                \() -> Expect.equal North robot.bearing
             ]   
            ) 
        , ...

The test extractor cannot deal with the let statement describe "init" (let ... in [...]). I've confirmed that in a unit test.

Solution 1: patch the test extractor so that it copy pastes the let block inside each of the tests.
Solution 2: change the tests from robot-simulator .

I need to amend my initial comment:

Or any code that compiles.

Obviously the code I was trying to run had some errors, the example code passes all the tests, doesn't ask for the test snippets and all is fine.
This behavior is tricky to check in a CI, because you would need for each exercise a solution that compiles but fails some (all?) the tests. The Debug.todo fails in a different way.
Maybe one could run the test extractor on all the test files and check that the number of snippets is equal to the number of test " strings in the files, or something like that.

[jiegillet]

Maybe one could run the test extractor on all the test files and check that the number of snippets is equal to the number of test " strings in the files, or something like that.

I threw a quick script together to do that locally, it seems that all other exercises are OK.

[ceddlyburge]

I've created a new issue for the test runner to handle let expressions, and I think the quickest thing to fix the robot simulator now is to refactor them to avoid the let expression, so I'll do that ...

[mpizenberg]

@ceddlyburge if you could also mention the test limitation in the guideline https://github.com/exercism/elm/blob/main/docs/contributing-concept.md would be awesome.