Tests

There is a set of tests, the first half of which date back to the original project, modified to keep them functioning. The rest I added as I implemented each new feature.

Each test consists of the test itself, a .txt file, and a corresponding .txt.expected file containing the expected output. All of the newer tests also include an equivalent C file you so you can test the behavior of different compilers. Some, like switch.c include comments detailing the differing behavior and the relevant specifications.

So for example to run the aforementioned switch test (assuming you’ve already built it) you’d do

~/C_Interpreter/build $ ./cinterpreter ../tests/switch.txt
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
1

which exactly matches the .expected file.

There are 2 bash scripts * runtests.sh * runtests_valgrind.sh

The first runs all of the tests and compares the output with the expected output using diff, the second runs them through valgrind to detect memory errors. Note the tests are split into regular tests and preprocessor tests. There are only 2 of the latter and they only run the preprocessor stage (-E option just like gcc).

The output should look like this

~/C_Interpreter/build $ ../runtests.sh
running normal tests
==================
executing ../tests/backslash.txt
executing ../tests/break_continue.txt
executing ../tests/problem6.txt
executing ../tests/declarations.txt
executing ../tests/dowhile.txt
executing ../tests/for_stmt.txt
executing ../tests/functions.txt
executing ../tests/goto_label.txt
executing ../tests/ifelse.txt
executing ../tests/newtest.txt
Warning, falling off the end of non-void function.
executing ../tests/scope.txt
executing ../tests/switch.txt
executing ../tests/ternary.txt
executing ../tests/test_assignment_basic1.txt
Warning, falling off the end of non-void function.
executing ../tests/test_assignment_basic2.txt
Warning, falling off the end of non-void function.
executing ../tests/test_assignment_variables1.txt
executing ../tests/test_assignment_variables2_operators.txt
Warning, falling off the end of non-void function.
executing ../tests/test_assignment_variables_no_init.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_if1.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_if2.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_if3_list2.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_if3_list.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_if4_list.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_ifif1.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_ifif2.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_ifif3.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_ifif4.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_i_if_f1.txt
executing ../tests/test_control_i_if_f2.txt
Parse Error: return statement with no expression in a function with a return type
, got ; at line 16, position 8
executing ../tests/test_control_i_if_f3.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_i_if_f4.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_i_if_f5.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_i_if_fif1.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_i_if_fif2.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_i_if_f_i_if_f1.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_i_if_f_i_if_f2.txt
Parse Error: return statement with no expression in a function with a return type
, got ; at line 72, position 8
executing ../tests/test_control_while1.txt
executing ../tests/test_control_while2.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_while3_fibonacci.txt
executing ../tests/test_control_while4.txt
executing ../tests/test_control_while5.txt
Warning, falling off the end of non-void function.
executing ../tests/test_control_whilewhile6.txt
executing ../tests/test_control_whilewhile7.txt
executing ../tests/test_control_whilewhile8.txt
executing ../tests/test_control_whilewhile9.txt
executing ../tests/test_control_whilewhile_if1.txt
executing ../tests/types.txt
comparing output of ../tests/backslash.txt
comparing output of ../tests/break_continue.txt
comparing output of ../tests/problem6.txt
comparing output of ../tests/declarations.txt
comparing output of ../tests/dowhile.txt
comparing output of ../tests/for_stmt.txt
comparing output of ../tests/functions.txt
comparing output of ../tests/goto_label.txt
31,32c31,32
< 160
< 160
---
> 0
> 0
comparing output of ../tests/ifelse.txt
comparing output of ../tests/newtest.txt
comparing output of ../tests/scope.txt
comparing output of ../tests/switch.txt
comparing output of ../tests/ternary.txt
comparing output of ../tests/test_assignment_basic1.txt
comparing output of ../tests/test_assignment_basic2.txt
comparing output of ../tests/test_assignment_variables1.txt
comparing output of ../tests/test_assignment_variables2_operators.txt
comparing output of ../tests/test_assignment_variables_no_init.txt
comparing output of ../tests/test_control_if1.txt
comparing output of ../tests/test_control_if2.txt
comparing output of ../tests/test_control_if3_list2.txt
comparing output of ../tests/test_control_if3_list.txt
comparing output of ../tests/test_control_if4_list.txt
comparing output of ../tests/test_control_ifif1.txt
comparing output of ../tests/test_control_ifif2.txt
comparing output of ../tests/test_control_ifif3.txt
comparing output of ../tests/test_control_ifif4.txt
comparing output of ../tests/test_control_i_if_f1.txt
comparing output of ../tests/test_control_i_if_f2.txt
1,2d0
< Parse Error: return statement with no expression in a function with a return type
< Got ; at line 16, position 8
comparing output of ../tests/test_control_i_if_f3.txt
comparing output of ../tests/test_control_i_if_f4.txt
comparing output of ../tests/test_control_i_if_f5.txt
comparing output of ../tests/test_control_i_if_fif1.txt
comparing output of ../tests/test_control_i_if_fif2.txt
comparing output of ../tests/test_control_i_if_f_i_if_f1.txt
comparing output of ../tests/test_control_i_if_f_i_if_f2.txt
1,2d0
< Parse Error: return statement with no expression in a function with a return type
< Got ; at line 72, position 8
comparing output of ../tests/test_control_while1.txt
comparing output of ../tests/test_control_while2.txt
comparing output of ../tests/test_control_while3_fibonacci.txt
comparing output of ../tests/test_control_while4.txt
comparing output of ../tests/test_control_while5.txt
comparing output of ../tests/test_control_whilewhile6.txt
comparing output of ../tests/test_control_whilewhile7.txt
comparing output of ../tests/test_control_whilewhile8.txt
comparing output of ../tests/test_control_whilewhile9.txt
comparing output of ../tests/test_control_whilewhile_if1.txt
comparing output of ../tests/types.txt

running preprocessor tests
==========================
running preprocessor on ../tests/preprocessor.txt
running preprocessor on ../tests/include.c
comparing preprocessor output of ../tests/preprocessor.txt
0a1,19
> # 0 "../tests/preprocessor.txt"
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
11,14d29
<
<
<
<
37c52
<
---
>
47a63
>
comparing preprocessor output of ../tests/include.c
1,2c1,4
< # 2 "include.c"
< # 1 "../tests/inc_test1.c" 1
---
> # 0 "../tests/include.c"
>
>
> # 0 "../tests/inc_test1.c"
6c8,9
< # 1 "../tests/../tests/inc_test2.c" 1
---
>
> # 0 "../tests/inc_test2.c"
16c19
< # 6 "../tests/inc_test1.c" 2
---
> # 5 "../tests/inc_test1.c"
51d53
<
62,63d63
< # 4 "include.c" 2
< a
66a67,68
> # 3 "../tests/include.c"
> a
68d69
<
84a86
>

In other words, the only output are correct warnings and parse errors, which aren’t piped correctly since they’re printed to stderr (TODO), undefined behavior (in the case of the goto test, I matched tcc output last I checked), and whitespace and generated preprocessor lines in the include test.

For the valgrind test, I just do

~/programming/C/C_Interpreter/build $ ../runtests_valgrind.sh | grep ERROR
==5106== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==5106== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==5107== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==5107== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==5108== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
...
==5154== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==5155== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==5155== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==5156== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==5156== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

because valgrind spits out a lot of output.