Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always verify mock arguments #29

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions src/aggregate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,14 @@ set -eu

src=$(dirname $0)

cat ${src}/header.sh
cat ${src}/utils.sh
cat ${src}/asserts.sh
cat ${src}/mocking.sh
cat ${src}/cli.sh
cat ${src}/test-runner.sh
include() {
cat $1
printf '\n'
}

include ${src}/header.sh
include ${src}/utils.sh
include ${src}/asserts.sh
include ${src}/mocking.sh
include ${src}/cli.sh
include ${src}/test-runner.sh
4 changes: 4 additions & 0 deletions src/asserts.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,7 @@ _assert_succeeded() {
_assert_failed() {
test ${1} -ne 0 || assertion_failed "Expected failure exit code\nGot: <$1>"
}

_assert_contains() {
echo "$1" | grep -q "$2" || assertion_failed "Expected: <$1>\nTo match pattern: <$2>"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be To contain pattern cause the assert says assert contains ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

}
9 changes: 6 additions & 3 deletions src/mocking.sh
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,12 @@ validate-args() {
args="\$@"

if [ "\${args}" != "${expected}" ]; then

cat <<OUT
cat > \$0_error <<OUT
Unexpected invocation for command 'some-command':
Got : <"\${args}">
Expected : <"${expected}">
OUT
exit 1
exit 1 # Kept for historical reasons. Proper usage would be mock xyz --with-args "arg1 arg2" --and exit-code 1.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no really, this exit 1's goal was to break the test in case of wrong arguments. since you check that afterward now you can take that off.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, it breaks scripts that invoke it when they fail because of the exit code or when they are running in set -e mode. Removing it changes the contract with the test subject.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh of course, i thought the error file lookup was after a mock but it's at the end of the test. You're right sir ;)

fi
EOF
}
Expand Down Expand Up @@ -159,5 +158,9 @@ _verify_mocks() {
assertion_failed "Command '${command}' was expected to be called $(_format_count ${expected_calls} "time")\nCalled : $(_format_count ${call_count} "time")"
fi
done

for invocation in $(find ${mock} -maxdepth 1 -name "invocation_*_error" 2>/dev/null || true); do
assertion_failed "$(cat $invocation)"
done
done
}
8 changes: 8 additions & 0 deletions test-fixtures/mocking-args-strict/src/code.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash
# N.B. no `set -e` here!

some-command one un
some-command three trois || true # Failure here doesnt matter to the actual code.
some-command two deux

exit 0
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/bash

test_fails_when_verifying_arguments_even_if_exit_code_is_ignored() {
mock some-command --with-args "one un" --once
mock some-command --with-args "two deux" --once
mock some-command --with-args "three trois" --once

./code.sh
}
30 changes: 29 additions & 1 deletion test/test_assert.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ test_assert_equals_with_strings_passing() {
test_assert_equals_with_strings_failing() {
assert "no" equals "yes" > assertion_output
result=${?}
rm .assertion_error # The test runner would think the test failed
_hide_assertion_failure_from_test_runner

assertion_error=$(cat assertion_output)

Expand All @@ -49,3 +49,31 @@ test_assert_multiworks_string_works() {

assert ${?} succeeded
}

test_assert_contains() {
assert "hello world" contains "^hello world$"
assert "hello world" contains "^hello"
assert "hello world" contains "world$"
assert "hello world" contains "w"
}

test_assert_contains_fails_with_proper_error_message() {
assert "abc" contains "z$" > assertion_output
result=${?}
_hide_assertion_failure_from_test_runner

assertion_error=$(cat assertion_output)

expected_error=$(cat <<-EXP
Expected: <abc>
To match pattern: <z$>
EXP
)
assert ${result} failed
assert "${assertion_error}" equals "${expected_error}"
}

_hide_assertion_failure_from_test_runner() {
# The test runner would think the test failed
rm .assertion_error
}
18 changes: 11 additions & 7 deletions test/test_mocking_args_matching.sh
Original file line number Diff line number Diff line change
@@ -1,21 +1,25 @@
#!/bin/bash

test_fails_if_given_arguments_isnt_right() {
mock some-command --with-args "one two three"
test_fails_when_verifying_arguments_even_when_exit_code_is_ignored() {
cp -aR ${TEST_ROOT_DIR}/../test-fixtures/mocking-args-strict/* .

some-command three two one > assertion_output
unset RUN_SINGLE_TEST
actual=$(${TEST_ROOT_DIR}/../target/sbtest.sh verify_arguments_order)
assert ${?} failed

expected_error=$(cat <<-EXP
Unexpected invocation for command 'some-command':
Got : <"three two one">
Expected : <"one two three">
Got : <"three trois">
Expected : <"two deux">
Unexpected invocation for command 'some-command':
Got : <"two deux">
Expected : <"three trois">
EXP
)
assert "$(cat assertion_output)" equals "${expected_error}"
assert "${actual}" contains "${expected_error}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

asserting how problems are displayed is an important part of the user experience of this library, by using contains instead of equals you seem to be hiding something, what is it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stuff being hidden is caused by the test setup, we essentially have a fake test suite and the output contains much more than the errors themselves. By using contains we can focus on the important part for that test. Otherwise, this test will break when we change, for example, the test report header or footer.

Copy link
Contributor Author

@mat128 mat128 Oct 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exact text is the following:

Running Simple Bash Tests
-------------------------

verify_arguments_order.fails_when_verifying_arguments_even_if_exit_code_is_ignored...FAILED

=========================
FAIL: verify_arguments_order.fails_when_verifying_arguments_even_if_exit_code_is_ignored
-------- STDOUT ---------
Unexpected invocation for command 'some-command':
Got :      <"three trois">
Expected : <"two deux">
Unexpected invocation for command 'some-command':
Got :      <"two deux">
Expected : <"three trois">
-------- STDERR ---------

-------------------------

-------------------------
Ran 1 test

>>> FAILURE (1 problem) <<<

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay i get the extent of this now, sorry for that.

The test you're replacing is kind of part of the doc on how to use mocking... yes the tests is the doc, so it's important that it's clear and understandable.

Therefore may i ask that the test remains there and that your test becomes a new one at the end of the file ?

And i will agree to the assert contains since this only changes stuff inside the mocking mechanism.

PS. (trailing new lines :) )

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done :)

}

test_fails_if_2_with_args_argments_are_given() {
test_fails_if_2_with_args_arguments_are_given() {
mock some-command --with-args "a" --with-args "b" > error
assert ${?} failed

Expand Down