-
Notifications
You must be signed in to change notification settings - Fork 637
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(testing): Add behavior-driven development #2067
Changes from 16 commits
4380bbc
771b352
a35179a
7246c77
2d5e8d8
7386899
e8f0b94
fd09907
3cf6422
39c17fa
a331353
9313cb4
e1e2b19
eb0b1ab
d01d75d
a93cdf2
27db3c3
84fd424
6583cea
1079bb3
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
|
@@ -278,6 +278,306 @@ Clears all registered benchmarks, so calling `runBenchmarks()` after it wont run | |||||
them. Filtering can be applied by setting `BenchmarkRunOptions.only` and/or | ||||||
`BenchmarkRunOptions.skip` to regular expressions matching benchmark names. | ||||||
|
||||||
## Behavior-driven development | ||||||
|
||||||
With the `bdd.ts` module you can write your tests in a familiar format for | ||||||
grouping tests and adding setup/teardown hooks used by other JavaScript testing | ||||||
frameworks like Jasmine, Jest, and Mocha. | ||||||
|
||||||
The `describe` function creates a block that groups together several related | ||||||
tests. The `it` function registers an individual test case. The `describe` and | ||||||
`it` functions have similar call signatures to `Deno.test`, making it easy to | ||||||
migrate from using `Deno.test`. | ||||||
|
||||||
### Hooks | ||||||
|
||||||
There are 4 types of hooks available for test suites. A test suite can have | ||||||
multiples of each type of hook, they will be called in the order that they are | ||||||
registered. The `afterEach` and `afterAll` hooks will be called whether or not | ||||||
the test case passes. The all hooks will be called once for the whole group | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe we should write There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||
while the each hooks will be called for each individual test case. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||
|
||||||
- `beforeAll`: Runs before all of the tests in the test suite. | ||||||
- `afterAll`: Runs after all of the tests in the test suite finish. | ||||||
- `beforeEach`: Runs before each of the individual test cases in the test suite. | ||||||
- `afterEach`: Runs after each of the individual test cases in the test suite. | ||||||
|
||||||
If a hook is registered at the top level, a global test suite will be registered | ||||||
and all tests will belong to it. Hooks registered at the top level must be | ||||||
registered before any individual test cases or test suites. | ||||||
|
||||||
### Focusing tests | ||||||
|
||||||
If you would like to only run specific individual test cases, you can do so by | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
calling `it.only` instead of `it`. If you would like to only run specific test | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
suites, you can do so by calling `describe.only` instead of `describe`. | ||||||
|
||||||
There is one limitation to this when the individual test cases or test suites | ||||||
belong to another test suite, they will be the only ones to run within the top | ||||||
level test suite. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you provide an example here? It's not obviously clear what this sentence means. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If you look at one of the user test examples I have, this issue can be reproduced by adding another test case or describe block that doesn't belong to the User describe block. If you add I'll still add an example to readme for this limitation. Just thought I'd try explaining it more here first. The reason this problem exists is because Deno.test is called the moment describe is called, and at that time, it doesn't know if any of it's steps have the only flag. If I had a hook into when tests are about to run, I could delay registration until the last moment, allowing me to determine if Deno.test should be called with only flag based on if one of it's child steps has the only flag. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Actually I thought about this more and think I could resolve limitation for test cases registered within the describe callback. It's just the flat style where it wouldn't be known at time of Deno.test call if a step registered after the describe call has the only flag. |
||||||
|
||||||
### Ignoring tests | ||||||
|
||||||
If you would like to not run specific individual test cases, you can do so by | ||||||
calling `it.ignore` instead of `it`. If you would like to only run specific test | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
suites, you can do so by calling `describe.ignore` instead of `describe`. | ||||||
|
||||||
### Sanitization options | ||||||
|
||||||
Like `Deno.TestDefinition`, the `DescribeDefinition` and `ItDefinition` have | ||||||
sanitization options. They work in the same way. | ||||||
|
||||||
- sanitizeExit: Ensure the test case does not prematurely cause the process to | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
exit, for example via a call to Deno.exit. Defaults to true. | ||||||
- sanitizeOps: Check that the number of async completed ops after the test is | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
the same as number of dispatched ops. Defaults to true. | ||||||
- sanitizeResources: Ensure the test case does not "leak" resources - ie. the | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
resource table after the test has exactly the same contents as before the | ||||||
test. Defaults to true. | ||||||
|
||||||
### Permissions option | ||||||
|
||||||
Like `Deno.TestDefinition`, the `DescribeDefintion` and `ItDefinition` have a | ||||||
permissions option. They specify the permissions that should be used to run an | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
individual test case or test suite. Set this to "inherit" to keep the calling | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
thread's permissions. Set this to "none" to revoke all permissions. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
Defaults to "inherit". | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
There is currently one limitation to this, you cannot use the permissions option | ||||||
on an individual test case or test suite that belongs to another test suite. | ||||||
|
||||||
### Migrating and usage | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Comparing Could you rephrase the section name to something like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm fine with that change. I just worded it that way as a way to describe how someone could switch to using bdd.ts if they prefer this style over the default style that they may already be using at the time of finding this. I'll try rewriting this in a way that makes it clear this isn't the preferred way of writing tests for Deno and that it's just another way of writing them. |
||||||
|
||||||
To migrate from `Deno.test`, all you have to do is replace `Deno.test` with | ||||||
`it`. If you are using the step API, you will need to replace `Deno.test` with | ||||||
describe and steps with `describe` or `it`. The callback for individual test | ||||||
cases can be syncronous or asyncronous. | ||||||
|
||||||
Below is an example of a test file using `Deno.test` and `t.step`. In the | ||||||
following sections there are examples of how it can be converted to using nested | ||||||
test grouping, flat test grouping, and a mix of both. | ||||||
|
||||||
```ts | ||||||
// https://deno.land/std@$STD_VERSION/testing/bdd_examples/user_test.ts | ||||||
import { | ||||||
assertEquals, | ||||||
assertStrictEquals, | ||||||
assertThrows, | ||||||
} from "https://deno.land/std@$STD_VERSION/testing/asserts.ts"; | ||||||
import { User } from "https://deno.land/std@$STD_VERSION/testing/bdd_examples/user.ts"; | ||||||
|
||||||
Deno.test("User.users initially empty", () => { | ||||||
assertEquals(User.users.size, 0); | ||||||
}); | ||||||
|
||||||
Deno.test("User constructor", () => { | ||||||
try { | ||||||
const user = new User("Kyle"); | ||||||
assertEquals(user.name, "Kyle"); | ||||||
assertStrictEquals(User.users.get("Kyle"), user); | ||||||
} finally { | ||||||
User.users.clear(); | ||||||
} | ||||||
}); | ||||||
|
||||||
Deno.test("User age", async (t) => { | ||||||
const user = new User("Kyle"); | ||||||
|
||||||
await t.step("getAge", () => { | ||||||
assertThrows(() => user.getAge(), Error, "Age unknown"); | ||||||
user.age = 18; | ||||||
assertEquals(user.getAge(), 18); | ||||||
}); | ||||||
|
||||||
await t.step("setAge", () => { | ||||||
user.setAge(18); | ||||||
assertEquals(user.getAge(), 18); | ||||||
}); | ||||||
}); | ||||||
``` | ||||||
|
||||||
#### Nested test grouping | ||||||
|
||||||
Tests created within the callback of a `describe` function call will belong to | ||||||
the new test suite it creates. The hooks can be created within it or be added to | ||||||
the options argument for describe. | ||||||
|
||||||
```ts | ||||||
// https://deno.land/std@$STD_VERSION/testing/bdd_examples/user_nested_test.ts | ||||||
import { | ||||||
assertEquals, | ||||||
assertStrictEquals, | ||||||
assertThrows, | ||||||
} from "https://deno.land/std@$STD_VERSION/testing/asserts.ts"; | ||||||
import { | ||||||
afterEach, | ||||||
beforeEach, | ||||||
describe, | ||||||
it, | ||||||
} from "https://deno.land/std@$STD_VERSION/testing/bdd.ts"; | ||||||
import { User } from "https://deno.land/std@$STD_VERSION/testing/bdd_examples/user.ts"; | ||||||
|
||||||
describe("User", () => { | ||||||
it("users initially empty", () => { | ||||||
assertEquals(User.users.size, 0); | ||||||
}); | ||||||
|
||||||
it("constructor", () => { | ||||||
try { | ||||||
const user = new User("Kyle"); | ||||||
assertEquals(user.name, "Kyle"); | ||||||
assertStrictEquals(User.users.get("Kyle"), user); | ||||||
} finally { | ||||||
User.users.clear(); | ||||||
} | ||||||
}); | ||||||
|
||||||
describe("age", () => { | ||||||
let user: User; | ||||||
|
||||||
beforeEach(() => { | ||||||
user = new User("Kyle"); | ||||||
}); | ||||||
|
||||||
afterEach(() => { | ||||||
User.users.clear(); | ||||||
}); | ||||||
|
||||||
it("getAge", function () { | ||||||
assertThrows(() => user.getAge(), Error, "Age unknown"); | ||||||
user.age = 18; | ||||||
assertEquals(user.getAge(), 18); | ||||||
}); | ||||||
|
||||||
it("setAge", function () { | ||||||
user.setAge(18); | ||||||
assertEquals(user.getAge(), 18); | ||||||
}); | ||||||
}); | ||||||
}); | ||||||
``` | ||||||
|
||||||
#### Flat test grouping | ||||||
|
||||||
The `describe` function returns a unique symbol that can be used to reference | ||||||
the test suite for adding tests to it without having to create them within a | ||||||
callback. The gives you the ability to have test grouping without any extra | ||||||
indentation in front of the grouped tests. | ||||||
|
||||||
```ts | ||||||
// https://deno.land/std@$STD_VERSION/testing/bdd_examples/user_flat_test.ts | ||||||
import { | ||||||
assertEquals, | ||||||
assertStrictEquals, | ||||||
assertThrows, | ||||||
} from "https://deno.land/std@$STD_VERSION/testing/asserts.ts"; | ||||||
import { | ||||||
describe, | ||||||
it, | ||||||
} from "https://deno.land/std@$STD_VERSION/testing/bdd.ts"; | ||||||
import { User } from "https://deno.land/std@$STD_VERSION/testing/bdd_examples/user.ts"; | ||||||
|
||||||
const userTests = describe("User"); | ||||||
|
||||||
it(userTests, "users initially empty", () => { | ||||||
assertEquals(User.users.size, 0); | ||||||
}); | ||||||
|
||||||
it(userTests, "constructor", () => { | ||||||
try { | ||||||
const user = new User("Kyle"); | ||||||
assertEquals(user.name, "Kyle"); | ||||||
assertStrictEquals(User.users.get("Kyle"), user); | ||||||
} finally { | ||||||
User.users.clear(); | ||||||
} | ||||||
}); | ||||||
|
||||||
const ageTests = describe({ | ||||||
name: "age", | ||||||
suite: userTests, | ||||||
beforeEach(this: { user: User }) { | ||||||
this.user = new User("Kyle"); | ||||||
}, | ||||||
afterEach() { | ||||||
User.users.clear(); | ||||||
}, | ||||||
}); | ||||||
|
||||||
it(ageTests, "getAge", function () { | ||||||
const { user } = this; | ||||||
assertThrows(() => user.getAge(), Error, "Age unknown"); | ||||||
user.age = 18; | ||||||
assertEquals(user.getAge(), 18); | ||||||
}); | ||||||
|
||||||
it(ageTests, "setAge", function () { | ||||||
const { user } = this; | ||||||
user.setAge(18); | ||||||
assertEquals(user.getAge(), 18); | ||||||
}); | ||||||
``` | ||||||
|
||||||
#### Mixed test grouping | ||||||
|
||||||
Both nested test grouping and flat test grouping can be used together. This can | ||||||
be useful if you'd like to create deep groupings without all the extra | ||||||
indentation in front of each line. | ||||||
|
||||||
```ts | ||||||
// https://deno.land/std@$STD_VERSION/testing/bdd_examples/user_mixed_test.ts | ||||||
import { | ||||||
assertEquals, | ||||||
assertStrictEquals, | ||||||
assertThrows, | ||||||
} from "https://deno.land/std@$STD_VERSION/testing/asserts.ts"; | ||||||
import { | ||||||
describe, | ||||||
it, | ||||||
} from "https://deno.land/std@$STD_VERSION/testing/bdd.ts"; | ||||||
import { User } from "https://deno.land/std@$STD_VERSION/testing/bdd_examples/user.ts"; | ||||||
|
||||||
describe("User", () => { | ||||||
it("users initially empty", () => { | ||||||
assertEquals(User.users.size, 0); | ||||||
}); | ||||||
|
||||||
it("constructor", () => { | ||||||
try { | ||||||
const user = new User("Kyle"); | ||||||
assertEquals(user.name, "Kyle"); | ||||||
assertStrictEquals(User.users.get("Kyle"), user); | ||||||
} finally { | ||||||
User.users.clear(); | ||||||
} | ||||||
}); | ||||||
|
||||||
const ageTests = describe({ | ||||||
name: "age", | ||||||
beforeEach(this: { user: User }) { | ||||||
this.user = new User("Kyle"); | ||||||
}, | ||||||
afterEach() { | ||||||
User.users.clear(); | ||||||
}, | ||||||
}); | ||||||
|
||||||
it(ageTests, "getAge", function () { | ||||||
const { user } = this; | ||||||
assertThrows(() => user.getAge(), Error, "Age unknown"); | ||||||
user.age = 18; | ||||||
assertEquals(user.getAge(), 18); | ||||||
}); | ||||||
|
||||||
it(ageTests, "setAge", function () { | ||||||
const { user } = this; | ||||||
user.setAge(18); | ||||||
assertEquals(user.getAge(), 18); | ||||||
}); | ||||||
}); | ||||||
``` | ||||||
|
||||||
## Mocking | ||||||
|
||||||
Test spies are function stand-ins that are used to assert if a function's | ||||||
|
@@ -412,8 +712,8 @@ second example. | |||||
|
||||||
Say we have two functions, `randomMultiple` and `randomInt`, if we want to | ||||||
assert that `randomInt` is called during execution of `randomMultiple` we need a | ||||||
way to spy on the `randomInt` function. That could be done with either either of | ||||||
the spying techniques previously mentioned. To be able to verify that the | ||||||
way to spy on the `randomInt` function. That could be done with either of the | ||||||
spying techniques previously mentioned. To be able to verify that the | ||||||
`randomMultiple` function returns the value we expect it to for what `randomInt` | ||||||
returns, the easiest way would be to replace the `randomInt` function's behavior | ||||||
with more predictable behavior. | ||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's drop the last clause as we generally don't recommend migration from Deno.test to BDD (more descriptions are in the below comment)