Merge pull request #362 from tsenart/improve-text-report-documentation

Improve text report documentation

README.md

@@ -7,23 +7,30 @@ It can be used both as a command line utility and a library.
 ![Vegeta](http://fc09.deviantart.net/fs49/i/2009/198/c/c/ssj2_vegeta_by_trunks24.jpg)
 
 ## Install
+
 ### Pre-compiled executables
+
 Get them [here](http://github.com/tsenart/vegeta/releases).
 
 ### Homebrew on Mac OS X
+
 You can install Vegeta using the [Homebrew](https://github.com/Homebrew/homebrew/) package manager on Mac OS X:
+
 ```shell
 $ brew update && brew install vegeta
 ```
 
 ### Source
+
 You need `go` installed and `GOBIN` in your `PATH`. Once that is done, run the
 command:
+
 ```shell
 $ go get -u github.com/tsenart/vegeta
 ```
 
 ## Versioning
+
 Both the library and the CLI are versioned with [SemVer v2.0.0](https://semver.org/spec/v2.0.0.html).
 
 After [v8.0.0](https://github.com/tsenart/vegeta/tree/v8.0.0), the two components
@@ -34,9 +41,11 @@ As for the library, new versions are tagged with both `lib/vMAJOR.MINOR.PATCH` a
 The latter tag is required for compatibility with `go mod`.
 
 ## Contributing
+
 See [CONTRIBUTING.md](.github/CONTRIBUTING.md).
 
 ## Usage manual
+
 ```console
 Usage: vegeta [global flags] <command> [command flags]
 
@@ -126,36 +135,44 @@ examples:
 ```
 
 #### `-cpus`
+
 Specifies the number of CPUs to be used internally.
 It defaults to the amount of CPUs available in the system.
 
 #### `-profile`
-Specifies which profiler to enable during execution. Both *cpu* and
-*heap* profiles are supported. It defaults to none.
+
+Specifies which profiler to enable during execution. Both _cpu_ and
+_heap_ profiles are supported. It defaults to none.
 
 #### `-version`
+
 Prints the version and exits.
 
 ### `attack` command
 
 #### `-body`
+
 Specifies the file whose content will be set as the body of every
 request unless overridden per attack target, see `-targets`.
 
 #### `-cert`
+
 Specifies the PEM encoded TLS client certificate file to be used with HTTPS requests.
 If `-key` isn't specified, it will be set to the value of this flag.
 
 #### `-connections`
+
 Specifies the maximum number of idle open connections per target host.
 
 #### `-duration`
+
 Specifies the amount of time to issue request to the targets.
 The internal concurrency structure's setup has this value as a variable.
 The actual run time of the test can be longer than specified due to the
 responses delay. Use 0 for an infinite attack.
 
 #### `-format`
+
 Specifies the targets format to decode.
 
 ##### `json` format
@@ -183,13 +200,15 @@ meant to be used by people writing targets by hand for simple use cases.
 Here are a few examples of valid targets files in the http format:
 
 ###### Simple targets
+
 ```
 GET http://goku:9090/path/to/dragon?item=ball
 GET http://user:password@goku:9090/path/to
 HEAD http://goku:9090/path/to/success
 ```
 
 ###### Targets with custom headers
+
 ```
 GET http://user:password@goku:9090/path/to
 X-Account-ID: 8675309
@@ -200,6 +219,7 @@ Authorization: Token DEADBEEF
 ```
 
 ###### Targets with custom bodies
+
 ```
 POST http://goku:9090/things
 @/path/to/newthing.json
@@ -209,97 +229,115 @@ PATCH http://goku:9090/thing/71988591
 ```
 
 ###### Targets with custom bodies and headers
+
 ```
 POST http://goku:9090/things
 X-Account-ID: 99
 @/path/to/newthing.json
 ```
 
 #### `-h2c`
+
 Specifies that HTTP2 requests are to be sent over TCP without TLS encryption.
 
 #### `-header`
+
 Specifies a request header to be used in all targets defined, see `-targets`.
 You can specify as many as needed by repeating the flag.
 
 #### `-http2`
+
 Specifies whether to enable HTTP/2 requests to servers which support it.
 
 #### `-insecure`
+
 Specifies whether to ignore invalid server TLS certificates.
 
 #### `-keepalive`
+
 Specifies whether to reuse TCP connections between HTTP requests.
 
 #### `-key`
+
 Specifies the PEM encoded TLS client certificate private key file to be
 used with HTTPS requests.
 
 #### `-laddr`
+
 Specifies the local IP address to be used.
 
 #### `-lazy`
+
 Specifies whether to read the input targets lazily instead of eagerly.
 This allows streaming targets into the attack command and reduces memory
 footprint.
 The trade-off is one of added latency in each hit against the targets.
 
 #### `-max-body`
+
 Specifies the maximum number of bytes to capture from the body of each
 response. Remaining unread bytes will be fully read but discarded.
 Set to -1 for no limit. It knows how to intepret values like these:
 
-* `"10 MB"` -> `10MB`
-* `"10240 g"` -> `10TB`
-* `"2000"` -> `2000B`
-* `"1tB"` -> `1TB`
-* `"5 peta"` -> `5PB`
-* `"28 kilobytes"` -> `28KB`
-* `"1 gigabyte"` -> `1GB`
+- `"10 MB"` -> `10MB`
+- `"10240 g"` -> `10TB`
+- `"2000"` -> `2000B`
+- `"1tB"` -> `1TB`
+- `"5 peta"` -> `5PB`
+- `"28 kilobytes"` -> `28KB`
+- `"1 gigabyte"` -> `1GB`
 
 #### `-name`
 
 Specifies the name of the attack to be recorded in responses.
 
 #### `-output`
+
 Specifies the output file to which the binary results will be written
 to. Made to be piped to the report command input. Defaults to stdout.
 
-####  `-rate`
+#### `-rate`
+
 Specifies the request rate per time unit to issue against
 the targets. The actual request rate can vary slightly due to things like
 garbage collection, but overall it should stay very close to the specified.
 If no time unit is provided, 1s is used.
 
 #### `-redirects`
+
 Specifies the max number of redirects followed on each request. The
 default is 10. When the value is -1, redirects are not followed but
 the response is marked as successful.
 
 #### `-resolvers`
+
 Specifies custom DNS resolver addresses to use for name resolution instead of
 the ones configured by the operating system. Works only on non Windows systems.
 
 #### `-root-certs`
+
 Specifies the trusted TLS root CAs certificate files as a comma separated
 list. If unspecified, the default system CAs certificates will be used.
 
 #### `-targets`
+
 Specifies the file from which to read targets, defaulting to stdin.
 See the [`-format`](#-format) section to learn about the different target formats.
 
 #### `-timeout`
+
 Specifies the timeout for each request. The default is 0 which disables
 timeouts.
 
 #### `-workers`
+
 Specifies the initial number of workers used in the attack. The actual
 number of workers will increase if necessary in order to sustain the
 requested rate.
 
 ### `report` command
 
-```
+```console
 Usage: vegeta report [options] [<file>...]
 
 Outputs a report of attack results.
@@ -325,6 +363,7 @@ Examples:
 ```
 
 #### `report -type=text`
+
 ```console
 Requests      [total, rate]             1200, 120.00
 Duration      [total, attack, wait]     10.094965987s, 9.949883921s, 145.082066ms
@@ -342,7 +381,36 @@ Get http://localhost:6060: net/http: transport closed before response was receiv
 Get http://localhost:6060: http: can't write HTTP request on broken connection
 ```
 
+The `Requests` row shows:
+
+- The `total` number of issued requests.
+- The real request `rate` sustained during the attack.
+
+The `Duration` row shows:
+
+- The `attack` time taken issuing all requests (`total` - `wait`)
+- The `wait` time waiting for the response to the last issued request (`total` - `attack`)
+- The `total` time taken in the attack (`attack` + `wait`)
+
+Latency is the amount of time taken for a response to a request to be read (including the `-max-body` bytes from the response body).
+
+- `mean` is the [arithmetic mean / average](https://en.wikipedia.org/wiki/Arithmetic_mean) of the latencies of all requests in an attack.
+- `50`, `95`, `99` are the 50th, 95th an 99th [percentiles](https://en.wikipedia.org/wiki/Percentile), respectively, of the latencies of all requests in an attack. To understand more about why these are useful, I recommend [this article](https://bravenewgeek.com/everything-you-know-about-latency-is-wrong/) from @tylertreat.
+- `max` is the maximum latency of all requests in an attack.
+
+The `Bytes In` and `Bytes Out` rows shows:
+
+- The `total` number of bytes sent (out) or received (in) with the request or response bodies.
+- The `mean` number of bytes sent (out) or received (in) with the request or response bodies.
+
+The `Success` ratio shows the percentage of requests whose responses didn't error and had status codes between **200** and **400** (non-inclusive).
+
+The `Status Codes` row shows a histogram of status codes. `0` status codes mean a request failed to be sent.
+
+The `Error Set` shows a unique set of errors returned by all issued requests. These include requests that got non-successful response status code.
+
 #### `report -type=json`
+
 ```json
 {
   "latencies": {
@@ -377,8 +445,10 @@ Get http://localhost:6060: http: can't write HTTP request on broken connection
 ```
 
 #### `report -type=hist`
+
 Computes and prints a text based histogram for the given buckets.
 Each bucket upper bound is non-inclusive.
+
 ```console
 cat results.bin | vegeta report -type='hist[0,2ms,4ms,6ms]'
 Bucket         #     %       Histogram
@@ -457,6 +527,7 @@ Examples:
 ```
 
 ## Usage: Distributed attacks
+
 Whenever your load test can't be conducted due to Vegeta hitting machine limits
 such as open files, memory, CPU or network bandwidth, it's a good idea to use Vegeta in a distributed manner.
 
@@ -497,6 +568,7 @@ Error Set:
 ```
 
 ## Usage: Real-time Analysis
+
 If you are a happy user of iTerm, you can integrate vegeta with [jplot](https://github.com/rs/jplot) using [jaggr](https://github.com/rs/jaggr) to plot a vegeta report in real-time in the comfort of you terminal:
 
 ```
@@ -553,22 +625,26 @@ func main() {
 ```
 
 #### Limitations
+
 There will be an upper bound of the supported `rate` which varies on the
 machine being used.
 You could be CPU bound (unlikely), memory bound (more likely) or
 have system resource limits being reached which ought to be tuned for
 the process execution. The important limits for us are file descriptors
 and processes. On a UNIX system you can get and set the current
 soft-limit values for a user.
+
 ```shell
 $ ulimit -n # file descriptors
 2560
 $ ulimit -u # processes / threads
 709
 ```
+
 Just pass a new number as the argument to change it.
 
 ## License
+
 See [LICENSE](LICENSE).
 
 ## Donate