*: add executor runtime info for `explain for connection` statement #19183

hesper-guo · 2020-08-13T10:42:34Z

What problem does this PR solve?

Issue Number: close #18233

Problem Summary:
add this runtime information in explain for connection statement.

Example:

When set @@tidb_enable_collect_execution_info=1;

test> show processlist;
+-----+------+-----------+------+---------+------+------------+-----------------------------------------------------------+
| Id  | User | Host      | db   | Command | Time | State      | Info                                                      |
+-----+------+-----------+------+---------+------+------------+-----------------------------------------------------------+
| 775 | root | 127.0.0.1 | test | Query   | 65   | autocommit | select count(*) from t1 tt1 join t1 tt2 where tt1.a>tt2.a |
| 1   | root | 127.0.0.1 | test | Query   | 0    | autocommit | show processlist                                          |
+-----+------+-----------+------+---------+------+------------+-----------------------------------------------------------+
test> explain for connection 775;
+---------------------------+-------------+---------+-----------+-----------------------+----------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| id                        | estRows     | actRows | task      | access object         | execution info                                                                                                       | operator info                                             | memory | disk |
+---------------------------+-------------+---------+-----------+-----------------------+----------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| HashAgg_11                | 1.00        | 0       | root      |                       | time:0s, loops:0                                                                                                     | funcs:count(1)->Column#7                                  | N/A    | N/A  |
| └─HashJoin_14             | 99800100.00 | 1083392 | root      |                       | time:11.67287184s, loops:1058                                                                                        | CARTESIAN inner join, other cond:gt(test.t1.a, test.t1.a) | N/A    | N/A  |
|   ├─IndexReader_19(Build) | 9990.00     | 482836  | root      |                       | time:138.962375ms, loops:473, cop_task: {num: 1, max:125.755703ms, proc_keys: 0, rpc_num: 1, rpc_time: 125.729723ms} | index:IndexFullScan_18                                    | N/A    | N/A  |
|   │ └─IndexFullScan_18    | 9990.00     | 0       | cop[tikv] | table:tt2, index:a(a) | time:0ns, loops:0                                                                                                    | keep order:false, stats:pseudo                            | N/A    | N/A  |
|   └─IndexReader_17(Probe) | 9990.00     | 5120    | root      |                       | time:126.260071ms, loops:5, cop_task: {num: 1, max:125.596953ms, proc_keys: 0, rpc_num: 1, rpc_time: 125.582473ms}   | index:IndexFullScan_16                                    | N/A    | N/A  |
|     └─IndexFullScan_16    | 9990.00     | 0       | cop[tikv] | table:tt1, index:a(a) | time:0ns, loops:0                                                                                                    | keep order:false, stats:pseudo                            | N/A    | N/A  |
+---------------------------+-------------+---------+-----------+-----------------------+----------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+

# after a while
test> explain for connection 775;
+---------------------------+-------------+-----------+-----------+-----------------------+----------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| id                        | estRows     | actRows   | task      | access object         | execution info                                                                   | operator info                                             | memory | disk |
+---------------------------+-------------+-----------+-----------+-----------------------+----------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+
| HashAgg_11                | 1.00        | 0         | root      |                       | time:0s, loops:0                                                                 | funcs:count(1)->Column#7                                  | N/A    | N/A  |
| └─HashJoin_14             | 99800100.00 | 145286144 | root      |                       | time:6m3.062011395s, loops:141881                                                | CARTESIAN inner join, other cond:gt(test.t1.a, test.t1.a) | N/A    | N/A  |
|   ├─IndexReader_19(Build) | 9990.00     | 482836    | root      |                       | time:138.962375ms, loops:473, cop_task: {num: 1, max:125.755703ms, proc_keys: 0} | index:IndexFullScan_18                                    | N/A    | N/A  |
|   │ └─IndexFullScan_18    | 9990.00     | 0         | cop[tikv] | table:tt2, index:a(a) | time:0ns, loops:0                                                                | keep order:false, stats:pseudo                            | N/A    | N/A  |
|   └─IndexReader_17(Probe) | 9990.00     | 19456     | root      |                       | time:127.068821ms, loops:19, cop_task: {num: 1, max:125.596953ms, proc_keys: 0}  | index:IndexFullScan_16                                    | N/A    | N/A  |
|     └─IndexFullScan_16    | 9990.00     | 0         | cop[tikv] | table:tt1, index:a(a) | time:0ns, loops:0                                                                | keep order:false, stats:pseudo                            | N/A    | N/A  |
+---------------------------+-------------+-----------+-----------+-----------------------+----------------------------------------------------------------------------------+-----------------------------------------------------------+--------+------+

When set @@tidb_enable_collect_execution_info=0;, Since TiDB doesn't collect runtime information, so explan for connection will also not display the executor info.

explain for connection 778;
+---------------------------+-----------------+-----------+-----------------------+-----------------------------------------------------------+
| id                        | estRows         | task      | access object         | operator info                                             |
+---------------------------+-----------------+-----------+-----------------------+-----------------------------------------------------------+
| HashAgg_11                | 1.00            | root      |                       | funcs:count(1)->Column#7                                  |
| └─HashJoin_14             | 233130602896.00 | root      |                       | CARTESIAN inner join, other cond:gt(test.t1.a, test.t1.a) |
|   ├─IndexReader_19(Build) | 482836.00       | root      |                       | index:IndexFullScan_18                                    |
|   │ └─IndexFullScan_18    | 482836.00       | cop[tikv] | table:tt2, index:a(a) | keep order:false                                          |
|   └─IndexReader_17(Probe) | 482836.00       | root      |                       | index:IndexFullScan_16                                    |
|     └─IndexFullScan_16    | 482836.00       | cop[tikv] | table:tt1, index:a(a) | keep order:false                                          |
+---------------------------+-----------------+-----------+-----------------------+-----------------------------------------------------------+

Check List

Tests

Manual test (add detailed scripts or steps below)

Side effects

Performance regression
- Consumes more CPU
- Consumes more MEM
Breaking backward compatibility

Release note

Add executor information in explain for connection result.

CLAassistant · 2020-08-13T10:42:40Z

All committers have signed the CLA.

sre-bot · 2020-09-08T11:37:47Z

Please follow PR Title Format:

pkg [, pkg2, pkg3]: what's changed
*: what's changed

crazycs520 · 2020-09-08T11:54:19Z

@qw4990 PTAL

executor/explainfor_test.go

crazycs520 · 2020-09-11T07:21:44Z

/run-all-tests

planner/core/encode.go

qw4990

LGTM

crazycs520 · 2020-09-23T02:49:23Z

/run-all-tests

crazycs520 · 2020-09-25T02:36:31Z

/run-all-tests

crazycs520 · 2020-09-26T02:47:08Z

/run-unit-test

crazycs520 · 2020-09-27T06:18:07Z

/run-all-tests

crazycs520 · 2020-09-27T06:40:57Z

/rebuild

Signed-off-by: crazycs520 <[email protected]>

…y_branch

crazycs520 · 2020-09-27T06:59:56Z

/run-all-tests

crazycs520 · 2020-09-27T12:31:00Z

/run-all-tests

crazycs520 · 2020-09-27T12:35:53Z

/run-all-tests

Signed-off-by: ti-srebot <[email protected]>

ti-srebot · 2020-10-10T06:55:57Z

cherry pick to release-4.0 in PR #20384

This reverts commit c022ad3.

…ingcap#19183)

…19183) (#20384) Signed-off-by: ti-srebot <[email protected]> Signed-off-by: crazycs520 <[email protected]>

hesper-guo requested a review from a team as a code owner August 13, 2020 10:42

hesper-guo requested review from SunRunAway and removed request for a team August 13, 2020 10:42

ti-srebot added the contribution This PR is from a community contributor. label Aug 13, 2020

hesper-guo changed the title ~~My branch~~ Add executor runtime info for explain for connection statement Aug 13, 2020

crazycs520 self-requested a review August 18, 2020 03:33

crazycs520 added the sig/executor label Aug 20, 2020

pingcap deleted a comment from sre-bot Aug 20, 2020

AndreMouche removed the sig/executor label Aug 20, 2020

qw4990 self-requested a review August 21, 2020 03:31

crazycs520 changed the title ~~Add executor runtime info for explain for connection statement~~ *: add executor runtime info for explain for connection statement Sep 8, 2020

hesper-guo requested a review from a team as a code owner September 10, 2020 03:32

github-actions bot added the sig/execution SIG execution label Sep 10, 2020

qw4990 reviewed Sep 10, 2020

View reviewed changes

executor/explainfor_test.go Show resolved Hide resolved

ti-srebot added challenge-program difficulty/easy labels Sep 18, 2020

qw4990 reviewed Sep 22, 2020

View reviewed changes

planner/core/encode.go Outdated Show resolved Hide resolved

SunRunAway requested review from lzmhhh123 and removed request for SunRunAway September 22, 2020 08:31

crazycs520 force-pushed the my_branch branch from 48a5859 to eb813a0 Compare September 23, 2020 02:43

qw4990 previously approved these changes Sep 23, 2020

View reviewed changes

ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 23, 2020

crazycs520 added 2 commits September 27, 2020 11:45

Merge branch 'master' into my_branch

c0fb557

Merge branch 'master' into my_branch

9735147

crazycs520 added 3 commits September 27, 2020 14:57

Merge branch 'master' into my_branch

0f04679

fix race

53658e6

Signed-off-by: crazycs520 <[email protected]>

Merge branch 'my_branch' of https://github.com/hesper-guo/tidb into m…

fb09b8c

…y_branch

crazycs520 added 2 commits September 27, 2020 16:32

Merge branch 'master' into my_branch

d7e8e4d

Merge branch 'master' into my_branch

779505e

crazycs520 approved these changes Sep 27, 2020

View reviewed changes

ti-srebot removed the status/LGT2 Indicates that a PR has LGTM 2. label Sep 27, 2020

ti-srebot approved these changes Sep 27, 2020

View reviewed changes

ti-srebot added the status/LGT3 The PR has already had 3 LGTM. label Sep 27, 2020

crazycs520 merged commit 17b7b5e into pingcap:master Sep 27, 2020

crazycs520 added the needs-cherry-pick-4.0 label Oct 10, 2020

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Oct 10, 2020

cherry pick pingcap#19183 to release-4.0

c022ad3

Signed-off-by: ti-srebot <[email protected]>

ti-srebot mentioned this pull request Oct 10, 2020

*: add executor runtime info for explain for connection statement (#19183) #20384

Merged

crazycs520 added a commit to ti-srebot/tidb that referenced this pull request Oct 22, 2020

Revert "cherry pick pingcap#19183 to release-4.0"

c968e27

This reverts commit c022ad3.

crazycs520 pushed a commit to ti-srebot/tidb that referenced this pull request Oct 22, 2020

*: add executor runtime info for explain for connection statement (p…

f8cfa5c

…ingcap#19183)

ti-srebot added a commit that referenced this pull request Oct 22, 2020

*: add executor runtime info for explain for connection statement (#…

fdd7a42

…19183) (#20384) Signed-off-by: ti-srebot <[email protected]> Signed-off-by: crazycs520 <[email protected]>

crazycs520 mentioned this pull request May 13, 2021

*: update doc for explain for connection pingcap/docs-cn#6260

Merged

10 tasks

This was referenced May 19, 2021

*: update doc for explain for connection (#6260) pingcap/docs-cn#6304

Merged

*: update doc for explain for connection (#6260) pingcap/docs-cn#6305

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: add executor runtime info for `explain for connection` statement #19183

*: add executor runtime info for `explain for connection` statement #19183

hesper-guo commented Aug 13, 2020 •

edited by ti-srebot

Loading

CLAassistant commented Aug 13, 2020 •

edited

Loading

sre-bot commented Sep 8, 2020

crazycs520 commented Sep 8, 2020

crazycs520 commented Sep 11, 2020

qw4990 left a comment

crazycs520 commented Sep 23, 2020

crazycs520 commented Sep 25, 2020

crazycs520 commented Sep 26, 2020

crazycs520 commented Sep 27, 2020

crazycs520 commented Sep 27, 2020

crazycs520 commented Sep 27, 2020

crazycs520 commented Sep 27, 2020

crazycs520 commented Sep 27, 2020

ti-srebot commented Oct 10, 2020

*: add executor runtime info for explain for connection statement #19183

*: add executor runtime info for explain for connection statement #19183

Conversation

hesper-guo commented Aug 13, 2020 • edited by ti-srebot Loading

What problem does this PR solve?

Check List

Release note

CLAassistant commented Aug 13, 2020 • edited Loading

sre-bot commented Sep 8, 2020

crazycs520 commented Sep 8, 2020

crazycs520 commented Sep 11, 2020

qw4990 left a comment

Choose a reason for hiding this comment

crazycs520 commented Sep 23, 2020

crazycs520 commented Sep 25, 2020

crazycs520 commented Sep 26, 2020

crazycs520 commented Sep 27, 2020

crazycs520 commented Sep 27, 2020

crazycs520 commented Sep 27, 2020

crazycs520 commented Sep 27, 2020

crazycs520 commented Sep 27, 2020

ti-srebot commented Oct 10, 2020

*: add executor runtime info for `explain for connection` statement #19183

*: add executor runtime info for `explain for connection` statement #19183

hesper-guo commented Aug 13, 2020 •

edited by ti-srebot

Loading

CLAassistant commented Aug 13, 2020 •

edited

Loading