Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

util: add tiflash wait time in execution summary #57058

Merged
merged 8 commits into from
Nov 13, 2024

Conversation

yibin87
Copy link
Contributor

@yibin87 yibin87 commented Nov 1, 2024

What problem does this PR solve?

Issue Number: close #57059 ref pingcap/tiflash#9489

Problem Summary:

What changed and how does it work?

TPCH_100 Q9 non-buzy situation

+--------------------------------------------------------------------------------------------+--------------+-----------+--------------+----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+
| id                                                                                         | estRows      | actRows   | task         | access object  | execution info                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | operator info                                                                                                                                                                                                                                                  | memory  | disk    |
+--------------------------------------------------------------------------------------------+--------------+-----------+--------------+----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+
| Sort_26                                                                                    | 2406.00      | 175       | root         |                | time:6.82s, loops:2, RU:627572.024989                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | tpch_100.nation.n_name, Column#53:desc                                                                                                                                                                                                                         | 17.5 KB | 0 Bytes |
| └─TableReader_225                                                                          | 2406.00      | 175       | root         |                | time:6.82s, loops:2, cop_task: {num: 24, max: 0s, min: 0s, avg: 0s, p95: 0s, copr_cache_hit_ratio: 0.00}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | MppVersion: 2, data:ExchangeSender_224                                                                                                                                                                                                                         | 7.39 KB | N/A     |
|   └─ExchangeSender_224                                                                     | 2406.00      | 175       | mpp[tiflash] |                | tiflash_task:{proc max:6.81s, min:6.8s, avg: 6.81s, p80:6.81s, p95:6.81s, iters:21, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | ExchangeType: PassThrough                                                                                                                                                                                                                                      | N/A     | N/A     |
|     └─Projection_29                                                                        | 2406.00      | 175       | mpp[tiflash] |                | tiflash_task:{proc max:6.81s, min:6.8s, avg: 6.81s, p80:6.81s, p95:6.81s, iters:21, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | tpch_100.nation.n_name, Column#53, Column#55                                                                                                                                                                                                                   | N/A     | N/A     |
|       └─Projection_220                                                                     | 2406.00      | 175       | mpp[tiflash] |                | tiflash_task:{proc max:6.81s, min:6.8s, avg: 6.81s, p80:6.81s, p95:6.81s, iters:21, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | Column#55, tpch_100.nation.n_name, Column#53                                                                                                                                                                                                                   | N/A     | N/A     |
|         └─HashAgg_221                                                                      | 2406.00      | 175       | mpp[tiflash] |                | tiflash_task:{proc max:6.81s, min:6.8s, avg: 6.81s, p80:6.81s, p95:6.81s, iters:21, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | group by:Column#72, tpch_100.nation.n_name, funcs:sum(Column#73)->Column#55, funcs:firstrow(tpch_100.nation.n_name)->tpch_100.nation.n_name, funcs:firstrow(Column#72)->Column#53, stream_count: 16                                                            | N/A     | N/A     |
|           └─ExchangeReceiver_223                                                           | 2406.00      | 525       | mpp[tiflash] |                | tiflash_task:{proc max:6.81s, min:6.8s, avg: 6.81s, p80:6.81s, p95:6.81s, iters:21, tasks:3, threads:48}, tiflash_wait: {pipeline_queue_wait: 4ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | stream_count: 16                                                                                                                                                                                                                                               | N/A     | N/A     |
|             └─ExchangeSender_222                                                           | 2406.00      | 525       | mpp[tiflash] |                | tiflash_task:{proc max:6.81s, min:0s, avg: 2.27s, p80:6.81s, p95:6.81s, iters:3, tasks:3, threads:3}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.nation.n_name, collate: utf8mb4_bin], stream_count: 16                                                                                                                              | N/A     | N/A     |
|               └─HashAgg_218                                                                | 2406.00      | 525       | mpp[tiflash] |                | tiflash_task:{proc max:6.81s, min:0s, avg: 2.27s, p80:6.81s, p95:6.81s, iters:3, tasks:3, threads:3}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | group by:Column#77, Column#78, funcs:sum(Column#76)->Column#73                                                                                                                                                                                                 | N/A     | N/A     |
|                 └─Projection_228                                                           | 475516500.66 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.78s, min:0s, avg: 2.26s, p80:6.78s, p95:6.78s, iters:3546, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | minus(mul(tpch_100.lineitem.l_extendedprice, minus(1, tpch_100.lineitem.l_discount)), mul(tpch_100.partsupp.ps_supplycost, tpch_100.lineitem.l_quantity))->Column#76, tpch_100.nation.n_name->Column#77, extract(YEAR, tpch_100.orders.o_orderdate)->Column#78 | N/A     | N/A     |
|                   └─Projection_207                                                         | 475516500.66 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.74s, min:0s, avg: 2.25s, p80:6.74s, p95:6.74s, iters:3546, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | tpch_100.lineitem.l_quantity, tpch_100.lineitem.l_extendedprice, tpch_100.lineitem.l_discount, tpch_100.partsupp.ps_supplycost, tpch_100.orders.o_orderdate, tpch_100.nation.n_name                                                                            | N/A     | N/A     |
|                     └─Projection_197                                                       | 475516500.66 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.74s, min:0s, avg: 2.25s, p80:6.74s, p95:6.74s, iters:3546, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | tpch_100.nation.n_name, tpch_100.lineitem.l_quantity, tpch_100.lineitem.l_extendedprice, tpch_100.lineitem.l_discount, tpch_100.partsupp.ps_supplycost, tpch_100.orders.o_orderdate, tpch_100.lineitem.l_orderkey                                              | N/A     | N/A     |
|                       └─HashJoin_196                                                       | 475516500.66 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.74s, min:0s, avg: 2.25s, p80:6.74s, p95:6.74s, iters:3546, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | inner join, equal:[eq(tpch_100.lineitem.l_orderkey, tpch_100.orders.o_orderkey)], stream_count: 16                                                                                                                                                             | N/A     | N/A     |
|                         ├─ExchangeReceiver_83(Build)                                       | 150000000.00 | 150000000 | mpp[tiflash] |                | tiflash_task:{proc max:593.6ms, min:0s, avg: 197.9ms, p80:593.6ms, p95:593.6ms, iters:16123, tasks:3, threads:48}, tiflash_wait: {pipeline_queue_wait: 12ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | stream_count: 16                                                                                                                                                                                                                                               | N/A     | N/A     |
|                         │ └─ExchangeSender_82                                              | 150000000.00 | 150000000 | mpp[tiflash] |                | tiflash_task:{proc max:1.77s, min:0s, avg: 589.5ms, p80:1.77s, p95:1.77s, iters:2400, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.orders.o_orderkey, collate: binary], stream_count: 16                                                                                                                               | N/A     | N/A     |
|                         │   └─TableFullScan_81                                             | 150000000.00 | 150000000 | mpp[tiflash] | table:orders   | tiflash_task:{proc max:240.6ms, min:0s, avg: 80.2ms, p80:240.6ms, p95:240.6ms, iters:2400, tasks:3, threads:48}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:90, remote_regions:0, tot_learner_read:32ms, region_balance:{instance_num: 3, max/min: 30/30=1.000000}, delta_rows:0, delta_bytes:0, segments:180, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:177ms, tot_build_inputstream:683ms, min_local_stream:0ms, max_local_stream:224ms, dtfile:{data_scanned_rows:150000000, data_skipped_rows:582144, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:168ms, tot_read:6385ms}}, tiflash_wait: {pipeline_queue_wait: 191ms}       | keep order:false                                                                                                                                                                                                                                               | N/A     | N/A     |
|                         └─ExchangeReceiver_80(Probe)                                       | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.35s, min:0s, avg: 2.12s, p80:6.35s, p95:6.35s, iters:3546, tasks:3, threads:48}, tiflash_wait: {pipeline_breaker_wait: 2154ms, pipeline_queue_wait: 1ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |                                                                                                                                                                                                                                                                | N/A     | N/A     |
|                           └─ExchangeSender_79                                              | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.8s, min:0s, avg: 2.27s, p80:6.8s, p95:6.8s, iters:3503, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.lineitem.l_orderkey, collate: binary]                                                                                                                                               | N/A     | N/A     |
|                             └─Projection_78                                                | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.44s, min:0s, avg: 2.15s, p80:6.44s, p95:6.44s, iters:3503, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | tpch_100.nation.n_name, tpch_100.lineitem.l_orderkey, tpch_100.lineitem.l_quantity, tpch_100.lineitem.l_extendedprice, tpch_100.lineitem.l_discount, tpch_100.partsupp.ps_supplycost, tpch_100.lineitem.l_suppkey, tpch_100.lineitem.l_partkey                 | N/A     | N/A     |
|                               └─HashJoin_51                                                | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.44s, min:0s, avg: 2.15s, p80:6.44s, p95:6.44s, iters:3503, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | inner join, equal:[eq(tpch_100.lineitem.l_suppkey, tpch_100.partsupp.ps_suppkey) eq(tpch_100.lineitem.l_partkey, tpch_100.partsupp.ps_partkey)], stream_count: 16                                                                                              | N/A     | N/A     |
|                                 ├─ExchangeReceiver_77(Build)                               | 80000000.00  | 80000000  | mpp[tiflash] |                | tiflash_task:{proc max:602.9ms, min:0s, avg: 201ms, p80:602.9ms, p95:602.9ms, iters:8596, tasks:3, threads:48}, tiflash_wait: {pipeline_queue_wait: 5ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | stream_count: 16                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                 │ └─ExchangeSender_76                                      | 80000000.00  | 80000000  | mpp[tiflash] |                | tiflash_task:{proc max:912.9ms, min:0s, avg: 304.3ms, p80:912.9ms, p95:912.9ms, iters:1291, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.partsupp.ps_suppkey, collate: binary], [name: tpch_100.partsupp.ps_partkey, collate: binary], stream_count: 16                                                                      | N/A     | N/A     |
|                                 │   └─TableFullScan_75                                     | 80000000.00  | 80000000  | mpp[tiflash] | table:partsupp | tiflash_task:{proc max:190.5ms, min:0s, avg: 63.5ms, p80:190.5ms, p95:190.5ms, iters:1291, tasks:3, threads:48}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:59, remote_regions:0, tot_learner_read:44ms, region_balance:{instance_num: 3, max/min: 20/19=1.052632}, delta_rows:0, delta_bytes:0, segments:119, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:227ms, tot_build_inputstream:687ms, min_local_stream:0ms, max_local_stream:174ms, dtfile:{data_scanned_rows:80000000, data_skipped_rows:0, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:266ms, tot_read:7016ms}}, tiflash_wait: {pipeline_queue_wait: 104ms}             | keep order:false                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                 └─ExchangeReceiver_74(Probe)                               | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:5.94s, min:0s, avg: 1.98s, p80:5.94s, p95:5.94s, iters:3503, tasks:3, threads:48}, tiflash_wait: {pipeline_breaker_wait: 1116ms, pipeline_queue_wait: 2ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |                                                                                                                                                                                                                                                                | N/A     | N/A     |
|                                   └─ExchangeSender_73                                      | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.79s, min:0s, avg: 2.26s, p80:6.79s, p95:6.79s, iters:42574, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.lineitem.l_suppkey, collate: binary], [name: tpch_100.lineitem.l_partkey, collate: binary]                                                                                          | N/A     | N/A     |
|                                     └─Projection_72                                        | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.63s, min:0s, avg: 2.21s, p80:6.63s, p95:6.63s, iters:42574, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | tpch_100.nation.n_name, tpch_100.lineitem.l_orderkey, tpch_100.lineitem.l_partkey, tpch_100.lineitem.l_suppkey, tpch_100.lineitem.l_quantity, tpch_100.lineitem.l_extendedprice, tpch_100.lineitem.l_discount                                                  | N/A     | N/A     |
|                                       └─HashJoin_52                                        | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:6.62s, min:0s, avg: 2.21s, p80:6.62s, p95:6.62s, iters:42574, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | inner join, equal:[eq(tpch_100.lineitem.l_partkey, tpch_100.part.p_partkey)], stream_count: 16                                                                                                                                                                 | N/A     | N/A     |
|                                         ├─ExchangeReceiver_71(Build)                       | 16000000.00  | 1085639   | mpp[tiflash] |                | tiflash_task:{proc max:355.8ms, min:0s, avg: 118.6ms, p80:355.8ms, p95:355.8ms, iters:144, tasks:3, threads:48}, tiflash_wait: {pipeline_queue_wait: 31ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | stream_count: 16                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                         │ └─ExchangeSender_70                              | 16000000.00  | 1085639   | mpp[tiflash] |                | tiflash_task:{proc max:229.2ms, min:0s, avg: 76.4ms, p80:229.2ms, p95:229.2ms, iters:318, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.part.p_partkey, collate: binary], stream_count: 16                                                                                                                                  | N/A     | N/A     |
|                                         │   └─Selection_69                                 | 16000000.00  | 1085639   | mpp[tiflash] |                | tiflash_task:{proc max:206ms, min:0s, avg: 68.7ms, p80:206ms, p95:206ms, iters:318, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | like(tpch_100.part.p_name, "%dim%", 92)                                                                                                                                                                                                                        | N/A     | N/A     |
|                                         │     └─TableFullScan_68                           | 20000000.00  | 20000000  | mpp[tiflash] | table:part     | tiflash_task:{proc max:178.2ms, min:0s, avg: 59.4ms, p80:178.2ms, p95:178.2ms, iters:318, tasks:3, threads:48}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:13, remote_regions:0, tot_learner_read:5ms, region_balance:{instance_num: 3, max/min: 5/4=1.250000}, delta_rows:0, delta_bytes:0, segments:27, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:15ms, tot_build_inputstream:59ms, min_local_stream:0ms, max_local_stream:172ms, dtfile:{data_scanned_rows:20000000, data_skipped_rows:0, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:16ms, tot_read:2260ms}}, tiflash_wait: {pipeline_queue_wait: 53ms}                      | pushed down filter:empty, keep order:false, stats:partial[p_partkey:allEvicted]                                                                                                                                                                                | N/A     | N/A     |
|                                         └─ExchangeReceiver_67(Probe)                       | 600845438.27 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:4.47s, min:0s, avg: 1.49s, p80:4.47s, p95:4.47s, iters:42574, tasks:3, threads:48}, tiflash_wait: {pipeline_breaker_wait: 356ms, pipeline_queue_wait: 52ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |                                                                                                                                                                                                                                                                | N/A     | N/A     |
|                                           └─ExchangeSender_66                              | 600845438.27 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:6.77s, min:0s, avg: 2.26s, p80:6.77s, p95:6.77s, iters:28073, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.lineitem.l_partkey, collate: binary]                                                                                                                                                | N/A     | N/A     |
|                                             └─Projection_65                                | 600845438.27 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:4.95s, min:0s, avg: 1.65s, p80:4.95s, p95:4.95s, iters:28073, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | tpch_100.nation.n_name, tpch_100.lineitem.l_orderkey, tpch_100.lineitem.l_partkey, tpch_100.lineitem.l_suppkey, tpch_100.lineitem.l_quantity, tpch_100.lineitem.l_extendedprice, tpch_100.lineitem.l_discount                                                  | N/A     | N/A     |
|                                               └─HashJoin_53                                | 600845438.27 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:4.94s, min:0s, avg: 1.65s, p80:4.94s, p95:4.94s, iters:28073, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | inner join, equal:[eq(tpch_100.supplier.s_suppkey, tpch_100.lineitem.l_suppkey)], stream_count: 16                                                                                                                                                             | N/A     | N/A     |
|                                                 ├─ExchangeReceiver_61(Build)               | 1000000.00   | 1000000   | mpp[tiflash] |                | tiflash_task:{proc max:150.9ms, min:0s, avg: 50.3ms, p80:150.9ms, p95:150.9ms, iters:144, tasks:3, threads:48}, tiflash_wait: {pipeline_queue_wait: 21ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | stream_count: 16                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                                 │ └─ExchangeSender_60                      | 1000000.00   | 1000000   | mpp[tiflash] |                | tiflash_task:{proc max:101.2ms, min:0s, avg: 33.7ms, p80:101.2ms, p95:101.2ms, iters:16, tasks:3, threads:16}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.supplier.s_suppkey, collate: binary], stream_count: 16                                                                                                                              | N/A     | N/A     |
|                                                 │   └─Projection_59                        | 1000000.00   | 1000000   | mpp[tiflash] |                | tiflash_task:{proc max:77.2ms, min:0s, avg: 25.7ms, p80:77.2ms, p95:77.2ms, iters:16, tasks:3, threads:16}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | tpch_100.nation.n_name, tpch_100.supplier.s_suppkey                                                                                                                                                                                                            | N/A     | N/A     |
|                                                 │     └─HashJoin_54                        | 1000000.00   | 1000000   | mpp[tiflash] |                | tiflash_task:{proc max:77.2ms, min:0s, avg: 25.7ms, p80:77.2ms, p95:77.2ms, iters:16, tasks:3, threads:16}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | inner join, equal:[eq(tpch_100.nation.n_nationkey, tpch_100.supplier.s_nationkey)]                                                                                                                                                                             | N/A     | N/A     |
|                                                 │       ├─ExchangeReceiver_57(Build)       | 25.00        | 25        | mpp[tiflash] |                | tiflash_task:{proc max:7.21ms, min:0s, avg: 2.4ms, p80:7.21ms, p95:7.21ms, iters:1, tasks:3, threads:16}, tiflash_wait: {pipeline_queue_wait: 1ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |                                                                                                                                                                                                                                                                | N/A     | N/A     |
|                                                 │       │ └─ExchangeSender_56              | 25.00        | 25        | mpp[tiflash] |                | tiflash_task:{proc max:18.8ms, min:0s, avg: 6.28ms, p80:18.8ms, p95:18.8ms, iters:1, tasks:3, threads:16}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | ExchangeType: Broadcast, Compression: FAST                                                                                                                                                                                                                     | N/A     | N/A     |
|                                                 │       │   └─TableFullScan_55             | 25.00        | 25        | mpp[tiflash] | table:nation   | tiflash_task:{proc max:17.8ms, min:0s, avg: 5.95ms, p80:17.8ms, p95:17.8ms, iters:1, tasks:3, threads:16}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:1, remote_regions:0, tot_learner_read:1ms, region_balance:{instance_num: 1, max/min: 1/1=1.000000}, delta_rows:0, delta_bytes:0, segments:1, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:0ms, tot_build_inputstream:1ms, min_local_stream:0ms, max_local_stream:4ms, dtfile:{data_scanned_rows:25, data_skipped_rows:0, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:0ms, tot_read:0ms}}                                                                                      | keep order:false                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                                 │       └─TableFullScan_58(Probe)          | 1000000.00   | 1000000   | mpp[tiflash] | table:supplier | tiflash_task:{proc max:63.2ms, min:0s, avg: 21.1ms, p80:63.2ms, p95:63.2ms, iters:16, tasks:3, threads:16}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:1, remote_regions:0, tot_learner_read:1ms, region_balance:{instance_num: 1, max/min: 1/1=1.000000}, delta_rows:0, delta_bytes:0, segments:3, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:0ms, tot_build_inputstream:1ms, min_local_stream:0ms, max_local_stream:58ms, dtfile:{data_scanned_rows:1000000, data_skipped_rows:0, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:0ms, tot_read:52ms}}, tiflash_wait: {pipeline_breaker_wait: 4ms, pipeline_queue_wait: 17ms}       | keep order:false                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                                 └─ExchangeReceiver_64(Probe)               | 600037902.00 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:680.5ms, min:0s, avg: 226.8ms, p80:680.5ms, p95:680.5ms, iters:28073, tasks:3, threads:48}, tiflash_wait: {pipeline_breaker_wait: 109ms, pipeline_queue_wait: 116ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |                                                                                                                                                                                                                                                                | N/A     | N/A     |
|                                                   └─ExchangeSender_63                      | 600037902.00 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:6.71s, min:0s, avg: 2.24s, p80:6.71s, p95:6.71s, iters:9670, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.lineitem.l_suppkey, collate: binary]                                                                                                                                                | N/A     | N/A     |
|                                                     └─TableFullScan_62                     | 600037902.00 | 600037902 | mpp[tiflash] | table:lineitem | tiflash_task:{proc max:405.3ms, min:0s, avg: 135.1ms, p80:405.3ms, p95:405.3ms, iters:9670, tasks:3, threads:48}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:462, remote_regions:0, tot_learner_read:44ms, region_balance:{instance_num: 3, max/min: 154/154=1.000000}, delta_rows:0, delta_bytes:0, segments:924, stale_read_regions:0, tot_build_snapshot:5ms, tot_build_bitmap:434ms, tot_build_inputstream:1545ms, min_local_stream:0ms, max_local_stream:381ms, dtfile:{data_scanned_rows:600037902, data_skipped_rows:686135, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:396ms, tot_read:38304ms}}, tiflash_wait: {pipeline_queue_wait: 221ms} | keep order:false                                                                                                                                                                                                                                               | N/A     | N/A     |
+--------------------------------------------------------------------------------------------+--------------+-----------+--------------+----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+

TPCH_100 Q9 buzy situation

+--------------------------------------------------------------------------------------------+--------------+-----------+--------------+----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+
| id                                                                                         | estRows      | actRows   | task         | access object  | execution info                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | operator info                                                                                                                                                                                                                                                  | memory  | disk    |
+--------------------------------------------------------------------------------------------+--------------+-----------+--------------+----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+
| Sort_26                                                                                    | 2406.00      | 175       | root         |                | time:1m6.3s, loops:2, RU:773920.691655                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | tpch_100.nation.n_name, Column#53:desc                                                                                                                                                                                                                         | 17.5 KB | 0 Bytes |
| └─TableReader_225                                                                          | 2406.00      | 175       | root         |                | time:1m6.3s, loops:2, cop_task: {num: 24, max: 0s, min: 0s, avg: 0s, p95: 0s, copr_cache_hit_ratio: 0.00}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | MppVersion: 2, data:ExchangeSender_224                                                                                                                                                                                                                         | 7.52 KB | N/A     |
|   └─ExchangeSender_224                                                                     | 2406.00      | 175       | mpp[tiflash] |                | tiflash_task:{proc max:1m6.3s, min:1m6.3s, avg: 1m6.3s, p80:1m6.3s, p95:1m6.3s, iters:21, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | ExchangeType: PassThrough                                                                                                                                                                                                                                      | N/A     | N/A     |
|     └─Projection_29                                                                        | 2406.00      | 175       | mpp[tiflash] |                | tiflash_task:{proc max:1m6.3s, min:1m6.3s, avg: 1m6.3s, p80:1m6.3s, p95:1m6.3s, iters:21, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | tpch_100.nation.n_name, Column#53, Column#55                                                                                                                                                                                                                   | N/A     | N/A     |
|       └─Projection_220                                                                     | 2406.00      | 175       | mpp[tiflash] |                | tiflash_task:{proc max:1m6.3s, min:1m6.3s, avg: 1m6.3s, p80:1m6.3s, p95:1m6.3s, iters:21, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | Column#55, tpch_100.nation.n_name, Column#53                                                                                                                                                                                                                   | N/A     | N/A     |
|         └─HashAgg_221                                                                      | 2406.00      | 175       | mpp[tiflash] |                | tiflash_task:{proc max:1m6.3s, min:1m6.3s, avg: 1m6.3s, p80:1m6.3s, p95:1m6.3s, iters:21, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | group by:Column#72, tpch_100.nation.n_name, funcs:sum(Column#73)->Column#55, funcs:firstrow(tpch_100.nation.n_name)->tpch_100.nation.n_name, funcs:firstrow(Column#72)->Column#53, stream_count: 16                                                            | N/A     | N/A     |
|           └─ExchangeReceiver_223                                                           | 2406.00      | 525       | mpp[tiflash] |                | tiflash_task:{proc max:1m6.3s, min:1m6.3s, avg: 1m6.3s, p80:1m6.3s, p95:1m6.3s, iters:21, tasks:3, threads:48}, tiflash_wait: {minTSO_wait: 42567ms, pipeline_queue_wait: 213ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | stream_count: 16                                                                                                                                                                                                                                               | N/A     | N/A     |
|             └─ExchangeSender_222                                                           | 2406.00      | 525       | mpp[tiflash] |                | tiflash_task:{proc max:1m6.3s, min:0s, avg: 22.1s, p80:1m6.3s, p95:1m6.3s, iters:3, tasks:3, threads:3}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.nation.n_name, collate: utf8mb4_bin], stream_count: 16                                                                                                                              | N/A     | N/A     |
|               └─HashAgg_218                                                                | 2406.00      | 525       | mpp[tiflash] |                | tiflash_task:{proc max:1m6.3s, min:0s, avg: 22.1s, p80:1m6.3s, p95:1m6.3s, iters:3, tasks:3, threads:3}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | group by:Column#77, Column#78, funcs:sum(Column#76)->Column#73                                                                                                                                                                                                 | N/A     | N/A     |
|                 └─Projection_228                                                           | 475516500.66 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m6.2s, min:0s, avg: 22.1s, p80:1m6.2s, p95:1m6.2s, iters:3513, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | minus(mul(tpch_100.lineitem.l_extendedprice, minus(1, tpch_100.lineitem.l_discount)), mul(tpch_100.partsupp.ps_supplycost, tpch_100.lineitem.l_quantity))->Column#76, tpch_100.nation.n_name->Column#77, extract(YEAR, tpch_100.orders.o_orderdate)->Column#78 | N/A     | N/A     |
|                   └─Projection_207                                                         | 475516500.66 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m6.2s, min:0s, avg: 22.1s, p80:1m6.2s, p95:1m6.2s, iters:3513, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | tpch_100.lineitem.l_quantity, tpch_100.lineitem.l_extendedprice, tpch_100.lineitem.l_discount, tpch_100.partsupp.ps_supplycost, tpch_100.orders.o_orderdate, tpch_100.nation.n_name                                                                            | N/A     | N/A     |
|                     └─Projection_197                                                       | 475516500.66 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m6.2s, min:0s, avg: 22.1s, p80:1m6.2s, p95:1m6.2s, iters:3513, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | tpch_100.nation.n_name, tpch_100.lineitem.l_quantity, tpch_100.lineitem.l_extendedprice, tpch_100.lineitem.l_discount, tpch_100.partsupp.ps_supplycost, tpch_100.orders.o_orderdate, tpch_100.lineitem.l_orderkey                                              | N/A     | N/A     |
|                       └─HashJoin_196                                                       | 475516500.66 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m6.2s, min:0s, avg: 22.1s, p80:1m6.2s, p95:1m6.2s, iters:3513, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | inner join, equal:[eq(tpch_100.lineitem.l_orderkey, tpch_100.orders.o_orderkey)], stream_count: 16                                                                                                                                                             | N/A     | N/A     |
|                         ├─ExchangeReceiver_83(Build)                                       | 150000000.00 | 150000000 | mpp[tiflash] |                | tiflash_task:{proc max:46.5s, min:0s, avg: 15.5s, p80:46.5s, p95:46.5s, iters:16091, tasks:3, threads:48}, tiflash_wait: {minTSO_wait: 42596ms, pipeline_queue_wait: 1905ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | stream_count: 16                                                                                                                                                                                                                                               | N/A     | N/A     |
|                         │ └─ExchangeSender_82                                              | 150000000.00 | 150000000 | mpp[tiflash] |                | tiflash_task:{proc max:46.9s, min:0s, avg: 15.6s, p80:46.9s, p95:46.9s, iters:2400, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.orders.o_orderkey, collate: binary], stream_count: 16                                                                                                                               | N/A     | N/A     |
|                         │   └─TableFullScan_81                                             | 150000000.00 | 150000000 | mpp[tiflash] | table:orders   | tiflash_task:{proc max:44.4s, min:0s, avg: 14.8s, p80:44.4s, p95:44.4s, iters:2400, tasks:3, threads:48}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:90, remote_regions:0, tot_learner_read:56ms, region_balance:{instance_num: 3, max/min: 30/30=1.000000}, delta_rows:0, delta_bytes:0, segments:180, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:428ms, tot_build_inputstream:1486ms, min_local_stream:420ms, max_local_stream:1848ms, dtfile:{data_scanned_rows:150000000, data_skipped_rows:582144, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:602ms, tot_read:3491ms}}, tiflash_wait: {minTSO_wait: 42540ms, pipeline_queue_wait: 1842ms}        | keep order:false                                                                                                                                                                                                                                               | N/A     | N/A     |
|                         └─ExchangeReceiver_80(Probe)                                       | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m5.4s, min:0s, avg: 21.8s, p80:1m5.4s, p95:1m5.4s, iters:3513, tasks:3, threads:48}, tiflash_wait: {minTSO_wait: 42564ms, pipeline_breaker_wait: 4503ms, pipeline_queue_wait: 143ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |                                                                                                                                                                                                                                                                | N/A     | N/A     |
|                           └─ExchangeSender_79                                              | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m6.2s, min:0s, avg: 22.1s, p80:1m6.2s, p95:1m6.2s, iters:3513, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.lineitem.l_orderkey, collate: binary]                                                                                                                                               | N/A     | N/A     |
|                             └─Projection_78                                                | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m6s, min:0s, avg: 22s, p80:1m6s, p95:1m6s, iters:3513, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | tpch_100.nation.n_name, tpch_100.lineitem.l_orderkey, tpch_100.lineitem.l_quantity, tpch_100.lineitem.l_extendedprice, tpch_100.lineitem.l_discount, tpch_100.partsupp.ps_supplycost, tpch_100.lineitem.l_suppkey, tpch_100.lineitem.l_partkey                 | N/A     | N/A     |
|                               └─HashJoin_51                                                | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m6s, min:0s, avg: 22s, p80:1m6s, p95:1m6s, iters:3513, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | inner join, equal:[eq(tpch_100.lineitem.l_suppkey, tpch_100.partsupp.ps_suppkey) eq(tpch_100.lineitem.l_partkey, tpch_100.partsupp.ps_partkey)], stream_count: 16                                                                                              | N/A     | N/A     |
|                                 ├─ExchangeReceiver_77(Build)                               | 80000000.00  | 80000000  | mpp[tiflash] |                | tiflash_task:{proc max:44.8s, min:0s, avg: 14.9s, p80:44.8s, p95:44.8s, iters:8577, tasks:3, threads:48}, tiflash_wait: {minTSO_wait: 42606ms, pipeline_queue_wait: 1267ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | stream_count: 16                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                 │ └─ExchangeSender_76                                      | 80000000.00  | 80000000  | mpp[tiflash] |                | tiflash_task:{proc max:44.5s, min:0s, avg: 14.8s, p80:44.5s, p95:44.5s, iters:1291, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.partsupp.ps_suppkey, collate: binary], [name: tpch_100.partsupp.ps_partkey, collate: binary], stream_count: 16                                                                      | N/A     | N/A     |
|                                 │   └─TableFullScan_75                                     | 80000000.00  | 80000000  | mpp[tiflash] | table:partsupp | tiflash_task:{proc max:43.7s, min:0s, avg: 14.6s, p80:43.7s, p95:43.7s, iters:1291, tasks:3, threads:48}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:59, remote_regions:0, tot_learner_read:38ms, region_balance:{instance_num: 3, max/min: 20/19=1.052632}, delta_rows:0, delta_bytes:0, segments:119, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:749ms, tot_build_inputstream:1705ms, min_local_stream:0ms, max_local_stream:1108ms, dtfile:{data_scanned_rows:80000000, data_skipped_rows:0, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:930ms, tot_read:3235ms}}, tiflash_wait: {minTSO_wait: 42622ms, pipeline_queue_wait: 1107ms}                | keep order:false                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                 └─ExchangeReceiver_74(Probe)                               | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m5.1s, min:0s, avg: 21.7s, p80:1m5.1s, p95:1m5.1s, iters:3513, tasks:3, threads:48}, tiflash_wait: {minTSO_wait: 42565ms, pipeline_breaker_wait: 2575ms, pipeline_queue_wait: 205ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |                                                                                                                                                                                                                                                                | N/A     | N/A     |
|                                   └─ExchangeSender_73                                      | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m6.2s, min:0s, avg: 22.1s, p80:1m6.2s, p95:1m6.2s, iters:42643, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.lineitem.l_suppkey, collate: binary], [name: tpch_100.lineitem.l_partkey, collate: binary]                                                                                          | N/A     | N/A     |
|                                     └─Projection_72                                        | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m6s, min:0s, avg: 22s, p80:1m6s, p95:1m6s, iters:42643, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | tpch_100.nation.n_name, tpch_100.lineitem.l_orderkey, tpch_100.lineitem.l_partkey, tpch_100.lineitem.l_suppkey, tpch_100.lineitem.l_quantity, tpch_100.lineitem.l_extendedprice, tpch_100.lineitem.l_discount                                                  | N/A     | N/A     |
|                                       └─HashJoin_52                                        | 478307009.02 | 32571554  | mpp[tiflash] |                | tiflash_task:{proc max:1m5.9s, min:0s, avg: 22s, p80:1m5.9s, p95:1m5.9s, iters:42643, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | inner join, equal:[eq(tpch_100.lineitem.l_partkey, tpch_100.part.p_partkey)], stream_count: 16                                                                                                                                                                 | N/A     | N/A     |
|                                         ├─ExchangeReceiver_71(Build)                       | 16000000.00  | 1085639   | mpp[tiflash] |                | tiflash_task:{proc max:43.9s, min:0s, avg: 14.6s, p80:43.9s, p95:43.9s, iters:144, tasks:3, threads:48}, tiflash_wait: {minTSO_wait: 42571ms, pipeline_queue_wait: 921ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | stream_count: 16                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                         │ └─ExchangeSender_70                              | 16000000.00  | 1085639   | mpp[tiflash] |                | tiflash_task:{proc max:43.4s, min:0s, avg: 14.5s, p80:43.4s, p95:43.4s, iters:318, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.part.p_partkey, collate: binary], stream_count: 16                                                                                                                                  | N/A     | N/A     |
|                                         │   └─Selection_69                                 | 16000000.00  | 1085639   | mpp[tiflash] |                | tiflash_task:{proc max:43.3s, min:0s, avg: 14.4s, p80:43.3s, p95:43.3s, iters:318, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | like(tpch_100.part.p_name, "%dim%", 92)                                                                                                                                                                                                                        | N/A     | N/A     |
|                                         │     └─TableFullScan_68                           | 20000000.00  | 20000000  | mpp[tiflash] | table:part     | tiflash_task:{proc max:43.3s, min:0s, avg: 14.4s, p80:43.3s, p95:43.3s, iters:318, tasks:3, threads:48}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:13, remote_regions:0, tot_learner_read:33ms, region_balance:{instance_num: 3, max/min: 5/4=1.250000}, delta_rows:0, delta_bytes:0, segments:27, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:160ms, tot_build_inputstream:466ms, min_local_stream:263ms, max_local_stream:745ms, dtfile:{data_scanned_rows:20000000, data_skipped_rows:0, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:258ms, tot_read:3234ms}}, tiflash_wait: {minTSO_wait: 42563ms, pipeline_queue_wait: 722ms}                     | pushed down filter:empty, keep order:false                                                                                                                                                                                                                     | N/A     | N/A     |
|                                         └─ExchangeReceiver_67(Probe)                       | 600845438.27 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:1m2.8s, min:0s, avg: 20.9s, p80:1m2.8s, p95:1m2.8s, iters:42643, tasks:3, threads:48}, tiflash_wait: {minTSO_wait: 42571ms, pipeline_breaker_wait: 1339ms, pipeline_queue_wait: 5127ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |                                                                                                                                                                                                                                                                | N/A     | N/A     |
|                                           └─ExchangeSender_66                              | 600845438.27 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:1m6.1s, min:0s, avg: 22s, p80:1m6.1s, p95:1m6.1s, iters:28044, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.lineitem.l_partkey, collate: binary]                                                                                                                                                | N/A     | N/A     |
|                                             └─Projection_65                                | 600845438.27 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:1m2.1s, min:0s, avg: 20.7s, p80:1m2.1s, p95:1m2.1s, iters:28044, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | tpch_100.nation.n_name, tpch_100.lineitem.l_orderkey, tpch_100.lineitem.l_partkey, tpch_100.lineitem.l_suppkey, tpch_100.lineitem.l_quantity, tpch_100.lineitem.l_extendedprice, tpch_100.lineitem.l_discount                                                  | N/A     | N/A     |
|                                               └─HashJoin_53                                | 600845438.27 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:1m2.1s, min:0s, avg: 20.7s, p80:1m2.1s, p95:1m2.1s, iters:28044, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | inner join, equal:[eq(tpch_100.supplier.s_suppkey, tpch_100.lineitem.l_suppkey)], stream_count: 16                                                                                                                                                             | N/A     | N/A     |
|                                                 ├─ExchangeReceiver_61(Build)               | 1000000.00   | 1000000   | mpp[tiflash] |                | tiflash_task:{proc max:43.3s, min:0s, avg: 14.4s, p80:43.3s, p95:43.3s, iters:144, tasks:3, threads:48}, tiflash_wait: {minTSO_wait: 42565ms, pipeline_queue_wait: 311ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | stream_count: 16                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                                 │ └─ExchangeSender_60                      | 1000000.00   | 1000000   | mpp[tiflash] |                | tiflash_task:{proc max:43.1s, min:0s, avg: 14.4s, p80:43.1s, p95:43.1s, iters:16, tasks:3, threads:16}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.supplier.s_suppkey, collate: binary], stream_count: 16                                                                                                                              | N/A     | N/A     |
|                                                 │   └─Projection_59                        | 1000000.00   | 1000000   | mpp[tiflash] |                | tiflash_task:{proc max:43.1s, min:0s, avg: 14.4s, p80:43.1s, p95:43.1s, iters:16, tasks:3, threads:16}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | tpch_100.nation.n_name, tpch_100.supplier.s_suppkey                                                                                                                                                                                                            | N/A     | N/A     |
|                                                 │     └─HashJoin_54                        | 1000000.00   | 1000000   | mpp[tiflash] |                | tiflash_task:{proc max:43.1s, min:0s, avg: 14.4s, p80:43.1s, p95:43.1s, iters:16, tasks:3, threads:16}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | inner join, equal:[eq(tpch_100.nation.n_nationkey, tpch_100.supplier.s_nationkey)]                                                                                                                                                                             | N/A     | N/A     |
|                                                 │       ├─ExchangeReceiver_57(Build)       | 25.00        | 25        | mpp[tiflash] |                | tiflash_task:{proc max:42.8s, min:0s, avg: 14.3s, p80:42.8s, p95:42.8s, iters:1, tasks:3, threads:16}, tiflash_wait: {minTSO_wait: 42634ms, pipeline_queue_wait: 59ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |                                                                                                                                                                                                                                                                | N/A     | N/A     |
|                                                 │       │ └─ExchangeSender_56              | 25.00        | 25        | mpp[tiflash] |                | tiflash_task:{proc max:42.7s, min:0s, avg: 14.2s, p80:42.7s, p95:42.7s, iters:1, tasks:3, threads:16}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | ExchangeType: Broadcast, Compression: FAST                                                                                                                                                                                                                     | N/A     | N/A     |
|                                                 │       │   └─TableFullScan_55             | 25.00        | 25        | mpp[tiflash] | table:nation   | tiflash_task:{proc max:42.7s, min:0s, avg: 14.2s, p80:42.7s, p95:42.7s, iters:1, tasks:3, threads:16}, tiflash_scan:{mvcc_input_rows:25, mvcc_input_bytes:425, mvcc_output_rows:25, lm_skip_rows:0, local_regions:1, remote_regions:0, tot_learner_read:1ms, region_balance:{instance_num: 1, max/min: 1/1=1.000000}, delta_rows:25, delta_bytes:3134, segments:1, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:1ms, tot_build_inputstream:3ms, min_local_stream:0ms, max_local_stream:90ms, dtfile:{data_scanned_rows:25, data_skipped_rows:0, mvcc_scanned_rows:25, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:0ms, tot_read:0ms}}, tiflash_wait: {minTSO_wait: 42634ms, pipeline_queue_wait: 67ms}                                    | keep order:false                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                                 │       └─TableFullScan_58(Probe)          | 1000000.00   | 1000000   | mpp[tiflash] | table:supplier | tiflash_task:{proc max:43.1s, min:0s, avg: 14.4s, p80:43.1s, p95:43.1s, iters:16, tasks:3, threads:16}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:1, remote_regions:0, tot_learner_read:1ms, region_balance:{instance_num: 1, max/min: 1/1=1.000000}, delta_rows:0, delta_bytes:0, segments:3, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:10ms, tot_build_inputstream:30ms, min_local_stream:385ms, max_local_stream:443ms, dtfile:{data_scanned_rows:1000000, data_skipped_rows:0, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:15ms, tot_read:24ms}}, tiflash_wait: {minTSO_wait: 42634ms, pipeline_breaker_wait: 187ms, pipeline_queue_wait: 256ms} | keep order:false                                                                                                                                                                                                                                               | N/A     | N/A     |
|                                                 └─ExchangeReceiver_64(Probe)               | 600037902.00 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:53.7s, min:0s, avg: 17.9s, p80:53.7s, p95:53.7s, iters:28044, tasks:3, threads:48}, tiflash_wait: {minTSO_wait: 42629ms, pipeline_breaker_wait: 546ms, pipeline_queue_wait: 6785ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |                                                                                                                                                                                                                                                                | N/A     | N/A     |
|                                                   └─ExchangeSender_63                      | 600037902.00 | 600037902 | mpp[tiflash] |                | tiflash_task:{proc max:1m6.1s, min:0s, avg: 22s, p80:1m6.1s, p95:1m6.1s, iters:9670, tasks:3, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | ExchangeType: HashPartition, Compression: FAST, Hash Cols: [name: tpch_100.lineitem.l_suppkey, collate: binary]                                                                                                                                                | N/A     | N/A     |
|                                                     └─TableFullScan_62                     | 600037902.00 | 600037902 | mpp[tiflash] | table:lineitem | tiflash_task:{proc max:47.2s, min:0s, avg: 15.7s, p80:47.2s, p95:47.2s, iters:9670, tasks:3, threads:48}, tiflash_scan:{mvcc_input_rows:0, mvcc_input_bytes:0, mvcc_output_rows:0, lm_skip_rows:0, local_regions:462, remote_regions:0, tot_learner_read:63ms, region_balance:{instance_num: 3, max/min: 154/154=1.000000}, delta_rows:0, delta_bytes:0, segments:923, stale_read_regions:0, tot_build_snapshot:6ms, tot_build_bitmap:623ms, tot_build_inputstream:2736ms, min_local_stream:0ms, max_local_stream:4609ms, dtfile:{data_scanned_rows:600037902, data_skipped_rows:0, mvcc_scanned_rows:0, mvcc_skipped_rows:0, lm_filter_scanned_rows:0, lm_filter_skipped_rows:0, tot_rs_index_check:916ms, tot_read:29229ms}}, tiflash_wait: {minTSO_wait: 42534ms, pipeline_queue_wait: 4608ms}           | keep order:false                                                                                                                                                                                                                                               | N/A     | N/A     |
+--------------------------------------------------------------------------------------------+--------------+-----------+--------------+----------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 1, 2024
Copy link

tiprow bot commented Nov 1, 2024

Hi @yibin87. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

yibin added 2 commits November 5, 2024 10:46
@yibin87 yibin87 force-pushed the tiflash_pipeline_wait branch from 14f9bad to be2df03 Compare November 5, 2024 02:48
yibin added 2 commits November 5, 2024 11:18
Signed-off-by: yibin <[email protected]>
Signed-off-by: yibin <[email protected]>
Copy link

codecov bot commented Nov 5, 2024

Codecov Report

Attention: Patch coverage is 79.16667% with 15 lines in your changes missing coverage. Please review.

Project coverage is 73.4358%. Comparing base (aa83e4e) to head (4907d50).
Report is 76 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #57058        +/-   ##
================================================
+ Coverage   73.1861%   73.4358%   +0.2497%     
================================================
  Files          1655       1672        +17     
  Lines        456260     476360     +20100     
================================================
+ Hits         333919     349819     +15900     
- Misses       101837     105415      +3578     
- Partials      20504      21126       +622     
Flag Coverage Δ
integration 43.3875% <29.1666%> (?)
unit 72.2949% <79.1666%> (-0.1713%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.7673% <ø> (-0.1806%) ⬇️
parser ∅ <ø> (∅)
br 44.3988% <ø> (-1.3109%) ⬇️

Signed-off-by: yibin <[email protected]>
@yibin87
Copy link
Contributor Author

yibin87 commented Nov 5, 2024

/cc @windtalker @wshwsh12

@ti-chi-bot ti-chi-bot bot requested review from windtalker and wshwsh12 November 5, 2024 07:50
Signed-off-by: yibin <[email protected]>
@yibin87
Copy link
Contributor Author

yibin87 commented Nov 5, 2024

/test check-dev2

@yibin87
Copy link
Contributor Author

yibin87 commented Nov 5, 2024

/test check-dev

Copy link

tiprow bot commented Nov 5, 2024

@yibin87: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test check-dev

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@yibin87
Copy link
Contributor Author

yibin87 commented Nov 5, 2024

/test unit-test

Copy link

tiprow bot commented Nov 5, 2024

@yibin87: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

pkg/util/execdetails/execdetails.go Outdated Show resolved Hide resolved
pkg/util/execdetails/execdetails.go Outdated Show resolved Hide resolved
pkg/util/execdetails/execdetails.go Outdated Show resolved Hide resolved
Signed-off-by: yibin <[email protected]>
Copy link
Contributor

@windtalker windtalker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-chi-bot ti-chi-bot bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 13, 2024
Copy link

ti-chi-bot bot commented Nov 13, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: windtalker, wshwsh12

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [windtalker,wshwsh12]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Nov 13, 2024
Copy link

ti-chi-bot bot commented Nov 13, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-11-13 07:26:47.688633378 +0000 UTC m=+427569.879502367: ☑️ agreed by windtalker.
  • 2024-11-13 07:53:12.458388349 +0000 UTC m=+429154.649257346: ☑️ agreed by wshwsh12.

Copy link

tiprow bot commented Nov 13, 2024

@yibin87: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
fast_test_tiprow 4907d50 link true /test fast_test_tiprow

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@ti-chi-bot ti-chi-bot bot merged commit aa9b6f4 into pingcap:master Nov 13, 2024
22 of 23 checks passed
@ti-chi-bot ti-chi-bot bot added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Nov 14, 2024
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request could not be created: failed to create pull request against pingcap/tidb#release-8.5 from head ti-chi-bot:cherry-pick-57058-to-release-8.5: status code 422 not one of [201], body: {"message":"Validation Failed","errors":[{"resource":"Issue","code":"custom","field":"body","message":"body is too long (maximum is 65536 characters)"}],"documentation_url":"https://docs.github.com/rest/pulls/pulls#create-a-pull-request","status":"422"}

@ti-chi-bot ti-chi-bot bot added needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. and removed needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. labels Nov 14, 2024
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request could not be created: failed to create pull request against pingcap/tidb#release-8.5 from head ti-chi-bot:cherry-pick-57058-to-release-8.5: status code 422 not one of [201], body: {"message":"Validation Failed","errors":[{"resource":"Issue","code":"custom","field":"body","message":"body is too long (maximum is 65536 characters)"}],"documentation_url":"https://docs.github.com/rest/pulls/pulls#create-a-pull-request","status":"422"}

@yibin87
Copy link
Contributor Author

yibin87 commented Nov 14, 2024

/cherrypick release-8.5

@ti-chi-bot
Copy link
Member

@yibin87: new pull request could not be created: failed to create pull request against pingcap/tidb#release-8.5 from head ti-chi-bot:cherry-pick-57058-to-release-8.5: status code 422 not one of [201], body: {"message":"Validation Failed","errors":[{"resource":"Issue","code":"custom","field":"body","message":"body is too long (maximum is 65536 characters)"}],"documentation_url":"https://docs.github.com/rest/pulls/pulls#create-a-pull-request","status":"422"}

In response to this:

/cherrypick release-8.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@yibin87
Copy link
Contributor Author

yibin87 commented Nov 14, 2024

/cherrypick release-8.5

@ti-chi-bot
Copy link
Member

@yibin87: new pull request created to branch release-8.5: #57371.

In response to this:

/cherrypick release-8.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add tiflash wait info in execution summary
4 participants