Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor CapturedSnapshotTest method/line probes #8262

Merged
merged 1 commit into from
Jan 23, 2025
Merged

Refactor CapturedSnapshotTest method/line probes #8262

merged 1 commit into from
Jan 23, 2025

Conversation

jpbempel
Copy link
Member

What Does This Do

make a clear distinction when creating method probes and line probes Removing lines varargs to use int for helper methods Use dedicated line probe ids in preparation of
id-based line probe source matching

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@jpbempel jpbempel added tag: no release notes Changes to exclude from release notes type: refactoring comp: debugger Dynamic Instrumentation labels Jan 21, 2025
@jpbempel jpbempel requested a review from a team as a code owner January 21, 2025 15:08
@jpbempel jpbempel requested review from shatzi and evanchooly and removed request for a team January 21, 2025 15:08
@pr-commenter
Copy link

pr-commenter bot commented Jan 21, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1737651015 1737651388
end_time 2025-01-23T16:51:32 2025-01-23T16:57:44
git_branch master jpbempel/refactor-line-probe-test
git_commit_sha 64e6e86 ed62ae3
start_time 2025-01-23T16:50:16 2025-01-23T16:56:28
See matching parameters
Baseline Candidate
ci_job_id 778108617 778108617
ci_pipeline_id 53819882 53819882
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1737650443 1737650443

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 2 metrics, 8 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+214.828µs; +375.831µs] or [+2.152%; +3.764%]		 worse
[+209.663µs; +390.922µs] or [+2.065%; +3.851%]		 worse
[+186.101µs; +398.954µs] or [+1.812%; +3.885%]		 worse
[+106.431µs; +513.688µs] or [+1.005%; +4.853%]		 worse
[-3.645op/s; -0.975op/s] or [-3.754%; -1.004%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-29.969µs; +66.028µs] or [-11.068%; +24.384%]		 unstable
[-40.430µs; +81.202µs] or [-13.041%; +26.192%]		 unstable
[-47.865µs; +93.474µs] or [-14.807%; +28.915%]		 unstable
[-373.223µs; +1012.286µs] or [-63.638%; +172.604%]		 same
scenario:basic same unstable
[-9.469µs; +25.291µs] or [-3.066%; +8.190%]		 unstable
[-17.094µs; +36.250µs] or [-5.325%; +11.291%]		 unstable
[-185.107µs; +568.701µs] or [-35.670%; +109.588%]		 unstable
[-218.691op/s; +90.486op/s] or [-8.529%; +3.529%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (310.023 µs) : 278, 342
.   : milestone, 310,
basic (308.799 µs) : 295, 323
.   : milestone, 309,
loop (10.152 ms) : 10119, 10185
.   : milestone, 10152,
section candidate
noprobe (330.409 µs) : 255, 406
.   : milestone, 330,
basic (316.71 µs) : 297, 336
.   : milestone, 317,
loop (10.452 ms) : 10338, 10567
.   : milestone, 10452,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 310.023 µs [277.696 µs, 342.349 µs]
basic 308.799 µs [294.538 µs, 323.061 µs]
loop 10.152 ms [10.119 ms, 10.185 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 330.409 µs [255.07 µs, 405.748 µs]
basic 316.71 µs [297.378 µs, 336.043 µs]
loop 10.452 ms [10.338 ms, 10.567 ms]

@jpbempel jpbempel force-pushed the jpbempel/refactor-line-probe-test branch from bd4c172 to d8b9e87 Compare January 21, 2025 15:41
@pr-commenter
Copy link

pr-commenter bot commented Jan 21, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/refactor-line-probe-test
git_commit_date 1737649960 1737650443
git_commit_sha 64e6e86 ed62ae3
release_version 1.46.0-SNAPSHOT~64e6e864bc 1.46.0-SNAPSHOT~ed62ae3491
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737652904 1737652904
ci_job_id 778108611 778108611
ci_pipeline_id 53819882 53819882
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-qsn5ud32-project-304-concurrent-2-q68vdc64 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-qsn5ud32-project-304-concurrent-2-q68vdc64 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 8 performance regressions! Performance is the same for 48 metrics, 7 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:GlobalTracer worse
[+14.585ms; +23.902ms] or [+6.324%; +10.363%]		 249.882ms 230.638ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:GlobalTracer worse
[+16.068ms; +17.636ms] or [+7.003%; +7.686%]		 246.308ms 229.456ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:GlobalTracer worse
[+16.969ms; +18.657ms] or [+7.413%; +8.150%]		 246.727ms 228.914ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config worse
[+14.209µs; +63.930µs] or [+2.230%; +10.032%]		 676.362µs 637.292µs
scenario:startup:insecure-bank:tracing:GlobalTracer worse
[+15.397ms; +17.436ms] or [+6.430%; +7.281%]		 255.883ms 239.466ms
scenario:startup:petclinic:appsec:GlobalTracer worse
[+14.798ms; +19.278ms] or [+6.270%; +8.168%]		 253.058ms 236.020ms
scenario:startup:petclinic:iast:GlobalTracer worse
[+16.309ms; +18.248ms] or [+7.090%; +7.933%]		 247.303ms 230.024ms
scenario:startup:petclinic:tracing:GlobalTracer worse
[+12.040ms; +18.297ms] or [+5.001%; +7.600%]		 255.926ms 240.757ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~ed62ae3491, baseline=1.46.0-SNAPSHOT~64e6e864bc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1047723
Total [baseline] (10.5 s) : 0, 10500007
Agent [candidate] (1.056 s) : 0, 1055935
Total [candidate] (10.511 s) : 0, 10510852
section appsec
Agent [baseline] (1.176 s) : 0, 1175809
Total [baseline] (10.795 s) : 0, 10794977
Agent [candidate] (1.189 s) : 0, 1188991
Total [candidate] (10.742 s) : 0, 10741511
section iast
Agent [baseline] (1.168 s) : 0, 1168461
Total [baseline] (10.908 s) : 0, 10907797
Agent [candidate] (1.186 s) : 0, 1185790
Total [candidate] (10.943 s) : 0, 10942772
section profiling
Agent [baseline] (1.252 s) : 0, 1251760
Total [baseline] (10.841 s) : 0, 10841013
Agent [candidate] (1.255 s) : 0, 1254790
Total [candidate] (10.823 s) : 0, 10823133
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.048 s -
Agent appsec 1.176 s 128.086 ms (12.2%)
Agent iast 1.168 s 120.737 ms (11.5%)
Agent profiling 1.252 s 204.036 ms (19.5%)
Total tracing 10.5 s -
Total appsec 10.795 s 294.971 ms (2.8%)
Total iast 10.908 s 407.79 ms (3.9%)
Total profiling 10.841 s 341.006 ms (3.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.056 s -
Agent appsec 1.189 s 133.055 ms (12.6%)
Agent iast 1.186 s 129.855 ms (12.3%)
Agent profiling 1.255 s 198.855 ms (18.8%)
Total tracing 10.511 s -
Total appsec 10.742 s 230.659 ms (2.2%)
Total iast 10.943 s 431.92 ms (4.1%)
Total profiling 10.823 s 312.281 ms (3.0%) 
gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~ed62ae3491, baseline=1.46.0-SNAPSHOT~64e6e864bc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (723.013 ms) : 0, 723013
BytebuddyAgent [candidate] (715.417 ms) : 0, 715417
GlobalTracer [baseline] (240.757 ms) : 0, 240757
GlobalTracer [candidate] (255.926 ms) : 0, 255926
AppSec [baseline] (56.617 ms) : 0, 56617
AppSec [candidate] (55.567 ms) : 0, 55567
Remote Config [baseline] (735.64 µs) : 0, 736
Remote Config [candidate] (743.215 µs) : 0, 743
Telemetry [baseline] (11.465 ms) : 0, 11465
Telemetry [candidate] (13.215 ms) : 0, 13215
section appsec
BytebuddyAgent [baseline] (730.04 ms) : 0, 730040
BytebuddyAgent [candidate] (731.419 ms) : 0, 731419
GlobalTracer [baseline] (236.02 ms) : 0, 236020
GlobalTracer [candidate] (253.058 ms) : 0, 253058
AppSec [baseline] (175.64 ms) : 0, 175640
AppSec [candidate] (170.972 ms) : 0, 170972
IAST [baseline] (19.942 ms) : 0, 19942
IAST [candidate] (19.36 ms) : 0, 19360
Remote Config [baseline] (671.945 µs) : 0, 672
Remote Config [candidate] (652.516 µs) : 0, 653
Telemetry [baseline] (8.579 ms) : 0, 8579
Telemetry [candidate] (8.171 ms) : 0, 8171
section iast
BytebuddyAgent [baseline] (834.601 ms) : 0, 834601
BytebuddyAgent [candidate] (834.396 ms) : 0, 834396
GlobalTracer [baseline] (230.024 ms) : 0, 230024
GlobalTracer [candidate] (247.303 ms) : 0, 247303
AppSec [baseline] (58.21 ms) : 0, 58210
AppSec [candidate] (58.1 ms) : 0, 58100
IAST [baseline] (21.289 ms) : 0, 21289
IAST [candidate] (21.48 ms) : 0, 21480
Remote Config [baseline] (665.228 µs) : 0, 665
Remote Config [candidate] (663.194 µs) : 0, 663
Telemetry [baseline] (8.637 ms) : 0, 8637
Telemetry [candidate] (8.801 ms) : 0, 8801
section profiling
BytebuddyAgent [baseline] (702.48 ms) : 0, 702480
BytebuddyAgent [candidate] (704.346 ms) : 0, 704346
GlobalTracer [baseline] (347.883 ms) : 0, 347883
GlobalTracer [candidate] (349.378 ms) : 0, 349378
AppSec [baseline] (54.792 ms) : 0, 54792
AppSec [candidate] (54.151 ms) : 0, 54151
Remote Config [baseline] (708.607 µs) : 0, 709
Remote Config [candidate] (717.232 µs) : 0, 717
Telemetry [baseline] (8.754 ms) : 0, 8754
Telemetry [candidate] (8.78 ms) : 0, 8780
ProfilingAgent [baseline] (95.277 ms) : 0, 95277
ProfilingAgent [candidate] (95.38 ms) : 0, 95380
Profiling [baseline] (95.301 ms) : 0, 95301
Profiling [candidate] (95.404 ms) : 0, 95404
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~ed62ae3491, baseline=1.46.0-SNAPSHOT~64e6e864bc

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.041 s) : 0, 1041256
Total [baseline] (8.669 s) : 0, 8668928
Agent [candidate] (1.058 s) : 0, 1057799
Total [candidate] (8.658 s) : 0, 8657589
section iast
Agent [baseline] (1.173 s) : 0, 1172789
Total [baseline] (9.235 s) : 0, 9235245
Agent [candidate] (1.194 s) : 0, 1194170
Total [candidate] (9.298 s) : 0, 9298368
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.167 s) : 0, 1166999
Total [baseline] (9.158 s) : 0, 9157997
Agent [candidate] (1.183 s) : 0, 1183192
Total [candidate] (9.178 s) : 0, 9177758
section iast_TELEMETRY_OFF
Agent [baseline] (1.161 s) : 0, 1160656
Total [baseline] (9.2 s) : 0, 9200054
Agent [candidate] (1.181 s) : 0, 1181367
Total [candidate] (9.198 s) : 0, 9198189
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.041 s -
Agent iast 1.173 s 131.533 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.167 s 125.742 ms (12.1%)
Agent iast_TELEMETRY_OFF 1.161 s 119.4 ms (11.5%)
Total tracing 8.669 s -
Total iast 9.235 s 566.317 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.158 s 489.068 ms (5.6%)
Total iast_TELEMETRY_OFF 9.2 s 531.126 ms (6.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.058 s -
Agent iast 1.194 s 136.371 ms (12.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.183 s 125.393 ms (11.9%)
Agent iast_TELEMETRY_OFF 1.181 s 123.569 ms (11.7%)
Total tracing 8.658 s -
Total iast 9.298 s 640.779 ms (7.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.178 s 520.169 ms (6.0%)
Total iast_TELEMETRY_OFF 9.198 s 540.6 ms (6.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~ed62ae3491, baseline=1.46.0-SNAPSHOT~64e6e864bc

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.086 ms) : 0, 716086
BytebuddyAgent [candidate] (714.133 ms) : 0, 714133
GlobalTracer [baseline] (239.466 ms) : 0, 239466
GlobalTracer [candidate] (255.883 ms) : 0, 255883
AppSec [baseline] (55.397 ms) : 0, 55397
AppSec [candidate] (56.254 ms) : 0, 56254
Remote Config [baseline] (712.826 µs) : 0, 713
Remote Config [candidate] (753.209 µs) : 0, 753
Telemetry [baseline] (14.59 ms) : 0, 14590
Telemetry [candidate] (15.731 ms) : 0, 15731
section iast
BytebuddyAgent [baseline] (837.833 ms) : 0, 837833
BytebuddyAgent [candidate] (838.623 ms) : 0, 838623
GlobalTracer [baseline] (230.638 ms) : 0, 230638
GlobalTracer [candidate] (249.882 ms) : 0, 249882
AppSec [baseline] (58.516 ms) : 0, 58516
AppSec [candidate] (58.998 ms) : 0, 58998
IAST [baseline] (21.217 ms) : 0, 21217
IAST [candidate] (21.853 ms) : 0, 21853
Remote Config [baseline] (648.007 µs) : 0, 648
Remote Config [candidate] (690.497 µs) : 0, 690
Telemetry [baseline] (8.82 ms) : 0, 8820
Telemetry [candidate] (9.037 ms) : 0, 9037
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (833.701 ms) : 0, 833701
BytebuddyAgent [candidate] (832.959 ms) : 0, 832959
GlobalTracer [baseline] (229.456 ms) : 0, 229456
GlobalTracer [candidate] (246.308 ms) : 0, 246308
AppSec [baseline] (58.098 ms) : 0, 58098
AppSec [candidate] (58.009 ms) : 0, 58009
IAST [baseline] (21.271 ms) : 0, 21271
IAST [candidate] (21.569 ms) : 0, 21569
Remote Config [baseline] (639.827 µs) : 0, 640
Remote Config [candidate] (657.522 µs) : 0, 658
Telemetry [baseline] (8.737 ms) : 0, 8737
Telemetry [candidate] (8.728 ms) : 0, 8728
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (828.765 ms) : 0, 828765
BytebuddyAgent [candidate] (830.825 ms) : 0, 830825
GlobalTracer [baseline] (228.914 ms) : 0, 228914
GlobalTracer [candidate] (246.727 ms) : 0, 246727
AppSec [baseline] (57.905 ms) : 0, 57905
AppSec [candidate] (58.473 ms) : 0, 58473
IAST [baseline] (20.884 ms) : 0, 20884
IAST [candidate] (20.962 ms) : 0, 20962
Remote Config [baseline] (637.292 µs) : 0, 637
Remote Config [candidate] (676.362 µs) : 0, 676
Telemetry [baseline] (8.597 ms) : 0, 8597
Telemetry [candidate] (8.689 ms) : 0, 8689
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-23T16:51:40 2025-01-23T16:58:45
git_branch master jpbempel/refactor-line-probe-test
git_commit_date 1737649960 1737650443
git_commit_sha 64e6e86 ed62ae3
release_version 1.46.0-SNAPSHOT~64e6e864bc 1.46.0-SNAPSHOT~ed62ae3491
start_time 2025-01-23T16:51:26 2025-01-23T16:58:31
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737651885 1737651885
ci_job_id 778108612 778108612
ci_pipeline_id 53819882 53819882
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-qsn5ud32-project-304-concurrent-3-kbnvzeu1 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-qsn5ud32-project-304-concurrent-3-kbnvzeu1 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~ed62ae3491, baseline=1.46.0-SNAPSHOT~64e6e864bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (377.935 µs) : 356, 399
.   : milestone, 378,
iast (507.806 µs) : 486, 530
.   : milestone, 508,
iast_FULL (739.564 µs) : 717, 763
.   : milestone, 740,
iast_GLOBAL (558.037 µs) : 535, 582
.   : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (511.097 µs) : 488, 534
.   : milestone, 511,
iast_INACTIVE (469.731 µs) : 447, 492
.   : milestone, 470,
iast_TELEMETRY_OFF (495.119 µs) : 472, 518
.   : milestone, 495,
tracing (451.876 µs) : 431, 472
.   : milestone, 452,
section candidate
no_agent (385.62 µs) : 366, 405
.   : milestone, 386,
iast (506.17 µs) : 484, 528
.   : milestone, 506,
iast_FULL (740.983 µs) : 719, 763
.   : milestone, 741,
iast_GLOBAL (562.086 µs) : 540, 584
.   : milestone, 562,
iast_HARDCODED_SECRET_DISABLED (512.723 µs) : 491, 535
.   : milestone, 513,
iast_INACTIVE (465.122 µs) : 444, 487
.   : milestone, 465,
iast_TELEMETRY_OFF (495.542 µs) : 474, 517
.   : milestone, 496,
tracing (453.796 µs) : 433, 475
.   : milestone, 454,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 377.935 µs [356.467 µs, 399.404 µs] -
iast 507.806 µs [486.045 µs, 529.566 µs] 129.87 µs (34.4%)
iast_FULL 739.564 µs [716.506 µs, 762.622 µs] 361.629 µs (95.7%)
iast_GLOBAL 558.037 µs [534.525 µs, 581.549 µs] 180.102 µs (47.7%)
iast_HARDCODED_SECRET_DISABLED 511.097 µs [487.996 µs, 534.198 µs] 133.162 µs (35.2%)
iast_INACTIVE 469.731 µs [447.231 µs, 492.23 µs] 91.795 µs (24.3%)
iast_TELEMETRY_OFF 495.119 µs [471.971 µs, 518.267 µs] 117.184 µs (31.0%)
tracing 451.876 µs [431.361 µs, 472.391 µs] 73.941 µs (19.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.62 µs [365.97 µs, 405.27 µs] -
iast 506.17 µs [484.376 µs, 527.965 µs] 120.55 µs (31.3%)
iast_FULL 740.983 µs [718.889 µs, 763.077 µs] 355.363 µs (92.2%)
iast_GLOBAL 562.086 µs [539.904 µs, 584.268 µs] 176.466 µs (45.8%)
iast_HARDCODED_SECRET_DISABLED 512.723 µs [490.865 µs, 534.581 µs] 127.103 µs (33.0%)
iast_INACTIVE 465.122 µs [443.724 µs, 486.521 µs] 79.502 µs (20.6%)
iast_TELEMETRY_OFF 495.542 µs [473.877 µs, 517.207 µs] 109.922 µs (28.5%)
tracing 453.796 µs [432.968 µs, 474.624 µs] 68.176 µs (17.7%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~ed62ae3491, baseline=1.46.0-SNAPSHOT~64e6e864bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.365 ms) : 1346, 1383
.   : milestone, 1365,
appsec (1.739 ms) : 1715, 1763
.   : milestone, 1739,
appsec_no_iast (1.738 ms) : 1715, 1762
.   : milestone, 1738,
iast (1.508 ms) : 1483, 1532
.   : milestone, 1508,
profiling (1.553 ms) : 1528, 1578
.   : milestone, 1553,
tracing (1.51 ms) : 1486, 1535
.   : milestone, 1510,
section candidate
no_agent (1.347 ms) : 1327, 1367
.   : milestone, 1347,
appsec (1.765 ms) : 1742, 1789
.   : milestone, 1765,
appsec_no_iast (1.782 ms) : 1758, 1805
.   : milestone, 1782,
iast (1.544 ms) : 1519, 1569
.   : milestone, 1544,
profiling (1.52 ms) : 1497, 1543
.   : milestone, 1520,
tracing (1.51 ms) : 1486, 1534
.   : milestone, 1510,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.365 ms [1.346 ms, 1.383 ms] -
appsec 1.739 ms [1.715 ms, 1.763 ms] 374.388 µs (27.4%)
appsec_no_iast 1.738 ms [1.715 ms, 1.762 ms] 373.734 µs (27.4%)
iast 1.508 ms [1.483 ms, 1.532 ms] 142.86 µs (10.5%)
profiling 1.553 ms [1.528 ms, 1.578 ms] 188.134 µs (13.8%)
tracing 1.51 ms [1.486 ms, 1.535 ms] 145.486 µs (10.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.347 ms [1.327 ms, 1.367 ms] -
appsec 1.765 ms [1.742 ms, 1.789 ms] 418.05 µs (31.0%)
appsec_no_iast 1.782 ms [1.758 ms, 1.805 ms] 434.469 µs (32.3%)
iast 1.544 ms [1.519 ms, 1.569 ms] 197.027 µs (14.6%)
profiling 1.52 ms [1.497 ms, 1.543 ms] 172.75 µs (12.8%)
tracing 1.51 ms [1.486 ms, 1.534 ms] 163.12 µs (12.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/refactor-line-probe-test
git_commit_date 1737649960 1737650443
git_commit_sha 64e6e86 ed62ae3
release_version 1.46.0-SNAPSHOT~64e6e864bc 1.46.0-SNAPSHOT~ed62ae3491
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1737652473 1737652473
ci_job_id 778108613 778108613
ci_pipeline_id 53819882 53819882
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-xugqgpm8-project-304-concurrent-0-edl3t3af 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-xugqgpm8-project-304-concurrent-0-edl3t3af 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~ed62ae3491, baseline=1.46.0-SNAPSHOT~64e6e864bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.468 ms) : 1456, 1479
.   : milestone, 1468,
appsec (2.345 ms) : 2301, 2389
.   : milestone, 2345,
iast (2.096 ms) : 2042, 2151
.   : milestone, 2096,
iast_GLOBAL (2.129 ms) : 2074, 2184
.   : milestone, 2129,
profiling (1.962 ms) : 1918, 2007
.   : milestone, 1962,
tracing (1.933 ms) : 1891, 1975
.   : milestone, 1933,
section candidate
no_agent (1.467 ms) : 1455, 1478
.   : milestone, 1467,
appsec (2.34 ms) : 2297, 2383
.   : milestone, 2340,
iast (2.092 ms) : 2038, 2147
.   : milestone, 2092,
iast_GLOBAL (2.132 ms) : 2078, 2187
.   : milestone, 2132,
profiling (1.982 ms) : 1937, 2027
.   : milestone, 1982,
tracing (1.942 ms) : 1900, 1985
.   : milestone, 1942,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.468 ms [1.456 ms, 1.479 ms] -
appsec 2.345 ms [2.301 ms, 2.389 ms] 877.252 µs (59.8%)
iast 2.096 ms [2.042 ms, 2.151 ms] 628.502 µs (42.8%)
iast_GLOBAL 2.129 ms [2.074 ms, 2.184 ms] 661.337 µs (45.1%)
profiling 1.962 ms [1.918 ms, 2.007 ms] 494.487 µs (33.7%)
tracing 1.933 ms [1.891 ms, 1.975 ms] 464.974 µs (31.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.467 ms [1.455 ms, 1.478 ms] -
appsec 2.34 ms [2.297 ms, 2.383 ms] 873.455 µs (59.6%)
iast 2.092 ms [2.038 ms, 2.147 ms] 625.602 µs (42.7%)
iast_GLOBAL 2.132 ms [2.078 ms, 2.187 ms] 665.681 µs (45.4%)
profiling 1.982 ms [1.937 ms, 2.027 ms] 515.365 µs (35.1%)
tracing 1.942 ms [1.9 ms, 1.985 ms] 475.691 µs (32.4%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~ed62ae3491, baseline=1.46.0-SNAPSHOT~64e6e864bc
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.787 s) : 14787000, 14787000
.   : milestone, 14787000,
appsec (15.153 s) : 15153000, 15153000
.   : milestone, 15153000,
iast (18.493 s) : 18493000, 18493000
.   : milestone, 18493000,
iast_GLOBAL (17.978 s) : 17978000, 17978000
.   : milestone, 17978000,
profiling (15.004 s) : 15004000, 15004000
.   : milestone, 15004000,
tracing (15.041 s) : 15041000, 15041000
.   : milestone, 15041000,
section candidate
no_agent (15.407 s) : 15407000, 15407000
.   : milestone, 15407000,
appsec (15.199 s) : 15199000, 15199000
.   : milestone, 15199000,
iast (18.667 s) : 18667000, 18667000
.   : milestone, 18667000,
iast_GLOBAL (18.111 s) : 18111000, 18111000
.   : milestone, 18111000,
profiling (15.554 s) : 15554000, 15554000
.   : milestone, 15554000,
tracing (15.024 s) : 15024000, 15024000
.   : milestone, 15024000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.787 s [14.787 s, 14.787 s] -
appsec 15.153 s [15.153 s, 15.153 s] 366.0 ms (2.5%)
iast 18.493 s [18.493 s, 18.493 s] 3.706 s (25.1%)
iast_GLOBAL 17.978 s [17.978 s, 17.978 s] 3.191 s (21.6%)
profiling 15.004 s [15.004 s, 15.004 s] 217.0 ms (1.5%)
tracing 15.041 s [15.041 s, 15.041 s] 254.0 ms (1.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.407 s [15.407 s, 15.407 s] -
appsec 15.199 s [15.199 s, 15.199 s] -208.0 ms (-1.4%)
iast 18.667 s [18.667 s, 18.667 s] 3.26 s (21.2%)
iast_GLOBAL 18.111 s [18.111 s, 18.111 s] 2.704 s (17.6%)
profiling 15.554 s [15.554 s, 15.554 s] 147.0 ms (1.0%)
tracing 15.024 s [15.024 s, 15.024 s] -383.0 ms (-2.5%)

@evanchooly
Copy link
Contributor

I don't see the part that scans the source file for the ID tag to determine the line. Was that supposed to be part of this or is it still pending?

@jpbempel
Copy link
Member Author

I don't see the part that scans the source file for the ID tag to determine the line. Was that supposed to be part of this or is it still pending?

next PR, it's just a preparation for that

@jpbempel
Refactor CapturedSnapshotTest method/line probes
ed62ae3 
make a clear distinction when creating method probes and line probes
Removing lines varargs to use int for helper methods
Use dedicated line probe ids in preparation of
id-based line probe source matching
@jpbempel jpbempel force-pushed the jpbempel/refactor-line-probe-test branch from d8b9e87 to ed62ae3 Compare January 23, 2025 16:40
@jpbempel jpbempel added comp: debugger Dynamic Instrumentation and removed comp: debugger Dynamic Instrumentation labels Jan 23, 2025
@jpbempel jpbempel merged commit afbea0b into master Jan 23, 2025
178 of 179 checks passed
@jpbempel jpbempel deleted the jpbempel/refactor-line-probe-test branch January 23, 2025 17:28
@github-actions github-actions bot added this to the 1.46.0 milestone Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@evanchooly evanchooly evanchooly approved these changes

@shatzi shatzi Awaiting requested review from shatzi shatzi is a code owner automatically assigned from DataDog/debugger-java

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation tag: no release notes Changes to exclude from release notes type: refactoring
Projects
None yet
Milestone
1.46.0
Development

Successfully merging this pull request may close these issues.
2 participants
@jpbempel @evanchooly