Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Decouple gradle JDK from compile/test JDK #7506

Draft
wants to merge 21 commits into
base: master
Choose a base branch
from
Draft

Decouple gradle JDK from compile/test JDK #7506

wants to merge 21 commits into from

Conversation

smola
Copy link
Member

@smola smola commented Aug 26, 2024 •
edited by jira bot
Loading

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: APMJAVA-1342

@smola smola added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Aug 26, 2024
@smola smola force-pushed the smola/gradle-java11 branch from dc1fa6e to 702851a Compare August 30, 2024 20:49
@pr-commenter
Copy link

pr-commenter bot commented Aug 30, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737969316 1737978808
git_commit_sha 19da739 4812847
release_version 1.46.0-SNAPSHOT~19da739a7c 1.46.0-SNAPSHOT~4812847170
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737981190 1737981190
ci_job_id 781276285 781276285
ci_pipeline_id 54044438 54044438
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-kcmwffss-project-304-concurrent-2-0oiw0kwk 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-kcmwffss-project-304-concurrent-2-0oiw0kwk 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 8 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~4812847170, baseline=1.46.0-SNAPSHOT~19da739a7c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.036 s) : 0, 1035933
Total [baseline] (10.494 s) : 0, 10493736
Agent [candidate] (1.038 s) : 0, 1038053
Total [candidate] (10.49 s) : 0, 10489644
section appsec
Agent [baseline] (1.188 s) : 0, 1188014
Total [baseline] (10.832 s) : 0, 10831726
Agent [candidate] (1.181 s) : 0, 1181358
Total [candidate] (10.807 s) : 0, 10806768
section iast
Agent [baseline] (1.177 s) : 0, 1176627
Total [baseline] (11.052 s) : 0, 11052259
Agent [candidate] (1.168 s) : 0, 1168225
Total [candidate] (10.948 s) : 0, 10947682
section profiling
Agent [baseline] (1.255 s) : 0, 1255333
Total [baseline] (10.874 s) : 0, 10873830
Agent [candidate] (1.255 s) : 0, 1255452
Total [candidate] (10.892 s) : 0, 10891782
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.036 s -
Agent appsec 1.188 s 152.082 ms (14.7%)
Agent iast 1.177 s 140.695 ms (13.6%)
Agent profiling 1.255 s 219.4 ms (21.2%)
Total tracing 10.494 s -
Total appsec 10.832 s 337.991 ms (3.2%)
Total iast 11.052 s 558.523 ms (5.3%)
Total profiling 10.874 s 380.094 ms (3.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.038 s -
Agent appsec 1.181 s 143.305 ms (13.8%)
Agent iast 1.168 s 130.172 ms (12.5%)
Agent profiling 1.255 s 217.399 ms (20.9%)
Total tracing 10.49 s -
Total appsec 10.807 s 317.124 ms (3.0%)
Total iast 10.948 s 458.038 ms (4.4%)
Total profiling 10.892 s 402.138 ms (3.8%) 
gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~4812847170, baseline=1.46.0-SNAPSHOT~19da739a7c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (714.216 ms) : 0, 714216
BytebuddyAgent [candidate] (715.07 ms) : 0, 715070
GlobalTracer [baseline] (238.963 ms) : 0, 238963
GlobalTracer [candidate] (239.416 ms) : 0, 239416
AppSec [baseline] (55.651 ms) : 0, 55651
AppSec [candidate] (55.506 ms) : 0, 55506
Remote Config [baseline] (709.668 µs) : 0, 710
Remote Config [candidate] (714.733 µs) : 0, 715
Telemetry [baseline] (11.503 ms) : 0, 11503
Telemetry [candidate] (12.278 ms) : 0, 12278
section appsec
BytebuddyAgent [baseline] (737.153 ms) : 0, 737153
BytebuddyAgent [candidate] (733.697 ms) : 0, 733697
GlobalTracer [baseline] (238.877 ms) : 0, 238877
GlobalTracer [candidate] (237.074 ms) : 0, 237074
AppSec [baseline] (177.87 ms) : 0, 177870
AppSec [candidate] (176.6 ms) : 0, 176600
Remote Config [baseline] (672.731 µs) : 0, 673
Remote Config [candidate] (668.571 µs) : 0, 669
Telemetry [baseline] (8.673 ms) : 0, 8673
Telemetry [candidate] (8.617 ms) : 0, 8617
IAST [baseline] (19.412 ms) : 0, 19412
IAST [candidate] (19.244 ms) : 0, 19244
section iast
BytebuddyAgent [baseline] (840.349 ms) : 0, 840349
BytebuddyAgent [candidate] (834.047 ms) : 0, 834047
GlobalTracer [baseline] (231.476 ms) : 0, 231476
GlobalTracer [candidate] (230.126 ms) : 0, 230126
AppSec [baseline] (58.461 ms) : 0, 58461
AppSec [candidate] (57.539 ms) : 0, 57539
Remote Config [baseline] (676.295 µs) : 0, 676
Remote Config [candidate] (655.954 µs) : 0, 656
Telemetry [baseline] (8.821 ms) : 0, 8821
Telemetry [candidate] (8.698 ms) : 0, 8698
IAST [baseline] (21.695 ms) : 0, 21695
IAST [candidate] (22.11 ms) : 0, 22110
section profiling
BytebuddyAgent [baseline] (704.065 ms) : 0, 704065
BytebuddyAgent [candidate] (704.541 ms) : 0, 704541
GlobalTracer [baseline] (349.355 ms) : 0, 349355
GlobalTracer [candidate] (349.321 ms) : 0, 349321
AppSec [baseline] (54.833 ms) : 0, 54833
AppSec [candidate] (54.298 ms) : 0, 54298
Remote Config [baseline] (719.284 µs) : 0, 719
Remote Config [candidate] (743.217 µs) : 0, 743
Telemetry [baseline] (8.736 ms) : 0, 8736
Telemetry [candidate] (8.859 ms) : 0, 8859
ProfilingAgent [baseline] (95.647 ms) : 0, 95647
ProfilingAgent [candidate] (95.732 ms) : 0, 95732
Profiling [baseline] (95.673 ms) : 0, 95673
Profiling [candidate] (95.756 ms) : 0, 95756
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~4812847170, baseline=1.46.0-SNAPSHOT~19da739a7c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1046541
Total [baseline] (8.654 s) : 0, 8654399
Agent [candidate] (1.035 s) : 0, 1035102
Total [candidate] (8.657 s) : 0, 8657177
section iast
Agent [baseline] (1.167 s) : 0, 1167051
Total [baseline] (9.248 s) : 0, 9247804
Agent [candidate] (1.178 s) : 0, 1178309
Total [candidate] (9.295 s) : 0, 9295121
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.165 s) : 0, 1165223
Total [baseline] (9.196 s) : 0, 9196414
Agent [candidate] (1.178 s) : 0, 1178323
Total [candidate] (9.201 s) : 0, 9201322
section iast_TELEMETRY_OFF
Agent [baseline] (1.161 s) : 0, 1160980
Total [baseline] (9.208 s) : 0, 9207653
Agent [candidate] (1.161 s) : 0, 1160880
Total [candidate] (9.236 s) : 0, 9236074
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent iast 1.167 s 120.511 ms (11.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.165 s 118.682 ms (11.3%)
Agent iast_TELEMETRY_OFF 1.161 s 114.439 ms (10.9%)
Total tracing 8.654 s -
Total iast 9.248 s 593.405 ms (6.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.196 s 542.015 ms (6.3%)
Total iast_TELEMETRY_OFF 9.208 s 553.254 ms (6.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.035 s -
Agent iast 1.178 s 143.207 ms (13.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.178 s 143.221 ms (13.8%)
Agent iast_TELEMETRY_OFF 1.161 s 125.778 ms (12.2%)
Total tracing 8.657 s -
Total iast 9.295 s 637.944 ms (7.4%)
Total iast_HARDCODED_SECRET_DISABLED 9.201 s 544.146 ms (6.3%)
Total iast_TELEMETRY_OFF 9.236 s 578.897 ms (6.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~4812847170, baseline=1.46.0-SNAPSHOT~19da739a7c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (720.107 ms) : 0, 720107
BytebuddyAgent [candidate] (713.644 ms) : 0, 713644
GlobalTracer [baseline] (240.239 ms) : 0, 240239
GlobalTracer [candidate] (238.846 ms) : 0, 238846
AppSec [baseline] (55.705 ms) : 0, 55705
AppSec [candidate] (55.382 ms) : 0, 55382
Remote Config [baseline] (729.651 µs) : 0, 730
Remote Config [candidate] (714.327 µs) : 0, 714
Telemetry [baseline] (14.678 ms) : 0, 14678
Telemetry [candidate] (11.519 ms) : 0, 11519
section iast
BytebuddyAgent [baseline] (833.365 ms) : 0, 833365
BytebuddyAgent [candidate] (841.946 ms) : 0, 841946
GlobalTracer [baseline] (229.442 ms) : 0, 229442
GlobalTracer [candidate] (231.507 ms) : 0, 231507
AppSec [baseline] (58.259 ms) : 0, 58259
AppSec [candidate] (57.824 ms) : 0, 57824
Remote Config [baseline] (663.612 µs) : 0, 664
Remote Config [candidate] (671.888 µs) : 0, 672
Telemetry [baseline] (8.851 ms) : 0, 8851
Telemetry [candidate] (8.807 ms) : 0, 8807
IAST [baseline] (21.337 ms) : 0, 21337
IAST [candidate] (22.399 ms) : 0, 22399
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (831.52 ms) : 0, 831520
BytebuddyAgent [candidate] (841.154 ms) : 0, 841154
GlobalTracer [baseline] (229.588 ms) : 0, 229588
GlobalTracer [candidate] (231.258 ms) : 0, 231258
AppSec [baseline] (57.54 ms) : 0, 57540
AppSec [candidate] (59.47 ms) : 0, 59470
Remote Config [baseline] (671.689 µs) : 0, 672
Remote Config [candidate] (661.437 µs) : 0, 661
Telemetry [baseline] (8.759 ms) : 0, 8759
Telemetry [candidate] (8.952 ms) : 0, 8952
IAST [baseline] (22.147 ms) : 0, 22147
IAST [candidate] (21.495 ms) : 0, 21495
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (828.424 ms) : 0, 828424
BytebuddyAgent [candidate] (828.431 ms) : 0, 828431
GlobalTracer [baseline] (229.523 ms) : 0, 229523
GlobalTracer [candidate] (228.755 ms) : 0, 228755
AppSec [baseline] (58.104 ms) : 0, 58104
AppSec [candidate] (58.502 ms) : 0, 58502
Remote Config [baseline] (654.434 µs) : 0, 654
Remote Config [candidate] (644.477 µs) : 0, 644
Telemetry [baseline] (8.675 ms) : 0, 8675
Telemetry [candidate] (8.627 ms) : 0, 8627
IAST [baseline] (20.636 ms) : 0, 20636
IAST [candidate] (20.861 ms) : 0, 20861
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-27T12:05:01 2025-01-27T12:12:05
git_branch master smola/gradle-java11
git_commit_date 1737969316 1737978808
git_commit_sha 19da739 4812847
release_version 1.46.0-SNAPSHOT~19da739a7c 1.46.0-SNAPSHOT~4812847170
start_time 2025-01-27T12:04:46 2025-01-27T12:11:51
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737980284 1737980284
ci_job_id 781276286 781276286
ci_pipeline_id 54044438 54044438
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-kcmwffss-project-304-concurrent-3-56h5j9bw 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-kcmwffss-project-304-concurrent-3-56h5j9bw 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~4812847170, baseline=1.46.0-SNAPSHOT~19da739a7c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.356 ms) : 1337, 1375
.   : milestone, 1356,
appsec (1.746 ms) : 1722, 1770
.   : milestone, 1746,
appsec_no_iast (1.749 ms) : 1724, 1773
.   : milestone, 1749,
iast (1.521 ms) : 1497, 1546
.   : milestone, 1521,
profiling (1.524 ms) : 1499, 1549
.   : milestone, 1524,
tracing (1.513 ms) : 1489, 1537
.   : milestone, 1513,
section candidate
no_agent (1.356 ms) : 1336, 1375
.   : milestone, 1356,
appsec (1.745 ms) : 1721, 1769
.   : milestone, 1745,
appsec_no_iast (1.75 ms) : 1727, 1773
.   : milestone, 1750,
iast (1.504 ms) : 1479, 1528
.   : milestone, 1504,
profiling (1.523 ms) : 1498, 1548
.   : milestone, 1523,
tracing (1.492 ms) : 1467, 1518
.   : milestone, 1492,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.356 ms [1.337 ms, 1.375 ms] -
appsec 1.746 ms [1.722 ms, 1.77 ms] 390.057 µs (28.8%)
appsec_no_iast 1.749 ms [1.724 ms, 1.773 ms] 392.665 µs (29.0%)
iast 1.521 ms [1.497 ms, 1.546 ms] 165.561 µs (12.2%)
profiling 1.524 ms [1.499 ms, 1.549 ms] 168.308 µs (12.4%)
tracing 1.513 ms [1.489 ms, 1.537 ms] 157.494 µs (11.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.356 ms [1.336 ms, 1.375 ms] -
appsec 1.745 ms [1.721 ms, 1.769 ms] 389.178 µs (28.7%)
appsec_no_iast 1.75 ms [1.727 ms, 1.773 ms] 394.374 µs (29.1%)
iast 1.504 ms [1.479 ms, 1.528 ms] 147.956 µs (10.9%)
profiling 1.523 ms [1.498 ms, 1.548 ms] 167.164 µs (12.3%)
tracing 1.492 ms [1.467 ms, 1.518 ms] 136.666 µs (10.1%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~4812847170, baseline=1.46.0-SNAPSHOT~19da739a7c
    dateFormat X
    axisFormat %s
section baseline
no_agent (380.043 µs) : 360, 400
.   : milestone, 380,
iast (515.799 µs) : 492, 540
.   : milestone, 516,
iast_FULL (743.131 µs) : 720, 766
.   : milestone, 743,
iast_GLOBAL (558.388 µs) : 534, 583
.   : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (514.36 µs) : 491, 538
.   : milestone, 514,
iast_INACTIVE (462.576 µs) : 441, 484
.   : milestone, 463,
iast_TELEMETRY_OFF (497.458 µs) : 474, 521
.   : milestone, 497,
tracing (457.982 µs) : 437, 479
.   : milestone, 458,
section candidate
no_agent (375.572 µs) : 355, 396
.   : milestone, 376,
iast (508.399 µs) : 486, 531
.   : milestone, 508,
iast_FULL (736.116 µs) : 714, 758
.   : milestone, 736,
iast_GLOBAL (550.245 µs) : 529, 572
.   : milestone, 550,
iast_HARDCODED_SECRET_DISABLED (514.315 µs) : 491, 538
.   : milestone, 514,
iast_INACTIVE (463.58 µs) : 442, 485
.   : milestone, 464,
iast_TELEMETRY_OFF (497.64 µs) : 475, 521
.   : milestone, 498,
tracing (453.723 µs) : 432, 475
.   : milestone, 454,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 380.043 µs [359.784 µs, 400.302 µs] -
iast 515.799 µs [491.61 µs, 539.989 µs] 135.756 µs (35.7%)
iast_FULL 743.131 µs [720.401 µs, 765.861 µs] 363.088 µs (95.5%)
iast_GLOBAL 558.388 µs [534.035 µs, 582.74 µs] 178.345 µs (46.9%)
iast_HARDCODED_SECRET_DISABLED 514.36 µs [491.082 µs, 537.638 µs] 134.317 µs (35.3%)
iast_INACTIVE 462.576 µs [441.141 µs, 484.011 µs] 82.533 µs (21.7%)
iast_TELEMETRY_OFF 497.458 µs [474.259 µs, 520.657 µs] 117.415 µs (30.9%)
tracing 457.982 µs [436.873 µs, 479.092 µs] 77.939 µs (20.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 375.572 µs [355.38 µs, 395.765 µs] -
iast 508.399 µs [485.702 µs, 531.096 µs] 132.827 µs (35.4%)
iast_FULL 736.116 µs [714.159 µs, 758.073 µs] 360.544 µs (96.0%)
iast_GLOBAL 550.245 µs [528.534 µs, 571.955 µs] 174.672 µs (46.5%)
iast_HARDCODED_SECRET_DISABLED 514.315 µs [490.973 µs, 537.656 µs] 138.742 µs (36.9%)
iast_INACTIVE 463.58 µs [441.746 µs, 485.414 µs] 88.008 µs (23.4%)
iast_TELEMETRY_OFF 497.64 µs [474.681 µs, 520.598 µs] 122.068 µs (32.5%)
tracing 453.723 µs [432.044 µs, 475.402 µs] 78.151 µs (20.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737969316 1737978808
git_commit_sha 19da739 4812847
release_version 1.46.0-SNAPSHOT~19da739a7c 1.46.0-SNAPSHOT~4812847170
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1737980892 1737980892
ci_job_id 781276287 781276287
ci_pipeline_id 54044438 54044438
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-xugqgpm8-project-304-concurrent-0-ze4nt4kr 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-xugqgpm8-project-304-concurrent-0-ze4nt4kr 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~4812847170, baseline=1.46.0-SNAPSHOT~19da739a7c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1480
.   : milestone, 1469,
appsec (2.363 ms) : 2320, 2407
.   : milestone, 2363,
iast (2.103 ms) : 2049, 2158
.   : milestone, 2103,
iast_GLOBAL (2.145 ms) : 2090, 2200
.   : milestone, 2145,
profiling (1.956 ms) : 1913, 1999
.   : milestone, 1956,
tracing (1.938 ms) : 1897, 1980
.   : milestone, 1938,
section candidate
no_agent (1.466 ms) : 1455, 1478
.   : milestone, 1466,
appsec (2.367 ms) : 2323, 2410
.   : milestone, 2367,
iast (2.106 ms) : 2051, 2160
.   : milestone, 2106,
iast_GLOBAL (2.144 ms) : 2089, 2199
.   : milestone, 2144,
profiling (1.981 ms) : 1936, 2025
.   : milestone, 1981,
tracing (1.941 ms) : 1899, 1983
.   : milestone, 1941,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.48 ms] -
appsec 2.363 ms [2.32 ms, 2.407 ms] 894.468 µs (60.9%)
iast 2.103 ms [2.049 ms, 2.158 ms] 634.434 µs (43.2%)
iast_GLOBAL 2.145 ms [2.09 ms, 2.2 ms] 676.173 µs (46.0%)
profiling 1.956 ms [1.913 ms, 1.999 ms] 487.197 µs (33.2%)
tracing 1.938 ms [1.897 ms, 1.98 ms] 469.294 µs (31.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.466 ms [1.455 ms, 1.478 ms] -
appsec 2.367 ms [2.323 ms, 2.41 ms] 900.282 µs (61.4%)
iast 2.106 ms [2.051 ms, 2.16 ms] 639.596 µs (43.6%)
iast_GLOBAL 2.144 ms [2.089 ms, 2.199 ms] 677.51 µs (46.2%)
profiling 1.981 ms [1.936 ms, 2.025 ms] 514.435 µs (35.1%)
tracing 1.941 ms [1.899 ms, 1.983 ms] 474.96 µs (32.4%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~4812847170, baseline=1.46.0-SNAPSHOT~19da739a7c
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.459 s) : 15459000, 15459000
.   : milestone, 15459000,
appsec (15.233 s) : 15233000, 15233000
.   : milestone, 15233000,
iast (19.024 s) : 19024000, 19024000
.   : milestone, 19024000,
iast_GLOBAL (17.934 s) : 17934000, 17934000
.   : milestone, 17934000,
profiling (14.99 s) : 14990000, 14990000
.   : milestone, 14990000,
tracing (14.816 s) : 14816000, 14816000
.   : milestone, 14816000,
section candidate
no_agent (15.672 s) : 15672000, 15672000
.   : milestone, 15672000,
appsec (14.923 s) : 14923000, 14923000
.   : milestone, 14923000,
iast (18.689 s) : 18689000, 18689000
.   : milestone, 18689000,
iast_GLOBAL (17.817 s) : 17817000, 17817000
.   : milestone, 17817000,
profiling (14.991 s) : 14991000, 14991000
.   : milestone, 14991000,
tracing (14.729 s) : 14729000, 14729000
.   : milestone, 14729000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.459 s [15.459 s, 15.459 s] -
appsec 15.233 s [15.233 s, 15.233 s] -226.0 ms (-1.5%)
iast 19.024 s [19.024 s, 19.024 s] 3.565 s (23.1%)
iast_GLOBAL 17.934 s [17.934 s, 17.934 s] 2.475 s (16.0%)
profiling 14.99 s [14.99 s, 14.99 s] -469.0 ms (-3.0%)
tracing 14.816 s [14.816 s, 14.816 s] -643.0 ms (-4.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.672 s [15.672 s, 15.672 s] -
appsec 14.923 s [14.923 s, 14.923 s] -749.0 ms (-4.8%)
iast 18.689 s [18.689 s, 18.689 s] 3.017 s (19.3%)
iast_GLOBAL 17.817 s [17.817 s, 17.817 s] 2.145 s (13.7%)
profiling 14.991 s [14.991 s, 14.991 s] -681.0 ms (-4.3%)
tracing 14.729 s [14.729 s, 14.729 s] -943.0 ms (-6.0%)

@smola smola force-pushed the smola/gradle-java11 branch from 702851a to d1abf3b Compare September 23, 2024 11:18
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 1, 2025

This pull request has been marked as stale because it has not had activity over the past quarter. It will be closed in 7 days if no further activity occurs. Feel free to reopen the PR if you are still working on it.

@github-actions github-actions bot added the tag: stale Stale pull requests label Jan 1, 2025
@smola smola force-pushed the smola/gradle-java11 branch 2 times, most recently from 0e79c09 to 0a43968 Compare January 15, 2025 15:19
@pr-commenter
Copy link

pr-commenter bot commented Jan 15, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1737357243 1737357619
end_time 2025-01-20T07:15:19 2025-01-20T07:21:36
git_branch master smola/gradle-java11
git_commit_sha 0b239ae 352736e
start_time 2025-01-20T07:14:04 2025-01-20T07:20:20
See matching parameters
Baseline Candidate
ci_job_id 772223798 772223798
ci_pipeline_id 53445931 53445931
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1737222884 1737222884

Summary

Found 0 performance improvements and 6 performance regressions! Performance is the same for 3 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:basic worse
[+3.576µs; +11.815µs] or [+1.324%; +4.374%]		 same same unstable
[-23.734µs; +39.109µs] or [-4.673%; +7.700%]		 unstable
[-239.733op/s; +111.528op/s] or [-9.350%; +4.350%]
scenario:loop worse
[+442.270µs; +522.596µs] or [+4.356%; +5.147%]		 worse
[+449.109µs; +550.782µs] or [+4.353%; +5.338%]		 worse
[+410.327µs; +544.125µs] or [+3.935%; +5.218%]		 worse
[+195.678µs; +703.388µs] or [+1.812%; +6.512%]		 worse
[-5.491op/s; -3.248op/s] or [-5.738%; -3.394%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-26.397µs; +21.584µs] or [-9.744%; +7.968%]		 unstable
[-39.342µs; +33.189µs] or [-12.663%; +10.682%]		 unstable
[-52.143µs; +42.935µs] or [-16.038%; +13.206%]		 unstable
[-94.545µs; +97.240µs] or [-16.186%; +16.647%]		 same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (310.695 µs) : 270, 352
.   : milestone, 311,
basic (301.977 µs) : 293, 310
.   : milestone, 302,
loop (10.317 ms) : 10257, 10378
.   : milestone, 10317,
section candidate
noprobe (307.618 µs) : 281, 334
.   : milestone, 308,
basic (309.887 µs) : 301, 319
.   : milestone, 310,
loop (10.817 ms) : 10788, 10847
.   : milestone, 10817,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 310.695 µs [269.778 µs, 351.611 µs]
basic 301.977 µs [293.457 µs, 310.498 µs]
loop 10.317 ms [10.257 ms, 10.378 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 307.618 µs [280.843 µs, 334.393 µs]
basic 309.887 µs [301.101 µs, 318.674 µs]
loop 10.817 ms [10.788 ms, 10.847 ms]

@smola smola force-pushed the smola/gradle-java11 branch 3 times, most recently from 22e3221 to 352736e Compare January 20, 2025 06:59
@pr-commenter
Copy link

pr-commenter bot commented Jan 20, 2025 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737969316 1737978808
git_commit_sha 19da739 4812847
See matching parameters
Baseline Candidate
ci_job_date 1737979916 1737979916
ci_job_id 781276289 781276289
ci_pipeline_id 54044438 54044438
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce unsure
[+1238.049op/s; +4895.775op/s] or [+0.917%; +3.628%]

@pr-commenter
Copy link

pr-commenter bot commented Jan 20, 2025 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/gradle-java11
git_commit_date 1737969316 1737978808
git_commit_sha 19da739 4812847
See matching parameters
Baseline Candidate
ci_job_date 1737979957 1737979957
ci_job_id 781276290 781276290
ci_pipeline_id 54044438 54044438
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@smola smola removed the tag: stale Stale pull requests label Jan 20, 2025
@smola smola force-pushed the smola/gradle-java11 branch 2 times, most recently from 920f3ba to 8573d24 Compare January 24, 2025 08:33
@smola smola mentioned this pull request Jan 24, 2025
Merged
@smola smola force-pushed the smola/gradle-java11 branch from 8573d24 to 4812847 Compare January 27, 2025 11:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@smola