Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use iterators to avoid calculating all properties of UnionOrIntersectionType #53346

Draft
wants to merge 23 commits into
base: main
Choose a base branch
from
Draft

Use iterators to avoid calculating all properties of UnionOrIntersectionType #53346

wants to merge 23 commits into from

Conversation

jakebailey
Copy link
Member

For #51188

Profiling gave me this big chunk:

image

getPropertiesOfUnionOrIntersectionType is the biggest chunk. What I noticed was that getReducedType didn't actually need all of the properties; all it wants to know is whether or not some property satisfies isNeverReducedProperty.

By fiddling with the code a bit, we can convert getPropertiesOfUnionOrIntersectionType into an iterator, allowing callers to stop calculating properties early if that benefits them. And, it turns out that most callers of getPropertiesOfUnionOrIntersectionType (all but one!) indeed don't actually care about every property, only whichever one matches some predicate.

After, the same code looks like:

image

hyperfine, for this test case:

Benchmark 1: main
  Time (mean ± σ):     19.403 s ±  0.300 s    [User: 16.662 s, System: 2.735 s]
  Range (min … max):   18.933 s … 19.988 s    20 runs
 
Benchmark 2: new
  Time (mean ± σ):     13.450 s ±  0.074 s    [User: 11.974 s, System: 1.470 s]
  Range (min … max):   13.341 s … 13.612 s    20 runs
 
Summary
  'new' ran
    1.44 ± 0.02 times faster than 'main'

aka 44% faster aka 1.44x faster aka 31% less time.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Mar 19, 2023
@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 19, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the perf test suite on this PR at c53294b. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - main..53346
Metric main 53346 Delta Best Worst p-value
Angular - node (v18.10.0, x64)
Memory used 361,557k (± 0.01%) 361,616k (± 0.01%) +59k (+ 0.02%) 361,563k 361,643k p=0.016 n=6
Parse Time 3.37s (± 0.35%) 3.39s (± 0.48%) +0.02s (+ 0.64%) 3.37s 3.41s p=0.042 n=6
Bind Time 1.12s (± 0.49%) 1.12s (± 1.08%) ~ 1.11s 1.14s p=0.855 n=6
Check Time 8.70s (± 0.37%) 8.73s (± 0.60%) ~ 8.66s 8.78s p=0.375 n=6
Emit Time 7.42s (± 0.93%) 7.45s (± 0.84%) ~ 7.39s 7.56s p=0.230 n=6
Total Time 20.60s (± 0.47%) 20.68s (± 0.48%) ~ 20.56s 20.82s p=0.149 n=6
Compiler-Unions - node (v18.10.0, x64)
Memory used 192,647k (± 1.50%) 185,819k (± 1.56%) 🟩-6,829k (- 3.54%) 183,879k 189,673k p=0.005 n=6
Parse Time 1.49s (± 0.89%) 1.50s (± 1.24%) ~ 1.48s 1.53s p=0.931 n=6
Bind Time 0.77s (± 0.53%) 0.77s (± 0.53%) ~ 0.76s 0.77s p=0.218 n=6
Check Time 9.36s (± 0.79%) 9.34s (± 0.80%) ~ 9.21s 9.42s p=0.873 n=6
Emit Time 2.72s (± 0.79%) 2.74s (± 0.68%) ~ 2.73s 2.78s p=0.052 n=6
Total Time 14.34s (± 0.55%) 14.36s (± 0.63%) ~ 14.21s 14.44s p=0.689 n=6
Monaco - node (v18.10.0, x64)
Memory used 346,121k (± 0.01%) 346,141k (± 0.01%) ~ 346,108k 346,176k p=0.378 n=6
Parse Time 2.58s (± 1.33%) 2.58s (± 1.12%) ~ 2.54s 2.63s p=1.000 n=6
Bind Time 1.00s (± 0.98%) 1.01s (± 0.63%) +0.01s (+ 1.17%) 1.00s 1.02s p=0.031 n=6
Check Time 6.97s (± 0.44%) 7.00s (± 0.31%) ~ 6.97s 7.02s p=0.088 n=6
Emit Time 4.24s (± 0.38%) 4.26s (± 0.94%) ~ 4.20s 4.30s p=0.332 n=6
Total Time 14.79s (± 0.32%) 14.84s (± 0.42%) ~ 14.78s 14.94s p=0.090 n=6
TFS - node (v18.10.0, x64)
Memory used 300,454k (± 0.01%) 300,440k (± 0.01%) ~ 300,407k 300,483k p=0.423 n=6
Parse Time 2.06s (± 0.95%) 2.06s (± 0.94%) ~ 2.03s 2.08s p=1.000 n=6
Bind Time 1.13s (± 0.46%) 1.14s (± 0.96%) ~ 1.12s 1.15s p=0.149 n=6
Check Time 6.47s (± 0.29%) 6.50s (± 0.25%) +0.03s (+ 0.46%) 6.48s 6.52s p=0.023 n=6
Emit Time 3.86s (± 0.63%) 3.85s (± 0.97%) ~ 3.81s 3.91s p=0.808 n=6
Total Time 13.52s (± 0.36%) 13.55s (± 0.18%) ~ 13.52s 13.58s p=0.259 n=6
material-ui - node (v18.10.0, x64)
Memory used 478,171k (± 0.01%) 478,428k (± 0.01%) +258k (+ 0.05%) 478,394k 478,491k p=0.005 n=6
Parse Time 2.98s (± 3.24%) 3.03s (± 2.39%) ~ 2.94s 3.09s p=0.332 n=6
Bind Time 1.00s (± 7.39%) 0.95s (± 8.14%) ~ 0.89s 1.07s p=0.253 n=6
Check Time 17.05s (± 0.70%) 17.14s (± 0.48%) ~ 17.07s 17.28s p=0.173 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.03s (± 0.53%) 21.13s (± 0.44%) ~ 21.04s 21.29s p=0.172 n=6
xstate - node (v18.10.0, x64)
Memory used 553,195k (± 0.01%) 553,218k (± 0.02%) ~ 553,073k 553,335k p=0.810 n=6
Parse Time 3.75s (± 0.34%) 3.76s (± 0.62%) ~ 3.73s 3.80s p=0.278 n=6
Bind Time 1.69s (± 0.71%) 1.69s (± 0.53%) ~ 1.68s 1.70s p=0.676 n=6
Check Time 2.78s (± 0.85%) 2.88s (± 1.31%) +0.10s (+ 3.72%) 2.84s 2.93s p=0.005 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 4.99%) ~ 0.08s 0.09s p=0.405 n=6
Total Time 8.31s (± 0.35%) 8.42s (± 0.42%) +0.11s (+ 1.28%) 8.38s 8.47s p=0.005 n=6
Angular - node (v16.17.1, x64)
Memory used 360,896k (± 0.01%) 360,965k (± 0.00%) +70k (+ 0.02%) 360,940k 360,988k p=0.005 n=6
Parse Time 3.52s (± 0.69%) 3.52s (± 0.64%) ~ 3.50s 3.55s p=1.000 n=6
Bind Time 1.18s (± 0.69%) 1.19s (± 0.68%) ~ 1.18s 1.20s p=0.077 n=6
Check Time 9.46s (± 0.52%) 9.49s (± 0.32%) ~ 9.44s 9.53s p=0.141 n=6
Emit Time 7.93s (± 0.68%) 7.88s (± 0.44%) ~ 7.84s 7.93s p=0.172 n=6
Total Time 22.10s (± 0.38%) 22.10s (± 0.24%) ~ 22.04s 22.18s p=0.810 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 192,531k (± 0.04%) 186,690k (± 0.90%) 🟩-5,842k (- 3.03%) 185,564k 188,994k p=0.005 n=6
Parse Time 1.57s (± 1.64%) 1.58s (± 1.48%) ~ 1.54s 1.61s p=0.618 n=6
Bind Time 0.82s (± 0.77%) 0.83s (± 0.66%) ~ 0.82s 0.83s p=0.201 n=6
Check Time 10.08s (± 0.62%) 10.04s (± 0.66%) ~ 9.96s 10.12s p=0.199 n=6
Emit Time 2.99s (± 0.63%) 2.99s (± 0.35%) ~ 2.98s 3.01s p=0.744 n=6
Total Time 15.47s (± 0.30%) 15.44s (± 0.58%) ~ 15.33s 15.54s p=0.687 n=6
Monaco - node (v16.17.1, x64)
Memory used 345,434k (± 0.01%) 345,448k (± 0.00%) ~ 345,431k 345,461k p=0.128 n=6
Parse Time 2.72s (± 0.36%) 2.73s (± 0.50%) ~ 2.71s 2.75s p=0.138 n=6
Bind Time 1.09s (± 0.50%) 1.09s (± 0.00%) ~ 1.09s 1.09s p=0.071 n=6
Check Time 7.66s (± 0.32%) 7.71s (± 0.59%) +0.05s (+ 0.67%) 7.68s 7.80s p=0.024 n=6
Emit Time 4.42s (± 0.57%) 4.46s (± 1.15%) ~ 4.38s 4.53s p=0.171 n=6
Total Time 15.89s (± 0.19%) 16.00s (± 0.51%) +0.11s (+ 0.67%) 15.93s 16.12s p=0.006 n=6
TFS - node (v16.17.1, x64)
Memory used 299,792k (± 0.01%) 299,792k (± 0.01%) ~ 299,751k 299,824k p=0.936 n=6
Parse Time 2.17s (± 0.48%) 2.18s (± 0.84%) ~ 2.16s 2.20s p=0.249 n=6
Bind Time 1.24s (± 0.88%) 1.24s (± 0.88%) ~ 1.22s 1.25s p=1.000 n=6
Check Time 7.17s (± 0.57%) 7.16s (± 0.63%) ~ 7.11s 7.22s p=0.748 n=6
Emit Time 4.31s (± 0.70%) 4.35s (± 0.70%) +0.04s (+ 1.04%) 4.31s 4.39s p=0.035 n=6
Total Time 14.88s (± 0.33%) 14.93s (± 0.42%) ~ 14.86s 15.03s p=0.148 n=6
material-ui - node (v16.17.1, x64)
Memory used 477,460k (± 0.02%) 477,732k (± 0.01%) +271k (+ 0.06%) 477,648k 477,769k p=0.005 n=6
Parse Time 3.21s (± 0.36%) 3.21s (± 0.91%) ~ 3.17s 3.24s p=0.513 n=6
Bind Time 0.95s (± 0.54%) 0.95s (± 0.57%) ~ 0.95s 0.96s p=0.640 n=6
Check Time 17.96s (± 0.49%) 18.18s (± 0.65%) +0.23s (+ 1.26%) 18.08s 18.41s p=0.005 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.12s (± 0.45%) 22.35s (± 0.56%) +0.23s (+ 1.03%) 22.24s 22.59s p=0.008 n=6
xstate - node (v16.17.1, x64)
Memory used 550,802k (± 0.02%) 550,846k (± 0.02%) ~ 550,728k 550,927k p=0.298 n=6
Parse Time 3.94s (± 0.35%) 3.96s (± 0.68%) ~ 3.93s 4.01s p=0.088 n=6
Bind Time 1.80s (± 0.30%) 1.79s (± 0.50%) ~ 1.78s 1.80s p=0.341 n=6
Check Time 3.03s (± 0.39%) 3.10s (± 0.58%) +0.07s (+ 2.25%) 3.08s 3.13s p=0.005 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 8.86s (± 0.27%) 8.94s (± 0.39%) +0.08s (+ 0.96%) 8.90s 9.00s p=0.006 n=6
Angular - node (v14.15.1, x64)
Memory used 354,718k (± 0.01%) 354,742k (± 0.01%) ~ 354,697k 354,773k p=0.230 n=6
Parse Time 3.58s (± 0.55%) 3.60s (± 0.71%) ~ 3.56s 3.63s p=0.169 n=6
Bind Time 1.23s (± 0.45%) 1.23s (± 0.66%) ~ 1.22s 1.24s p=0.088 n=6
Check Time 9.70s (± 0.46%) 9.74s (± 0.36%) ~ 9.70s 9.80s p=0.171 n=6
Emit Time 8.34s (± 0.90%) 8.37s (± 0.94%) ~ 8.24s 8.48s p=0.689 n=6
Total Time 22.83s (± 0.49%) 22.93s (± 0.32%) ~ 22.84s 23.05s p=0.172 n=6
Compiler-Unions - node (v14.15.1, x64)
Memory used 187,762k (± 0.01%) 180,820k (± 0.02%) 🟩-6,943k (- 3.70%) 180,781k 180,864k p=0.005 n=6
Parse Time 1.60s (± 0.73%) 1.60s (± 0.51%) ~ 1.59s 1.61s p=0.738 n=6
Bind Time 0.84s (± 0.65%) 0.85s (± 0.48%) ~ 0.84s 0.85s p=0.282 n=6
Check Time 10.16s (± 0.53%) 10.09s (± 0.49%) ~ 10.02s 10.16s p=0.054 n=6
Emit Time 3.13s (± 1.03%) 3.12s (± 1.59%) ~ 3.03s 3.16s p=0.810 n=6
Total Time 15.73s (± 0.41%) 15.66s (± 0.50%) ~ 15.57s 15.76s p=0.126 n=6
Monaco - node (v14.15.1, x64)
Memory used 340,407k (± 0.00%) 340,410k (± 0.01%) ~ 340,355k 340,449k p=0.748 n=6
Parse Time 2.84s (± 0.75%) 2.84s (± 0.52%) ~ 2.82s 2.86s p=0.869 n=6
Bind Time 1.10s (± 1.06%) 1.11s (± 0.47%) ~ 1.10s 1.11s p=0.191 n=6
Check Time 8.04s (± 0.30%) 8.04s (± 0.33%) ~ 7.99s 8.07s p=0.871 n=6
Emit Time 4.68s (± 0.63%) 4.70s (± 0.63%) ~ 4.67s 4.74s p=0.294 n=6
Total Time 16.65s (± 0.21%) 16.68s (± 0.23%) ~ 16.64s 16.73s p=0.169 n=6
TFS - node (v14.15.1, x64)
Memory used 294,862k (± 0.00%) 294,871k (± 0.00%) +9k (+ 0.00%) 294,856k 294,877k p=0.045 n=6
Parse Time 2.40s (± 0.34%) 2.40s (± 0.94%) ~ 2.37s 2.42s p=0.415 n=6
Bind Time 1.07s (± 0.48%) 1.07s (± 0.51%) ~ 1.06s 1.07s p=0.640 n=6
Check Time 7.46s (± 0.60%) 7.47s (± 0.65%) ~ 7.41s 7.54s p=0.747 n=6
Emit Time 4.29s (± 1.08%) 4.29s (± 0.81%) ~ 4.24s 4.34s p=0.569 n=6
Total Time 15.21s (± 0.41%) 15.23s (± 0.52%) ~ 15.13s 15.34s p=0.468 n=6
material-ui - node (v14.15.1, x64)
Memory used 472,968k (± 0.01%) 473,254k (± 0.00%) +286k (+ 0.06%) 473,230k 473,281k p=0.005 n=6
Parse Time 3.33s (± 0.29%) 3.35s (± 0.41%) ~ 3.33s 3.37s p=0.054 n=6
Bind Time 1.00s (± 0.41%) 1.01s (± 0.51%) ~ 1.00s 1.01s p=0.112 n=6
Check Time 18.97s (± 0.46%) 19.17s (± 0.66%) +0.20s (+ 1.06%) 19.03s 19.36s p=0.020 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 23.30s (± 0.41%) 23.52s (± 0.60%) +0.22s (+ 0.93%) 23.36s 23.73s p=0.020 n=6
xstate - node (v14.15.1, x64)
Memory used 539,282k (± 0.01%) 539,319k (± 0.00%) ~ 539,297k 539,358k p=0.054 n=6
Parse Time 4.22s (± 0.36%) 4.26s (± 0.42%) +0.04s (+ 0.87%) 4.23s 4.28s p=0.014 n=6
Bind Time 1.68s (± 0.49%) 1.68s (± 0.73%) ~ 1.66s 1.69s p=1.000 n=6
Check Time 3.19s (± 0.68%) 3.25s (± 0.83%) +0.06s (+ 1.72%) 3.21s 3.29s p=0.010 n=6
Emit Time 0.09s (± 0.00%) 0.10s (± 4.15%) +0.01s (+ 9.26%) 0.09s 0.10s p=0.007 n=6
Total Time 9.18s (± 0.23%) 9.28s (± 0.57%) +0.10s (+ 1.03%) 9.23s 9.37s p=0.005 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v18.10.0, x64)
  • Angular - node (v16.17.1, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v18.10.0, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v18.10.0, x64)
  • Monaco - node (v16.17.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v18.10.0, x64)
  • TFS - node (v16.17.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v18.10.0, x64)
  • material-ui - node (v16.17.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v18.10.0, x64)
  • xstate - node (v16.17.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 53346 6
Baseline main 6

TSServer

Comparison Report - main..53346
Metric main 53346 Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,381ms (± 0.52%) 2,392ms (± 0.71%) ~ 2,373ms 2,411ms p=0.230 n=6
Req 2 - geterr 5,340ms (± 0.54%) 5,382ms (± 0.63%) ~ 5,334ms 5,426ms p=0.066 n=6
Req 3 - references 338ms (± 1.43%) 338ms (± 0.83%) ~ 332ms 340ms p=1.000 n=6
Req 4 - navto 279ms (± 0.27%) 280ms (± 0.58%) ~ 278ms 282ms p=0.117 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 80ms (± 6.30%) 83ms (± 0.76%) ~ 82ms 84ms p=0.157 n=6
CompilerTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 2,510ms (± 1.07%) 2,517ms (± 0.89%) ~ 2,492ms 2,544ms p=0.575 n=6
Req 2 - geterr 4,012ms (± 0.58%) 4,012ms (± 0.68%) ~ 3,979ms 4,045ms p=0.936 n=6
Req 3 - references 349ms (± 1.03%) 349ms (± 0.97%) ~ 343ms 353ms p=0.936 n=6
Req 4 - navto 290ms (± 1.01%) 294ms (± 1.03%) ~ 289ms 298ms p=0.102 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 79ms (± 6.67%) 83ms (± 3.96%) ~ 79ms 87ms p=0.225 n=6
xstateTSServer - node (v18.10.0, x64)
Req 1 - updateOpen 3,025ms (± 0.89%) 3,021ms (± 0.43%) ~ 3,009ms 3,038ms p=1.000 n=6
Req 2 - geterr 1,624ms (± 1.04%) 1,645ms (± 0.87%) ~ 1,628ms 1,662ms p=0.128 n=6
Req 3 - references 108ms (± 1.98%) 108ms (± 1.65%) ~ 105ms 110ms p=1.000 n=6
Req 4 - navto 359ms (± 1.22%) 359ms (± 2.26%) ~ 354ms 375ms p=0.369 n=6
Req 5 - completionInfo count 2,861 (± 0.00%) 2,861 (± 0.00%) ~ 2,861 2,861 p=1.000 n=6
Req 5 - completionInfo 382ms (± 1.51%) 374ms (± 1.68%) -8ms (- 2.01%) 362ms 379ms p=0.036 n=6
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,507ms (± 0.63%) 2,508ms (± 0.91%) ~ 2,474ms 2,542ms p=1.000 n=6
Req 2 - geterr 5,749ms (± 0.70%) 5,791ms (± 0.76%) ~ 5,726ms 5,836ms p=0.149 n=6
Req 3 - references 348ms (± 0.85%) 345ms (± 0.41%) ~ 343ms 347ms p=0.142 n=6
Req 4 - navto 279ms (± 1.05%) 278ms (± 0.53%) ~ 277ms 281ms p=0.453 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 74ms (± 1.11%) 73ms (± 1.11%) ~ 72ms 74ms p=0.389 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,702ms (± 1.36%) 2,703ms (± 1.45%) ~ 2,669ms 2,766ms p=1.000 n=6
Req 2 - geterr 4,376ms (± 0.25%) 4,374ms (± 0.26%) ~ 4,359ms 4,390ms p=0.936 n=6
Req 3 - references 359ms (± 0.43%) 360ms (± 1.43%) ~ 352ms 368ms p=0.622 n=6
Req 4 - navto 289ms (± 0.51%) 291ms (± 0.73%) ~ 289ms 294ms p=0.099 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 73ms (± 6.49%) 73ms (± 6.91%) ~ 67ms 80ms p=1.000 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 3,160ms (± 0.30%) 3,167ms (± 0.25%) ~ 3,161ms 3,182ms p=0.336 n=6
Req 2 - geterr 1,763ms (± 0.77%) 1,796ms (± 0.47%) +33ms (+ 1.88%) 1,785ms 1,806ms p=0.005 n=6
Req 3 - references 117ms (± 1.08%) 117ms (± 1.37%) ~ 114ms 118ms p=1.000 n=6
Req 4 - navto 342ms (± 0.40%) 341ms (± 0.44%) ~ 339ms 343ms p=0.190 n=6
Req 5 - completionInfo count 2,861 (± 0.00%) 2,861 (± 0.00%) ~ 2,861 2,861 p=1.000 n=6
Req 5 - completionInfo 404ms (± 5.37%) 391ms (± 1.67%) ~ 380ms 397ms p=0.229 n=6
Compiler-UnionsTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,584ms (± 0.58%) 2,589ms (± 0.73%) ~ 2,564ms 2,615ms p=0.936 n=6
Req 2 - geterr 6,127ms (± 0.33%) 6,161ms (± 0.91%) ~ 6,099ms 6,238ms p=0.471 n=6
Req 3 - references 366ms (± 1.45%) 365ms (± 0.66%) ~ 362ms 369ms p=0.872 n=6
Req 4 - navto 276ms (± 1.01%) 277ms (± 1.13%) ~ 275ms 283ms p=0.652 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 102ms (± 0.96%) 101ms (± 2.75%) ~ 96ms 104ms p=0.868 n=6
CompilerTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 2,808ms (± 0.34%) 2,810ms (± 0.51%) ~ 2,786ms 2,829ms p=0.689 n=6
Req 2 - geterr 4,460ms (± 0.39%) 4,542ms (± 2.74%) ~ 4,430ms 4,707ms p=0.298 n=6
Req 3 - references 421ms (± 4.79%) 410ms (± 7.39%) ~ 369ms 433ms p=0.374 n=6
Req 4 - navto 289ms (± 0.99%) 290ms (± 0.65%) ~ 287ms 292ms p=0.624 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 85ms (± 5.27%) 84ms (± 9.10%) ~ 74ms 92ms p=0.849 n=6
xstateTSServer - node (v14.15.1, x64)
Req 1 - updateOpen 3,443ms (± 0.35%) 3,456ms (± 0.53%) ~ 3,428ms 3,477ms p=0.229 n=6
Req 2 - geterr 1,857ms (± 0.61%) 1,871ms (± 0.85%) +14ms (+ 0.73%) 1,840ms 1,884ms p=0.045 n=6
Req 3 - references 126ms (± 2.00%) 125ms (± 0.82%) ~ 124ms 127ms p=0.503 n=6
Req 4 - navto 371ms (± 0.36%) 370ms (± 0.28%) ~ 368ms 371ms p=0.098 n=6
Req 5 - completionInfo count 2,861 (± 0.00%) 2,861 (± 0.00%) ~ 2,861 2,861 p=1.000 n=6
Req 5 - completionInfo 412ms (± 1.42%) 410ms (± 0.98%) ~ 407ms 416ms p=0.375 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v18.10.0, x64)
  • node (v16.17.1, x64)
  • node (v14.15.1, x64)
Scenarios
  • Compiler-UnionsTSServer - node (v18.10.0, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v14.15.1, x64)
  • CompilerTSServer - node (v18.10.0, x64)
  • CompilerTSServer - node (v16.17.1, x64)
  • CompilerTSServer - node (v14.15.1, x64)
  • xstateTSServer - node (v18.10.0, x64)
  • xstateTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v14.15.1, x64)
Benchmark Name Iterations
Current 53346 6
Baseline main 6

Startup

Comparison Report - main..53346
Metric main 53346 Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 141.19ms (± 0.18%) 141.82ms (± 0.21%) +0.62ms (+ 0.44%) 141.06ms 145.67ms p=0.000 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 226.11ms (± 0.18%) 227.22ms (± 0.19%) +1.11ms (+ 0.49%) 226.20ms 234.19ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 227.89ms (± 0.15%) 229.05ms (± 0.17%) +1.17ms (+ 0.51%) 227.97ms 234.32ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 208.82ms (± 0.15%) 209.64ms (± 0.18%) +0.82ms (+ 0.39%) 208.71ms 216.39ms p=0.000 n=600
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53346 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 19, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at 8a6ba6d. You can monitor the build here.

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 19, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at b8d3664. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..53346

Metric main 53346 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 361,130k (± 0.01%) 361,113k (± 0.01%) ~ 361,069k 361,155k p=0.336 n=6
Parse Time 3.54s (± 0.64%) 3.56s (± 0.41%) ~ 3.54s 3.58s p=0.120 n=6
Bind Time 1.18s (± 0.46%) 1.19s (± 0.98%) ~ 1.18s 1.21s p=0.342 n=6
Check Time 9.50s (± 0.57%) 9.56s (± 0.57%) ~ 9.49s 9.62s p=0.091 n=6
Emit Time 7.94s (± 0.74%) 8.00s (± 0.63%) ~ 7.92s 8.05s p=0.170 n=6
Total Time 22.16s (± 0.40%) 22.31s (± 0.45%) +0.15s (+ 0.66%) 22.16s 22.41s p=0.045 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 192,596k (± 0.02%) 186,371k (± 0.74%) 🟩-6,225k (- 3.23%) 185,763k 189,172k p=0.005 n=6
Parse Time 1.56s (± 1.62%) 1.58s (± 1.13%) ~ 1.55s 1.60s p=0.191 n=6
Bind Time 0.82s (± 0.92%) 0.83s (± 0.49%) +0.01s (+ 1.22%) 0.82s 0.83s p=0.029 n=6
Check Time 10.08s (± 0.63%) 10.06s (± 0.40%) ~ 9.99s 10.10s p=0.572 n=6
Emit Time 2.97s (± 0.72%) 2.99s (± 1.46%) ~ 2.96s 3.07s p=0.570 n=6
Total Time 15.43s (± 0.56%) 15.45s (± 0.13%) ~ 15.42s 15.47s p=0.518 n=6
Monaco - node (v16.17.1, x64)
Memory used 345,520k (± 0.00%) 345,578k (± 0.01%) +58k (+ 0.02%) 345,546k 345,605k p=0.005 n=6
Parse Time 2.73s (± 0.36%) 2.75s (± 0.30%) +0.02s (+ 0.67%) 2.74s 2.76s p=0.008 n=6
Bind Time 1.09s (± 0.96%) 1.09s (± 0.58%) ~ 1.08s 1.10s p=0.388 n=6
Check Time 7.67s (± 0.59%) 7.75s (± 0.48%) +0.08s (+ 1.00%) 7.69s 7.79s p=0.020 n=6
Emit Time 4.44s (± 0.48%) 4.47s (± 0.59%) ~ 4.44s 4.50s p=0.062 n=6
Total Time 15.93s (± 0.37%) 16.06s (± 0.30%) +0.13s (+ 0.82%) 16.02s 16.14s p=0.005 n=6
TFS - node (v16.17.1, x64)
Memory used 299,823k (± 0.01%) 299,815k (± 0.01%) ~ 299,777k 299,853k p=0.688 n=6
Parse Time 2.17s (± 0.45%) 2.19s (± 0.48%) +0.02s (+ 0.77%) 2.17s 2.20s p=0.024 n=6
Bind Time 1.24s (± 0.79%) 1.25s (± 0.94%) ~ 1.23s 1.26s p=0.157 n=6
Check Time 7.17s (± 0.43%) 7.19s (± 0.39%) ~ 7.15s 7.22s p=0.331 n=6
Emit Time 4.36s (± 0.81%) 4.36s (± 1.22%) ~ 4.30s 4.45s p=0.809 n=6
Total Time 14.93s (± 0.30%) 14.98s (± 0.57%) ~ 14.89s 15.11s p=0.376 n=6
material-ui - node (v16.17.1, x64)
Memory used 477,491k (± 0.01%) 0k ~ 0k 0k p=1.000 n=6+0
Parse Time 3.22s (± 0.80%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Bind Time 0.96s (± 0.57%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Check Time 18.06s (± 0.47%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Emit Time 0.00s (± 0.00%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Total Time 22.23s (± 0.32%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
xstate - node (v16.17.1, x64)
Memory used 550,687k (± 0.01%) 550,781k (± 0.02%) ~ 550,625k 550,906k p=0.173 n=6
Parse Time 3.94s (± 0.26%) 3.96s (± 0.44%) ~ 3.95s 3.99s p=0.064 n=6
Bind Time 1.79s (± 0.65%) 1.80s (± 0.76%) ~ 1.78s 1.82s p=0.117 n=6
Check Time 3.03s (± 0.94%) 3.13s (± 1.23%) +0.10s (+ 3.30%) 3.08s 3.19s p=0.005 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 8.86s (± 0.29%) 9.00s (± 0.59%) +0.14s (+ 1.64%) 8.92s 9.07s p=0.005 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53346 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member Author

My latest version, while clearer, turns out to be worse than the previous iteration, so I guess I'll have to revert it. Really weird.

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 19, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at 21ccbb9. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..53346

Metric main 53346 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 361,043k (± 0.01%) 361,100k (± 0.01%) +57k (+ 0.02%) 361,076k 361,139k p=0.005 n=6
Parse Time 3.53s (± 0.87%) 3.50s (± 0.55%) ~ 3.47s 3.53s p=0.071 n=6
Bind Time 1.18s (± 0.46%) 1.18s (± 0.46%) ~ 1.18s 1.19s p=1.000 n=6
Check Time 9.48s (± 0.49%) 9.51s (± 0.60%) ~ 9.47s 9.62s p=0.332 n=6
Emit Time 7.93s (± 0.82%) 7.96s (± 1.30%) ~ 7.90s 8.17s p=0.688 n=6
Total Time 22.13s (± 0.53%) 22.16s (± 0.76%) ~ 22.06s 22.50s p=0.687 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 192,558k (± 0.02%) 185,691k (± 0.02%) 🟩-6,867k (- 3.57%) 185,644k 185,767k p=0.005 n=6
Parse Time 1.59s (± 0.86%) 1.56s (± 1.22%) -0.03s (- 2.09%) 1.53s 1.58s p=0.012 n=6
Bind Time 0.82s (± 0.63%) 0.82s (± 1.00%) ~ 0.81s 0.83s p=0.140 n=6
Check Time 10.15s (± 0.87%) 9.97s (± 0.37%) -0.18s (- 1.76%) 9.92s 10.02s p=0.005 n=6
Emit Time 3.00s (± 0.25%) 3.00s (± 1.14%) ~ 2.97s 3.05s p=0.418 n=6
Total Time 15.56s (± 0.51%) 15.35s (± 0.43%) -0.22s (- 1.39%) 15.26s 15.44s p=0.005 n=6
Monaco - node (v16.17.1, x64)
Memory used 345,570k (± 0.01%) 345,577k (± 0.00%) ~ 345,560k 345,591k p=0.471 n=6
Parse Time 2.73s (± 0.60%) 2.72s (± 0.28%) ~ 2.71s 2.73s p=0.216 n=6
Bind Time 1.09s (± 0.47%) 1.09s (± 1.71%) ~ 1.08s 1.13s p=0.666 n=6
Check Time 7.70s (± 0.41%) 7.71s (± 0.46%) ~ 7.68s 7.77s p=0.872 n=6
Emit Time 4.46s (± 0.49%) 4.44s (± 0.69%) ~ 4.41s 4.49s p=0.226 n=6
Total Time 15.98s (± 0.35%) 15.96s (± 0.33%) ~ 15.91s 16.04s p=0.630 n=6
TFS - node (v16.17.1, x64)
Memory used 299,817k (± 0.01%) 299,821k (± 0.01%) ~ 299,771k 299,860k p=0.810 n=6
Parse Time 2.17s (± 0.54%) 2.17s (± 0.48%) ~ 2.15s 2.18s p=0.406 n=6
Bind Time 1.24s (± 0.97%) 1.25s (± 0.65%) ~ 1.24s 1.26s p=0.677 n=6
Check Time 7.17s (± 0.54%) 7.16s (± 0.65%) ~ 7.09s 7.21s p=0.684 n=6
Emit Time 4.34s (± 0.63%) 4.33s (± 0.34%) ~ 4.31s 4.35s p=0.250 n=6
Total Time 14.93s (± 0.24%) 14.90s (± 0.40%) ~ 14.82s 14.97s p=0.421 n=6
material-ui - node (v16.17.1, x64)
Memory used 476,408k (± 0.01%) 0k ~ 0k 0k p=1.000 n=6+0
Parse Time 3.21s (± 0.61%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Bind Time 0.95s (± 0.43%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Check Time 17.91s (± 0.86%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Emit Time 0.00s (± 0.00%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
Total Time 22.08s (± 0.62%) 0.00s ~ 0.00s 0.00s p=1.000 n=6+0
xstate - node (v16.17.1, x64)
Memory used 550,624k (± 0.02%) 550,798k (± 0.03%) +175k (+ 0.03%) 550,613k 551,006k p=0.031 n=6
Parse Time 3.95s (± 0.16%) 3.94s (± 0.25%) ~ 3.93s 3.95s p=0.103 n=6
Bind Time 1.80s (± 0.29%) 1.79s (± 0.93%) ~ 1.76s 1.81s p=0.437 n=6
Check Time 3.04s (± 0.34%) 3.12s (± 0.59%) +0.08s (+ 2.80%) 3.10s 3.14s p=0.005 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 8.88s (± 0.17%) 8.95s (± 0.13%) +0.08s (+ 0.84%) 8.94s 8.97s p=0.005 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53346 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey jakebailey mentioned this pull request Mar 19, 2023
Merged
@jakebailey
Copy link
Member Author

@typescript-bot test this
@typescript-bot test top100
@typescript-bot user test this
@typescript-bot run dt
@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 19, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the extended test suite on this PR at 21ccbb9. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 19, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the diff-based top-repos suite on this PR at 21ccbb9. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 19, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the parallelized Definitely Typed test suite on this PR at 21ccbb9. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 19, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at 21ccbb9. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..53346

Metric main 53346 Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 361,718k (± 0.01%) 361,785k (± 0.01%) +67k (+ 0.02%) 361,717k 361,818k p=0.031 n=6
Parse Time 3.53s (± 0.81%) 3.52s (± 0.71%) ~ 3.49s 3.55s p=0.745 n=6
Bind Time 1.18s (± 0.99%) 1.18s (± 0.88%) ~ 1.17s 1.19s p=0.437 n=6
Check Time 9.51s (± 0.22%) 9.54s (± 0.58%) ~ 9.46s 9.63s p=0.157 n=6
Emit Time 7.96s (± 0.76%) 7.92s (± 0.43%) ~ 7.87s 7.96s p=0.466 n=6
Total Time 22.18s (± 0.37%) 22.17s (± 0.33%) ~ 22.07s 22.25s p=0.748 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,581k (± 0.88%) 185,682k (± 0.03%) 🟩-7,899k (- 4.08%) 185,613k 185,758k p=0.005 n=6
Parse Time 1.58s (± 1.09%) 1.59s (± 0.26%) ~ 1.59s 1.60s p=0.389 n=6
Bind Time 0.82s (± 0.50%) 0.82s (± 0.50%) ~ 0.81s 0.82s p=0.218 n=6
Check Time 10.30s (± 0.43%) 10.19s (± 0.60%) -0.11s (- 1.07%) 10.15s 10.31s p=0.019 n=6
Emit Time 2.99s (± 0.61%) 2.99s (± 0.62%) ~ 2.97s 3.02s p=0.870 n=6
Total Time 15.70s (± 0.35%) 15.60s (± 0.43%) -0.10s (- 0.65%) 15.55s 15.73s p=0.030 n=6
Monaco - node (v16.17.1, x64)
Memory used 345,729k (± 0.01%) 345,750k (± 0.01%) ~ 345,724k 345,778k p=0.078 n=6
Parse Time 2.72s (± 0.43%) 2.71s (± 0.39%) -0.02s (- 0.61%) 2.69s 2.72s p=0.040 n=6
Bind Time 1.08s (± 0.95%) 1.08s (± 1.51%) ~ 1.06s 1.11s p=1.000 n=6
Check Time 7.80s (± 0.44%) 7.79s (± 0.47%) ~ 7.75s 7.84s p=0.810 n=6
Emit Time 4.45s (± 0.39%) 4.46s (± 0.50%) ~ 4.42s 4.48s p=0.685 n=6
Total Time 16.06s (± 0.32%) 16.04s (± 0.31%) ~ 15.97s 16.10s p=0.629 n=6
TFS - node (v16.17.1, x64)
Memory used 300,020k (± 0.01%) 300,010k (± 0.01%) ~ 299,980k 300,033k p=0.378 n=6
Parse Time 2.16s (± 0.62%) 2.16s (± 0.41%) ~ 2.15s 2.17s p=0.801 n=6
Bind Time 1.24s (± 0.83%) 1.23s (± 0.84%) ~ 1.22s 1.24s p=0.652 n=6
Check Time 7.22s (± 0.73%) 7.20s (± 0.50%) ~ 7.16s 7.26s p=0.470 n=6
Emit Time 4.37s (± 1.05%) 4.33s (± 0.60%) ~ 4.30s 4.37s p=0.076 n=6
Total Time 14.99s (± 0.63%) 14.92s (± 0.35%) ~ 14.84s 14.98s p=0.261 n=6
material-ui - node (v16.17.1, x64)
Memory used 476,458k (± 0.01%) 476,720k (± 0.00%) +262k (+ 0.05%) 476,699k 476,744k p=0.005 n=6
Parse Time 3.22s (± 0.38%) 3.22s (± 0.39%) ~ 3.21s 3.24s p=0.675 n=6
Bind Time 0.95s (± 0.54%) 0.95s (± 0.54%) ~ 0.95s 0.96s p=1.000 n=6
Check Time 18.21s (± 0.37%) 18.41s (± 0.63%) +0.19s (+ 1.06%) 18.24s 18.60s p=0.016 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.39s (± 0.28%) 22.57s (± 0.55%) +0.19s (+ 0.85%) 22.41s 22.79s p=0.020 n=6
xstate - node (v16.17.1, x64)
Memory used 550,607k (± 0.02%) 550,711k (± 0.02%) ~ 550,560k 550,951k p=0.230 n=6
Parse Time 3.93s (± 0.49%) 3.93s (± 0.41%) ~ 3.92s 3.96s p=1.000 n=6
Bind Time 1.76s (± 0.43%) 1.75s (± 0.59%) ~ 1.74s 1.77s p=0.351 n=6
Check Time 3.09s (± 1.02%) 3.16s (± 0.48%) +0.07s (+ 2.16%) 3.14s 3.18s p=0.006 n=6
Emit Time 0.09s (± 0.00%) 0.09s (± 0.00%) ~ 0.09s 0.09s p=1.000 n=6
Total Time 8.87s (± 0.60%) 8.93s (± 0.29%) ~ 8.90s 8.96s p=0.064 n=6
System
Machine Namets-ci-ubuntu
Platformlinux 5.4.0-135-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current 53346 6
Baseline main 6

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member Author

So overall it seems like Compiler-Unions is consistently faster and uses less memory (expected), but mui and xstate get slower. Not sure what to make of that. I'm going to hold off on this until I (hopefully soon) get better benchmarks rolling.

@jakebailey jakebailey marked this pull request as draft April 6, 2023 22:04
@jakebailey
Copy link
Member Author

Drafting this for now until I can see where the slowdown is and if it can be avoided.

@jakebailey
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 22, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the regular perf test suite on this PR at 391f090. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 300,344k (± 0.01%) 300,423k (± 0.01%) +79k (+ 0.03%) 300,397k 300,453k p=0.005 n=6
Parse Time 3.03s (± 0.13%) 3.02s (± 0.18%) ~ 3.02s 3.03s p=0.282 n=6
Bind Time 0.93s (± 0.00%) 0.93s (± 0.00%) ~ 0.93s 0.93s p=1.000 n=6
Check Time 9.48s (± 0.32%) 9.49s (± 0.19%) ~ 9.46s 9.51s p=0.566 n=6
Emit Time 7.61s (± 0.11%) 7.63s (± 0.26%) ~ 7.61s 7.66s p=0.271 n=6
Total Time 21.05s (± 0.14%) 21.07s (± 0.12%) ~ 21.04s 21.11s p=0.227 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,914k (± 0.01%) 187,001k (± 0.01%) 🟩-6,912k (- 3.56%) 186,983k 187,019k p=0.005 n=6
Parse Time 1.57s (± 1.17%) 1.55s (± 0.88%) ~ 1.54s 1.58s p=0.226 n=6
Bind Time 0.79s (± 0.52%) 0.79s (± 0.00%) ~ 0.79s 0.79s p=0.405 n=6
Check Time 9.94s (± 0.33%) 9.84s (± 0.40%) -0.10s (- 1.01%) 9.78s 9.90s p=0.008 n=6
Emit Time 2.74s (± 0.33%) 2.74s (± 0.38%) ~ 2.73s 2.76s p=0.673 n=6
Total Time 15.04s (± 0.34%) 14.93s (± 0.34%) -0.11s (- 0.75%) 14.86s 15.00s p=0.013 n=6
Monaco - node (v16.17.1, x64)
Memory used 347,136k (± 0.00%) 347,140k (± 0.01%) ~ 347,099k 347,157k p=0.573 n=6
Parse Time 2.69s (± 0.31%) 2.68s (± 0.24%) ~ 2.67s 2.69s p=0.226 n=6
Bind Time 0.99s (± 0.00%) 0.99s (± 0.00%) ~ 0.99s 0.99s p=1.000 n=6
Check Time 7.93s (± 0.33%) 7.93s (± 0.26%) ~ 7.90s 7.94s p=0.549 n=6
Emit Time 4.27s (± 0.38%) 4.26s (± 0.21%) ~ 4.25s 4.27s p=0.458 n=6
Total Time 15.88s (± 0.16%) 15.86s (± 0.17%) ~ 15.81s 15.88s p=0.506 n=6
TFS - node (v16.17.1, x64)
Memory used 301,145k (± 0.00%) 301,147k (± 0.00%) ~ 301,141k 301,157k p=0.810 n=6
Parse Time 2.18s (± 0.54%) 2.17s (± 1.07%) ~ 2.15s 2.21s p=0.249 n=6
Bind Time 1.11s (± 0.00%) 1.11s (± 0.57%) ~ 1.10s 1.12s p=1.000 n=6
Check Time 7.28s (± 0.58%) 7.24s (± 0.30%) ~ 7.21s 7.27s p=0.076 n=6
Emit Time 3.97s (± 0.25%) 3.98s (± 0.31%) ~ 3.96s 3.99s p=0.601 n=6
Total Time 14.55s (± 0.32%) 14.50s (± 0.28%) ~ 14.45s 14.55s p=0.170 n=6
material-ui - node (v16.17.1, x64)
Memory used 479,467k (± 0.00%) 479,791k (± 0.00%) +324k (+ 0.07%) 479,774k 479,810k p=0.005 n=6
Parse Time 3.15s (± 0.13%) 3.15s (± 0.13%) ~ 3.15s 3.16s p=1.000 n=6
Bind Time 0.91s (± 0.00%) 0.91s (± 0.00%) ~ 0.91s 0.91s p=1.000 n=6
Check Time 17.97s (± 0.21%) 18.09s (± 0.39%) +0.12s (+ 0.65%) 17.98s 18.19s p=0.013 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.03s (± 0.16%) 22.15s (± 0.32%) +0.12s (+ 0.54%) 22.04s 22.25s p=0.013 n=6
xstate - node (v16.17.1, x64)
Memory used 542,834k (± 0.01%) 542,979k (± 0.01%) +145k (+ 0.03%) 542,896k 543,092k p=0.006 n=6
Parse Time 3.70s (± 0.22%) 3.70s (± 0.17%) ~ 3.69s 3.71s p=0.432 n=6
Bind Time 1.34s (± 0.30%) 1.36s (± 3.67%) ~ 1.33s 1.46s p=0.673 n=6
Check Time 3.35s (± 1.06%) 3.36s (± 1.82%) ~ 3.25s 3.42s p=0.520 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 4.99%) ~ 0.08s 0.09s p=0.405 n=6
Total Time 8.47s (± 0.43%) 8.50s (± 0.31%) ~ 8.46s 8.53s p=0.260 n=6
System info unknown
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,493ms (± 0.13%) 2,493ms (± 0.20%) ~ 2,487ms 2,501ms p=0.627 n=6
Req 2 - geterr 5,937ms (± 0.24%) 5,965ms (± 0.14%) +28ms (+ 0.47%) 5,954ms 5,974ms p=0.010 n=6
Req 3 - references 342ms (± 0.12%) 345ms (± 0.92%) +3ms (+ 0.78%) 342ms 351ms p=0.021 n=6
Req 4 - navto 277ms (± 0.27%) 278ms (± 0.78%) ~ 276ms 282ms p=0.858 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 82ms (± 7.63%) 86ms (± 9.23%) ~ 76ms 93ms p=0.415 n=6
CompilerTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,624ms (± 0.50%) 2,627ms (± 0.87%) ~ 2,603ms 2,665ms p=1.000 n=6
Req 2 - geterr 4,772ms (± 0.17%) 4,771ms (± 0.26%) ~ 4,755ms 4,793ms p=0.520 n=6
Req 3 - references 350ms (± 0.12%) 350ms (± 0.24%) ~ 349ms 351ms p=0.285 n=6
Req 4 - navto 269ms (± 0.19%) 269ms (± 0.33%) ~ 268ms 270ms p=0.541 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 79ms (± 0.95%) 79ms (± 0.80%) ~ 78ms 80ms p=0.718 n=6
xstateTSServer - node (v16.17.1, x64)
Req 1 - updateOpen 2,715ms (± 0.19%) 2,712ms (± 0.18%) ~ 2,706ms 2,717ms p=0.197 n=6
Req 2 - geterr 1,934ms (± 2.01%) 1,996ms (± 0.91%) +62ms (+ 3.21%) 1,972ms 2,018ms p=0.005 n=6
Req 3 - references 134ms (± 7.52%) 140ms (± 1.41%) ~ 137ms 141ms p=0.209 n=6
Req 4 - navto 354ms (± 0.46%) 353ms (± 0.45%) ~ 351ms 355ms p=0.683 n=6
Req 5 - completionInfo count 2,071 (± 0.00%) 2,071 (± 0.00%) ~ 2,071 2,071 p=1.000 n=6
Req 5 - completionInfo 319ms (± 2.42%) 325ms (± 1.93%) ~ 315ms 330ms p=0.072 n=6
System info unknown
Hosts
  • node (v16.17.1, x64)
Scenarios
  • CompilerTSServer - node (v16.17.1, x64)
  • Compiler-UnionsTSServer - node (v16.17.1, x64)
  • xstateTSServer - node (v16.17.1, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v16.17.1, x64)
Execution time 156.23ms (± 0.16%) 156.27ms (± 0.18%) ~ 155.16ms 158.70ms p=0.076 n=600
tsserver-startup - node (v16.17.1, x64)
Execution time 230.22ms (± 0.22%) 230.38ms (± 0.12%) +0.16ms (+ 0.07%) 229.36ms 232.81ms p=0.000 n=600
tsserverlibrary-startup - node (v16.17.1, x64)
Execution time 234.85ms (± 0.11%) 236.32ms (± 0.14%) +1.47ms (+ 0.63%) 234.16ms 241.69ms p=0.000 n=600
typescript-startup - node (v16.17.1, x64)
Execution time 235.70ms (± 0.13%) 236.35ms (± 0.13%) +0.64ms (+ 0.27%) 234.54ms 241.19ms p=0.000 n=600
System info unknown
Hosts
  • node (v16.17.1, x64)
Scenarios
  • tsc-startup - node (v16.17.1, x64)
  • tsserver-startup - node (v16.17.1, x64)
  • tsserverlibrary-startup - node (v16.17.1, x64)
  • typescript-startup - node (v16.17.1, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Aug 24, 2023 •
edited
Loading

Heya @jakebailey, I've started to run the tsc-only perf test suite on this PR at 6148cde. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v16.17.1, x64)
Memory used 300,244k (± 0.01%) 300,319k (± 0.01%) +74k (+ 0.02%) 300,297k 300,345k p=0.005 n=6
Parse Time 3.03s (± 0.17%) 3.03s (± 0.29%) ~ 3.02s 3.04s p=0.541 n=6
Bind Time 0.93s (± 0.44%) 0.93s (± 0.00%) ~ 0.93s 0.93s p=0.405 n=6
Check Time 9.28s (± 0.25%) 9.33s (± 0.27%) +0.05s (+ 0.56%) 9.28s 9.35s p=0.009 n=6
Emit Time 7.62s (± 0.36%) 7.65s (± 0.40%) ~ 7.62s 7.69s p=0.142 n=6
Total Time 20.85s (± 0.23%) 20.94s (± 0.18%) +0.09s (+ 0.43%) 20.88s 20.99s p=0.016 n=6
Compiler-Unions - node (v16.17.1, x64)
Memory used 193,955k (± 0.01%) 187,043k (± 0.00%) 🟩-6,913k (- 3.56%) 187,028k 187,051k p=0.005 n=6
Parse Time 1.58s (± 0.26%) 1.58s (± 0.00%) ~ 1.58s 1.58s p=0.405 n=6
Bind Time 0.80s (± 0.51%) 0.80s (± 0.51%) ~ 0.79s 0.80s p=1.000 n=6
Check Time 9.92s (± 0.37%) 9.82s (± 0.33%) -0.10s (- 0.96%) 9.78s 9.87s p=0.006 n=6
Emit Time 2.74s (± 0.15%) 2.74s (± 0.15%) ~ 2.73s 2.74s p=1.000 n=6
Total Time 15.03s (± 0.20%) 14.94s (± 0.24%) -0.09s (- 0.63%) 14.90s 14.99s p=0.006 n=6
Monaco - node (v16.17.1, x64)
Memory used 347,142k (± 0.01%) 347,157k (± 0.01%) ~ 347,123k 347,177k p=0.575 n=6
Parse Time 2.69s (± 0.37%) 2.69s (± 0.19%) ~ 2.68s 2.69s p=0.931 n=6
Bind Time 0.99s (± 0.41%) 0.99s (± 0.00%) ~ 0.99s 0.99s p=0.405 n=6
Check Time 7.90s (± 0.16%) 7.91s (± 0.33%) ~ 7.89s 7.96s p=0.285 n=6
Emit Time 4.27s (± 0.18%) 4.27s (± 0.48%) ~ 4.24s 4.30s p=0.864 n=6
Total Time 15.84s (± 0.10%) 15.86s (± 0.13%) ~ 15.84s 15.90s p=0.162 n=6
TFS - node (v16.17.1, x64)
Memory used 301,156k (± 0.00%) 301,153k (± 0.00%) ~ 301,133k 301,171k p=0.810 n=6
Parse Time 2.17s (± 0.80%) 2.17s (± 0.54%) ~ 2.15s 2.18s p=0.680 n=6
Bind Time 1.11s (± 0.88%) 1.11s (± 0.46%) ~ 1.11s 1.12s p=0.504 n=6
Check Time 7.21s (± 0.29%) 7.20s (± 0.39%) ~ 7.16s 7.23s p=0.567 n=6
Emit Time 3.98s (± 0.46%) 3.98s (± 0.45%) ~ 3.96s 4.01s p=0.935 n=6
Total Time 14.48s (± 0.25%) 14.47s (± 0.23%) ~ 14.41s 14.50s p=0.513 n=6
material-ui - node (v16.17.1, x64)
Memory used 479,446k (± 0.00%) 479,768k (± 0.00%) +322k (+ 0.07%) 479,749k 479,791k p=0.005 n=6
Parse Time 3.15s (± 0.27%) 3.15s (± 0.20%) ~ 3.14s 3.16s p=0.340 n=6
Bind Time 0.91s (± 0.00%) 0.91s (± 0.00%) ~ 0.91s 0.91s p=1.000 n=6
Check Time 17.82s (± 0.49%) 17.99s (± 0.34%) +0.17s (+ 0.94%) 17.95s 18.11s p=0.020 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 21.88s (± 0.41%) 22.05s (± 0.28%) +0.17s (+ 0.78%) 22.00s 22.17s p=0.020 n=6
xstate - node (v16.17.1, x64)
Memory used 542,848k (± 0.02%) 542,913k (± 0.01%) ~ 542,852k 543,007k p=0.066 n=6
Parse Time 3.70s (± 0.00%) 3.70s (± 0.17%) ~ 3.69s 3.71s p=1.000 n=6
Bind Time 1.34s (± 0.39%) 1.34s (± 0.00%) ~ 1.34s 1.34s p=0.174 n=6
Check Time 3.32s (± 0.49%) 3.35s (± 0.44%) +0.03s (+ 0.75%) 3.33s 3.37s p=0.034 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 4.99%) ~ 0.08s 0.09s p=0.405 n=6
Total Time 8.44s (± 0.27%) 8.47s (± 0.21%) +0.03s (+ 0.39%) 8.46s 8.50s p=0.034 n=6
System info unknown
Hosts
  • node (v16.17.1, x64)
Scenarios
  • Angular - node (v16.17.1, x64)
  • Compiler-Unions - node (v16.17.1, x64)
  • Monaco - node (v16.17.1, x64)
  • TFS - node (v16.17.1, x64)
  • material-ui - node (v16.17.1, x64)
  • xstate - node (v16.17.1, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey jakebailey changed the title Avoid calculating all properties of UnionOrIntersectionType when caller can exit early Use iterators to avoid calculating all properties of UnionOrIntersectionType Aug 30, 2023
@sandersn
Copy link
Member

@jakebailey should this still be a draft PR? Do you need another look at it before it's actually ready to merge?

@jakebailey
Copy link
Member Author

I need to do a perf investigation to see if it's possible to avoid the negative performance hit in some benchmarks. That and I think I'm still on the fence as to whether or not this is too clever or if I can get some of the same benefits for the benchmark I was using another way.

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 20, 2024 •
edited
Loading

Heya @jakebailey, I've started to run the faster perf test suite on this PR at 1e89756. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,645k (± 0.00%) 295,701k (± 0.01%) +56k (+ 0.02%) 295,676k 295,733k p=0.005 n=6
Parse Time 2.66s (± 0.34%) 2.66s (± 0.24%) ~ 2.65s 2.67s p=1.000 n=6
Bind Time 0.83s (± 0.90%) 0.83s (± 1.52%) ~ 0.82s 0.85s p=0.676 n=6
Check Time 8.18s (± 0.47%) 8.22s (± 0.43%) ~ 8.17s 8.27s p=0.090 n=6
Emit Time 7.10s (± 0.33%) 7.08s (± 0.20%) ~ 7.06s 7.10s p=0.169 n=6
Total Time 18.78s (± 0.26%) 18.79s (± 0.16%) ~ 18.76s 18.83s p=0.810 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 193,513k (± 1.57%) 186,128k (± 1.60%) 🟩-7,385k (- 3.82%) 184,157k 189,997k p=0.005 n=6
Parse Time 1.37s (± 0.90%) 1.36s (± 0.46%) ~ 1.35s 1.37s p=0.599 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.34s (± 0.31%) 9.26s (± 0.32%) -0.08s (- 0.86%) 9.21s 9.29s p=0.005 n=6
Emit Time 2.62s (± 0.79%) 2.62s (± 0.40%) ~ 2.60s 2.63s p=1.000 n=6
Total Time 14.04s (± 0.19%) 13.96s (± 0.24%) -0.08s (- 0.59%) 13.91s 13.99s p=0.005 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,469k (± 0.01%) 347,467k (± 0.01%) ~ 347,443k 347,494k p=0.688 n=6
Parse Time 2.48s (± 0.67%) 2.48s (± 0.25%) ~ 2.47s 2.49s p=0.720 n=6
Bind Time 0.93s (± 0.44%) 0.93s (± 0.00%) ~ 0.93s 0.93s p=0.405 n=6
Check Time 6.93s (± 0.20%) 6.94s (± 0.16%) ~ 6.93s 6.95s p=0.498 n=6
Emit Time 4.07s (± 0.63%) 4.06s (± 0.57%) ~ 4.03s 4.10s p=0.568 n=6
Total Time 14.41s (± 0.27%) 14.41s (± 0.20%) ~ 14.37s 14.45s p=0.935 n=6
TFS - node (v18.15.0, x64)
Memory used 302,838k (± 0.01%) 302,832k (± 0.00%) ~ 302,810k 302,845k p=0.471 n=6
Parse Time 2.01s (± 1.49%) 2.01s (± 0.51%) ~ 2.00s 2.02s p=0.666 n=6
Bind Time 1.00s (± 0.00%) 1.00s (± 0.98%) ~ 0.99s 1.02s p=1.000 n=6
Check Time 6.32s (± 0.31%) 6.33s (± 0.41%) ~ 6.31s 6.37s p=0.257 n=6
Emit Time 3.59s (± 0.56%) 3.60s (± 0.57%) ~ 3.58s 3.64s p=0.506 n=6
Total Time 12.92s (± 0.44%) 12.96s (± 0.17%) ~ 12.93s 12.99s p=0.335 n=6
material-ui - node (v18.15.0, x64)
Memory used 511,288k (± 0.01%) 511,573k (± 0.00%) +286k (+ 0.06%) 511,539k 511,597k p=0.005 n=6
Parse Time 2.65s (± 0.69%) 2.65s (± 0.57%) ~ 2.63s 2.67s p=0.616 n=6
Bind Time 1.00s (± 0.55%) 1.00s (± 0.75%) ~ 0.99s 1.01s p=0.476 n=6
Check Time 17.19s (± 0.37%) 17.40s (± 0.23%) +0.21s (+ 1.24%) 17.35s 17.45s p=0.005 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.83s (± 0.37%) 21.06s (± 0.19%) +0.23s (+ 1.09%) 21.01s 21.11s p=0.005 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,695,927k (± 0.00%) 1,696,921k (± 0.00%) +994k (+ 0.06%) 1,696,876k 1,696,968k p=0.005 n=6
Parse Time 6.53s (± 0.21%) 6.54s (± 0.38%) ~ 6.50s 6.57s p=1.000 n=6
Bind Time 2.35s (± 0.44%) 2.35s (± 0.50%) ~ 2.34s 2.37s p=0.279 n=6
Check Time 55.40s (± 0.69%) 56.00s (± 0.24%) +0.59s (+ 1.07%) 55.79s 56.19s p=0.010 n=6
Emit Time 0.16s (± 2.52%) 0.17s (± 3.10%) ~ 0.16s 0.17s p=0.112 n=6
Total Time 64.45s (± 0.58%) 65.05s (± 0.24%) +0.59s (+ 0.92%) 64.80s 65.24s p=0.013 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,413,178k (± 0.03%) 2,407,985k (± 0.02%) -5,193k (- 0.22%) 2,407,006k 2,408,432k p=0.005 n=6
Parse Time 4.96s (± 1.07%) 4.93s (± 0.80%) ~ 4.87s 4.97s p=0.687 n=6
Bind Time 1.88s (± 1.06%) 1.88s (± 0.80%) ~ 1.87s 1.91s p=0.675 n=6
Check Time 33.57s (± 0.24%) 33.61s (± 0.57%) ~ 33.44s 33.88s p=0.810 n=6
Emit Time 2.70s (± 1.73%) 2.72s (± 1.35%) ~ 2.68s 2.76s p=0.199 n=6
Total Time 43.13s (± 0.28%) 43.16s (± 0.45%) ~ 42.94s 43.44s p=0.936 n=6
self-compiler - node (v18.15.0, x64)
Memory used 419,737k (± 0.02%) 416,598k (± 0.00%) -3,139k (- 0.75%) 416,566k 416,622k p=0.005 n=6
Parse Time 2.74s (± 3.08%) 2.77s (± 2.94%) ~ 2.65s 2.85s p=0.687 n=6
Bind Time 1.15s (± 6.86%) 1.12s (± 6.93%) ~ 1.07s 1.23s p=0.278 n=6
Check Time 15.13s (± 0.28%) 15.11s (± 0.23%) ~ 15.07s 15.15s p=0.377 n=6
Emit Time 1.16s (± 0.44%) 1.14s (± 1.43%) -0.02s (- 1.72%) 1.12s 1.17s p=0.039 n=6
Total Time 20.19s (± 0.16%) 20.15s (± 0.21%) ~ 20.10s 20.22s p=0.054 n=6
vscode - node (v18.15.0, x64)
Memory used 2,807,359k (± 0.00%) 2,807,177k (± 0.00%) -182k (- 0.01%) 2,807,155k 2,807,200k p=0.005 n=6
Parse Time 10.63s (± 0.20%) 10.62s (± 0.33%) ~ 10.59s 10.68s p=0.333 n=6
Bind Time 3.38s (± 0.39%) 3.38s (± 0.26%) ~ 3.37s 3.39s p=0.801 n=6
Check Time 59.56s (± 0.33%) 60.00s (± 0.48%) +0.44s (+ 0.74%) 59.68s 60.45s p=0.020 n=6
Emit Time 16.11s (± 0.39%) 16.11s (± 0.57%) ~ 15.97s 16.25s p=0.810 n=6
Total Time 89.69s (± 0.24%) 90.11s (± 0.29%) +0.42s (+ 0.47%) 89.75s 90.49s p=0.016 n=6
webpack - node (v18.15.0, x64)
Memory used 392,338k (± 0.01%) 392,362k (± 0.01%) ~ 392,322k 392,440k p=0.470 n=6
Parse Time 3.05s (± 0.70%) 3.05s (± 0.69%) ~ 3.03s 3.08s p=1.000 n=6
Bind Time 1.40s (± 0.83%) 1.40s (± 0.98%) ~ 1.38s 1.42s p=0.410 n=6
Check Time 13.98s (± 0.44%) 14.00s (± 0.17%) ~ 13.98s 14.04s p=0.421 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 18.43s (± 0.35%) 18.45s (± 0.29%) ~ 18.41s 18.54s p=0.422 n=6
xstate - node (v18.15.0, x64)
Memory used 513,401k (± 0.01%) 513,507k (± 0.02%) ~ 513,370k 513,662k p=0.093 n=6
Parse Time 3.27s (± 0.36%) 3.28s (± 0.30%) ~ 3.27s 3.29s p=0.157 n=6
Bind Time 1.54s (± 0.67%) 1.54s (± 0.27%) ~ 1.53s 1.54s p=0.390 n=6
Check Time 2.85s (± 0.56%) 2.88s (± 0.75%) +0.03s (+ 1.11%) 2.86s 2.92s p=0.012 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 4.99%) ~ 0.08s 0.09s p=0.405 n=6
Total Time 7.75s (± 0.25%) 7.79s (± 0.24%) +0.04s (+ 0.54%) 7.77s 7.82s p=0.006 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 27, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,025 -128 (- 0.21%) ~ ~ p=0.001 n=6
Types 50,242 33,050 🟩-17,192 (-34.22%) ~ ~ p=0.001 n=6
Memory used 193,320k (± 0.94%) 185,932k (± 0.99%) 🟩-7,388k (- 3.82%) 184,724k 188,321k p=0.005 n=6
Parse Time 1.31s (± 0.31%) 1.31s (± 0.57%) ~ 1.30s 1.32s p=0.389 n=6
Bind Time 0.71s (± 0.57%) 0.71s ~ ~ ~ p=0.405 n=6
Check Time 9.42s (± 0.38%) 9.35s (± 0.53%) -0.07s (- 0.76%) 9.28s 9.43s p=0.030 n=6
Emit Time 2.75s (± 0.76%) 2.75s (± 0.30%) ~ 2.75s 2.77s p=0.720 n=6
Total Time 14.19s (± 0.21%) 14.13s (± 0.34%) -0.06s (- 0.43%) 14.06s 14.20s p=0.045 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 943,936 -178 (- 0.02%) ~ ~ p=0.001 n=6
Types 407,050 407,001 -49 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 1,218,357k (± 0.00%) 1,218,215k (± 0.00%) -141k (- 0.01%) 1,218,173k 1,218,276k p=0.005 n=6
Parse Time 6.67s (± 1.02%) 6.70s (± 0.78%) ~ 6.61s 6.75s p=0.470 n=6
Bind Time 1.87s (± 1.06%) 1.86s (± 0.65%) ~ 1.85s 1.88s p=0.621 n=6
Check Time 30.57s (± 0.59%) 30.67s (± 0.28%) ~ 30.56s 30.77s p=0.149 n=6
Emit Time 13.60s (± 0.37%) 13.56s (± 0.39%) ~ 13.49s 13.63s p=0.199 n=6
Total Time 52.71s (± 0.28%) 52.80s (± 0.29%) ~ 52.57s 52.99s p=0.334 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,133,042 2,086,342 -46,700 (- 2.19%) ~ ~ p=0.001 n=6
Types 926,437 925,178 -1,259 (- 0.14%) ~ ~ p=0.001 n=6
Memory used 2,115,495k (± 0.01%) 2,052,430k (± 0.00%) -63,065k (- 2.98%) 2,052,361k 2,052,501k p=0.005 n=6
Parse Time 6.64s (± 0.39%) 6.65s (± 0.62%) ~ 6.57s 6.69s p=0.567 n=6
Bind Time 2.33s (± 1.21%) 2.33s (± 0.70%) ~ 2.30s 2.34s p=0.858 n=6
Check Time 70.17s (± 1.68%) 62.81s (± 1.38%) 🟩-7.36s (-10.49%) 62.24s 64.57s p=0.005 n=6
Emit Time 0.14s (± 3.77%) 0.14s ~ ~ ~ p=0.174 n=6
Total Time 79.28s (± 1.47%) 71.93s (± 1.14%) 🟩-7.35s (- 9.27%) 71.37s 73.58s p=0.005 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,231,640 1,228,743 -2,897 (- 0.24%) ~ ~ p=0.001 n=6
Types 261,187 259,689 -1,498 (- 0.57%) ~ ~ p=0.001 n=6
Memory used 2,347,039k (± 0.01%) 2,342,836k (± 0.03%) -4,203k (- 0.18%) 2,342,007k 2,343,353k p=0.005 n=6
Parse Time 5.03s (± 0.53%) 5.07s (± 1.21%) ~ 5.00s 5.17s p=0.171 n=6
Bind Time 1.93s (± 0.85%) 1.93s (± 0.78%) ~ 1.92s 1.96s p=0.514 n=6
Check Time 34.14s (± 0.25%) 34.21s (± 0.21%) ~ 34.11s 34.28s p=0.173 n=6
Emit Time 2.78s (± 1.33%) 2.73s (± 1.65%) ~ 2.65s 2.79s p=0.078 n=6
Total Time 43.89s (± 0.19%) 43.97s (± 0.23%) ~ 43.85s 44.12s p=0.149 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,231,640 1,228,743 -2,897 (- 0.24%) ~ ~ p=0.001 n=6
Types 261,187 259,689 -1,498 (- 0.57%) ~ ~ p=0.001 n=6
Memory used 2,424,172k (± 0.04%) 2,419,242k (± 0.04%) -4,930k (- 0.20%) 2,417,325k 2,420,005k p=0.005 n=6
Parse Time 6.25s (± 0.69%) 6.25s (± 1.18%) ~ 6.13s 6.35s p=0.936 n=6
Bind Time 2.03s (± 0.96%) 2.04s (± 0.74%) ~ 2.02s 2.06s p=0.462 n=6
Check Time 40.45s (± 0.23%) 40.64s (± 0.13%) +0.19s (+ 0.47%) 40.59s 40.74s p=0.005 n=6
Emit Time 3.22s (± 2.96%) 3.22s (± 2.93%) ~ 3.08s 3.36s p=1.000 n=6
Total Time 51.97s (± 0.29%) 52.17s (± 0.28%) +0.20s (+ 0.39%) 51.88s 52.29s p=0.045 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,818 256,680 -2,138 (- 0.83%) ~ ~ p=0.001 n=6
Types 104,842 103,913 -929 (- 0.89%) ~ ~ p=0.001 n=6
Memory used 428,227k (± 0.01%) 424,690k (± 0.02%) -3,538k (- 0.83%) 424,608k 424,832k p=0.005 n=6
Parse Time 4.08s (± 0.29%) 4.11s (± 0.69%) +0.03s (+ 0.73%) 4.08s 4.16s p=0.042 n=6
Bind Time 1.62s (± 1.03%) 1.61s (± 1.39%) ~ 1.59s 1.65s p=0.570 n=6
Check Time 21.94s (± 0.20%) 21.93s (± 0.54%) ~ 21.74s 22.07s p=0.936 n=6
Emit Time 1.72s (± 0.95%) 1.73s (± 1.29%) ~ 1.70s 1.76s p=0.560 n=6
Total Time 29.36s (± 0.11%) 29.38s (± 0.38%) ~ 29.26s 29.53s p=0.630 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 223,775 -790 (- 0.35%) ~ ~ p=0.001 n=6
Types 93,734 93,676 -58 (- 0.06%) ~ ~ p=0.001 n=6
Memory used 369,505k (± 0.04%) 367,893k (± 0.03%) -1,612k (- 0.44%) 367,782k 368,082k p=0.005 n=6
Parse Time 2.76s (± 0.85%) 2.80s (± 0.63%) +0.04s (+ 1.45%) 2.77s 2.82s p=0.012 n=6
Bind Time 1.59s (± 0.95%) 1.59s (± 0.62%) ~ 1.58s 1.60s p=0.498 n=6
Check Time 15.47s (± 0.14%) 15.48s (± 0.41%) ~ 15.42s 15.57s p=1.000 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.81s (± 0.22%) 19.87s (± 0.39%) ~ 19.77s 19.97s p=0.227 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,878,585 2,878,375 -210 (- 0.01%) ~ ~ p=0.001 n=6
Types 975,168 975,084 -84 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 3,042,077k (± 0.00%) 3,041,922k (± 0.00%) -155k (- 0.01%) 3,041,876k 3,042,026k p=0.005 n=6
Parse Time 13.55s (± 0.15%) 13.53s (± 0.44%) ~ 13.44s 13.59s p=1.000 n=6
Bind Time 4.18s (± 0.35%) 4.18s (± 0.64%) ~ 4.14s 4.22s p=0.454 n=6
Check Time 73.28s (± 0.16%) 73.37s (± 0.14%) ~ 73.23s 73.50s p=0.298 n=6
Emit Time 23.99s (± 0.53%) 24.00s (± 0.67%) ~ 23.83s 24.28s p=1.000 n=6
Total Time 115.00s (± 0.16%) 115.09s (± 0.21%) ~ 114.79s 115.44s p=0.471 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,598k (± 0.02%) 411,605k (± 0.01%) ~ 411,522k 411,657k p=0.689 n=6
Parse Time 3.81s (± 0.40%) 3.82s (± 0.46%) ~ 3.80s 3.85s p=0.315 n=6
Bind Time 1.69s (± 0.48%) 1.70s (± 0.32%) ~ 1.69s 1.70s p=0.088 n=6
Check Time 16.72s (± 0.32%) 16.78s (± 0.39%) ~ 16.69s 16.88s p=0.146 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.23s (± 0.30%) 22.30s (± 0.28%) ~ 22.23s 22.40s p=0.127 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,869k (± 0.08%) 462,945k (± 0.07%) ~ 462,566k 463,363k p=0.810 n=6
Parse Time 3.18s (± 0.94%) 3.16s (± 0.74%) ~ 3.13s 3.20s p=0.317 n=6
Bind Time 1.17s (± 0.54%) 1.17s (± 0.35%) ~ 1.16s 1.17s p=0.673 n=6
Check Time 17.89s (± 0.27%) 18.00s (± 0.31%) +0.11s (+ 0.61%) 17.94s 18.10s p=0.010 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.23s (± 0.20%) 22.33s (± 0.32%) +0.10s (+ 0.43%) 22.23s 22.44s p=0.045 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member Author

Wow, now this PR gets a 10% speedup in mui-docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@weswigham weswigham weswigham approved these changes

@ahejlsberg ahejlsberg Awaiting requested review from ahejlsberg

Assignees

@jakebailey jakebailey

Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jakebailey @typescript-bot @sandersn @weswigham