Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source files not affected by all module resolution options #55790

Merged
merged 4 commits into from
Sep 20, 2023

Conversation

sheetalkamat
Copy link
Member

@sheetalkamat sheetalkamat commented Sep 19, 2023

Stops taking dependency on all module resolution options when sharing sourceFiles and instead moves resolvedModules and resolvedTypeReferenceDirectiveNames from sourceFile to Program so it is safe by program to copy the resolutions across.

This came up as part of testing with #54785 where axion and some more (which i dont recollect) didnt pass incremental resolution check. This is shown in ddaf1c7 specifically (ddaf1c7#diff-215d7c39ab1082f80d4c3a4f1c8bd9dd1c9ccb1c6450192bbaf409b8e5376579R140) where the resolved module on sourceFile changes and results in issues program referencing other resolution.
The issue here lies in fact that "as type" resolution takes into account typeRoots and if config had it its good and used as a key for affecting resolution but if user didnt specify it depends on "config file location" if present otherwise "currentDirectory" . This is issue because you could include either by types or default types (matching in both projects) a type thats in parent directories node_modules/@types but then that type could include types directive of say node which in both projects will be resolved say from config location which is different and if those directories have their own node_modules/@types/node then the location which is picked will depend on which project is picked first
Investigated few options for this

  • adding default type roots as key but that mean resolutions cannot be shared between two projects with config file in different directories which is very common.
  • On top of above option adding only present node_modules/@types in the key but that means on each npm install we would need to update module resolutions from scratch as its change in "key" and parse "sourceFiles" so that doesnt seem practical
  • The current change where decoupling sourceFiles and module resolution storage, This doesnt fix module resolution cache sharing across projects (which is only in tsbuild for now which doesnt share the caches between watch events and in future we will share in tsserver but that one will watch failed lookup locations so we could somehow use that so probably can be handled in some way without having to invalidate everything)

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Sep 19, 2023
@sheetalkamat

This comment was marked as resolved.

@typescript-bot

This comment was marked as resolved.

@typescript-bot

This comment was marked as resolved.

@typescript-bot

This comment was marked as resolved.

@typescript-bot

This comment was marked as resolved.

@typescript-bot

This comment was marked as resolved.

@typescript-bot

This comment was marked as resolved.

@typescript-bot

This comment was marked as resolved.

@typescript-bot

This comment was marked as resolved.

@typescript-bot

This comment was marked as resolved.

@sheetalkamat sheetalkamat force-pushed the sourceFilesNotAffectedByModuleResolution branch from dc1a244 to 63226c2 Compare September 19, 2023 23:16
@typescript-bot

This comment was marked as resolved.

@sheetalkamat
Copy link
Member Author

@typescript-bot test this
@typescript-bot test top100
@typescript-bot test tsserver top100
@typescript-bot user test this
@typescript-bot user test tsserver
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 20, 2023

Heya @sheetalkamat, I've started to run the diff-based user code test suite on this PR at 6a43bc1. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 20, 2023

Heya @sheetalkamat, I've started to run the diff-based top-repos suite on this PR at 6a43bc1. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 20, 2023

Heya @sheetalkamat, I've started to run the diff-based top-repos suite (tsserver) on this PR at 6a43bc1. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 20, 2023

Heya @sheetalkamat, I've started to run the regular perf test suite on this PR at 6a43bc1. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 20, 2023

Heya @sheetalkamat, I've started to run the diff-based user code test suite (tsserver) on this PR at 6a43bc1. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@sheetalkamat Here are the results of running the user test suite comparing main and refs/pull/55790/merge:

There were infrastructure failures potentially unrelated to your change:

  • 1 instance of "Unknown failure"
  • 2 instances of "Package install failed"

Otherwise...

Everything looks good!

@typescript-bot
Copy link
Collaborator

@sheetalkamat
The results of the perf run you requested are in!

Here they are:

Compiler

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 294,913k (± 0.01%) 294,919k (± 0.01%) ~ 294,872k 294,950k p=0.936 n=6
Parse Time 2.63s (± 0.21%) 2.63s (± 0.48%) ~ 2.61s 2.64s p=0.663 n=6
Bind Time 0.84s (± 1.22%) 0.84s (± 1.17%) ~ 0.83s 0.85s p=0.784 n=6
Check Time 8.00s (± 0.29%) 8.00s (± 0.38%) ~ 7.96s 8.05s p=1.000 n=6
Emit Time 7.02s (± 0.19%) 7.01s (± 0.24%) ~ 6.99s 7.03s p=0.505 n=6
Total Time 18.49s (± 0.12%) 18.48s (± 0.13%) ~ 18.45s 18.51s p=0.511 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,525k (± 1.54%) 193,575k (± 1.67%) ~ 190,610k 196,653k p=0.298 n=6
Parse Time 1.36s (± 0.72%) 1.36s (± 0.55%) ~ 1.35s 1.37s p=1.000 n=6
Bind Time 0.73s (± 0.00%) 0.73s (± 0.00%) ~ 0.73s 0.73s p=1.000 n=6
Check Time 9.07s (± 0.52%) 9.07s (± 0.31%) ~ 9.04s 9.11s p=0.872 n=6
Emit Time 2.60s (± 0.52%) 2.61s (± 1.10%) ~ 2.58s 2.66s p=0.624 n=6
Total Time 13.77s (± 0.41%) 13.78s (± 0.40%) ~ 13.71s 13.84s p=0.746 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,182k (± 0.00%) 347,201k (± 0.00%) +20k (+ 0.01%) 347,189k 347,215k p=0.013 n=6
Parse Time 2.47s (± 0.30%) 2.47s (± 0.36%) ~ 2.46s 2.48s p=0.798 n=6
Bind Time 0.94s (± 0.55%) 0.94s (± 0.43%) ~ 0.94s 0.95s p=0.114 n=6
Check Time 6.86s (± 0.54%) 6.88s (± 0.47%) ~ 6.83s 6.92s p=0.466 n=6
Emit Time 4.03s (± 0.56%) 4.04s (± 0.48%) ~ 4.01s 4.06s p=0.569 n=6
Total Time 14.30s (± 0.34%) 14.32s (± 0.26%) ~ 14.27s 14.38s p=0.416 n=6
TFS - node (v18.15.0, x64)
Memory used 302,474k (± 0.01%) 302,476k (± 0.01%) ~ 302,457k 302,496k p=0.575 n=6
Parse Time 2.00s (± 0.77%) 2.01s (± 0.75%) ~ 1.99s 2.03s p=0.170 n=6
Bind Time 1.01s (± 1.62%) 1.00s (± 1.03%) ~ 0.99s 1.02s p=0.869 n=6
Check Time 6.25s (± 0.40%) 6.24s (± 0.40%) ~ 6.21s 6.28s p=0.413 n=6
Emit Time 3.51s (± 1.06%) 3.50s (± 0.38%) ~ 3.48s 3.51s p=0.605 n=6
Total Time 12.78s (± 0.35%) 12.76s (± 0.23%) ~ 12.73s 12.80s p=0.332 n=6
material-ui - node (v18.15.0, x64)
Memory used 470,439k (± 0.00%) 470,423k (± 0.00%) ~ 470,410k 470,453k p=0.173 n=6
Parse Time 2.59s (± 0.66%) 2.58s (± 0.79%) ~ 2.55s 2.61s p=0.408 n=6
Bind Time 1.00s (± 0.55%) 0.99s (± 1.18%) ~ 0.98s 1.01s p=0.498 n=6
Check Time 16.63s (± 0.37%) 16.59s (± 0.45%) ~ 16.47s 16.68s p=0.258 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.21s (± 0.25%) 20.16s (± 0.41%) ~ 20.05s 20.27s p=0.226 n=6
xstate - node (v18.15.0, x64)
Memory used 512,192k (± 0.01%) 512,468k (± 0.02%) +276k (+ 0.05%) 512,375k 512,633k p=0.005 n=6
Parse Time 3.26s (± 0.42%) 3.27s (± 0.36%) +0.02s (+ 0.51%) 3.25s 3.28s p=0.048 n=6
Bind Time 1.55s (± 0.41%) 1.55s (± 0.41%) ~ 1.54s 1.56s p=1.000 n=6
Check Time 2.77s (± 0.80%) 2.79s (± 0.67%) ~ 2.76s 2.81s p=0.168 n=6
Emit Time 0.08s (± 0.00%) 0.08s (± 0.00%) ~ 0.08s 0.08s p=1.000 n=6
Total Time 7.65s (± 0.42%) 7.68s (± 0.23%) ~ 7.66s 7.71s p=0.108 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,379ms (± 2.05%) 2,362ms (± 1.18%) ~ 2,331ms 2,395ms p=0.575 n=6
Req 2 - geterr 5,320ms (± 1.69%) 5,353ms (± 1.52%) ~ 5,243ms 5,433ms p=0.936 n=6
Req 3 - references 332ms (± 1.46%) 331ms (± 1.11%) ~ 326ms 335ms p=0.872 n=6
Req 4 - navto 277ms (± 0.54%) 276ms (± 0.50%) ~ 274ms 277ms p=0.187 n=6
Req 5 - completionInfo count 1,356 (± 0.00%) 1,356 (± 0.00%) ~ 1,356 1,356 p=1.000 n=6
Req 5 - completionInfo 80ms (± 7.73%) 85ms (± 8.46%) ~ 76ms 90ms p=0.204 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,469ms (± 0.66%) 2,461ms (± 0.72%) ~ 2,437ms 2,482ms p=0.298 n=6
Req 2 - geterr 4,097ms (± 2.00%) 4,100ms (± 1.77%) ~ 4,027ms 4,196ms p=0.688 n=6
Req 3 - references 338ms (± 1.42%) 339ms (± 1.66%) ~ 332ms 344ms p=0.808 n=6
Req 4 - navto 283ms (± 0.36%) 284ms (± 0.29%) ~ 282ms 284ms p=0.672 n=6
Req 5 - completionInfo count 1,518 (± 0.00%) 1,518 (± 0.00%) ~ 1,518 1,518 p=1.000 n=6
Req 5 - completionInfo 82ms (± 8.01%) 82ms (± 7.78%) ~ 76ms 88ms p=0.859 n=6
xstateTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,600ms (± 0.14%) 2,597ms (± 0.53%) ~ 2,576ms 2,613ms p=0.521 n=6
Req 2 - geterr 1,700ms (± 2.34%) 1,684ms (± 2.53%) ~ 1,622ms 1,732ms p=0.630 n=6
Req 3 - references 114ms (± 9.21%) 116ms (± 7.35%) ~ 106ms 126ms p=0.871 n=6
Req 4 - navto 365ms (± 0.67%) 365ms (± 0.15%) ~ 364ms 365ms p=0.675 n=6
Req 5 - completionInfo count 2,071 (± 0.00%) 2,071 (± 0.00%) ~ 2,071 2,071 p=1.000 n=6
Req 5 - completionInfo 304ms (± 1.66%) 307ms (± 0.49%) ~ 304ms 308ms p=0.091 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstateTSServer - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 151.95ms (± 0.17%) 151.87ms (± 0.17%) -0.08ms (- 0.05%) 150.78ms 154.16ms p=0.001 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 228.56ms (± 0.14%) 227.27ms (± 0.30%) -1.29ms (- 0.57%) 225.18ms 231.65ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 228.10ms (± 0.15%) 228.14ms (± 0.16%) ~ 226.47ms 234.63ms p=0.425 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 227.83ms (± 0.15%) 227.81ms (± 0.16%) ~ 226.37ms 232.82ms p=0.379 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@sheetalkamat Here are the results of running the user test suite comparing main and refs/pull/55790/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@sheetalkamat Here are the results of running the top-repos suite comparing main and refs/pull/55790/merge:

Everything looks good!

1 similar comment
@typescript-bot
Copy link
Collaborator

@sheetalkamat Here are the results of running the top-repos suite comparing main and refs/pull/55790/merge:

Everything looks good!

@sheetalkamat sheetalkamat force-pushed the sourceFilesNotAffectedByModuleResolution branch from 6a43bc1 to 7be4b04 Compare September 20, 2023 17:28
@sheetalkamat sheetalkamat changed the title [WIP] Source files not affected by module resolution Source files not affected by all module resolution options Sep 20, 2023
@sheetalkamat sheetalkamat marked this pull request as ready for review September 20, 2023 17:37
@DanielRosenwasser
Copy link
Member

DanielRosenwasser commented Sep 20, 2023

Can you give some more details on what this change is regarding in the PR description? I can see baselines changing, but I'm not entirely sure what was wrong before.

@sheetalkamat
Copy link
Member Author

@DanielRosenwasser Was just in process of writing

@@ -1,4 +1,5 @@
[
"Module 'ph' was resolved as locally declared ambient module in file '/a/b/node_modules/@types/node/ph.d.ts'.",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is something to do with how we compile the program twice. which i dont understand clearly but when i run the scenario as "tsc" this trace is expected and was missing before.

@@ -69,7 +69,7 @@ Info seq [hh:mm:ss:mss] Open files:
Info seq [hh:mm:ss:mss] FileName: /user/username/projects/myproject/index.ts ProjectRootPath: undefined
Info seq [hh:mm:ss:mss] Projects: /user/username/projects/myproject/tsconfig.json
DocumentRegistry::
Key:: undefined|undefined|undefined|undefined|false|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined|undefined
Key:: undefined|undefined|undefined|false|undefined|undefined|undefined|undefined|undefined|undefined
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This just shortens the key because now many of the module resolution affecting options are not accounted in sourceFile sharing

@jakebailey
Copy link
Member

Just to confirm my understanding, this moves properties off of SourceFile onto Program, such that the SourceFile can be shared more and has fewer props, and the diff is just all of the old places now calling the Program to get the same info, without any other behavior changes?

@andrewbranch
Copy link
Member

It seems like this could actually inflate memory usage (you can see a couple modest increases in some of the perf tests) when there are many programs with identical moduleResolution-related settings and some common dependencies. If the resolution-affecting options are the same between programs, those cases already had optimal SourceFile reuse but will now duplicate the resolutions (or at least maps of pointers to them) across each Program. Would it be valuable to move the resolutions not just to Program but to a shared, properly-keyed data structure similar to DocumentRegistry?

@sheetalkamat
Copy link
Member Author

I was thinking as a next step we could try and use module resolution cache to get the result instead of storing it in the program but that needs some thinking( since in most cases resolution already exists in the cache)

storing on document registry would need so comparisons and invalidation logic that would lead to similar issue as I am trying to fix here (resolving type reference of auto type from different locations when two projects don’t specify type roots )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants