Skip to content
This repository has been archived by the owner on Aug 31, 2023. It is now read-only.

refactor(rome_formatter): Pre-compute suppressions #2902

Merged
merged 8 commits into from
Jul 20, 2022

Conversation

MichaReiser
Copy link
Contributor

@MichaReiser MichaReiser commented Jul 19, 2022

Summary

This PR changes how the formatter extracts formatter suppression comments. The existing implementation lazily traversed to a node's first token and it checks if the token has any leading suppression comments inside of context.is_suppressed().

This PR iterates over the CST before starting the actual formatting to extract all suppressed nodes and it stores them in an index that can be queried during formatting.

This is mainly preparing for the migration to extract leading and trailing node comments with the re-work of the comments handling. Determining if a node is suppressed then becomes as simple as iterating over the leading comments of a node and determining if it has any suppression comment.

This change also shows that the current implementation that calls node.first_token is roughly as expensive as iterating the whole tree ahead of time and computing the (suppression) comments.

This change has the added benefit that suppressions are now a core formatter concept.

Part of #2768

Other changes

This PR further fixes a few places where the formatter didn't correctly check if a node has any suppression comment and if so, prevents it from changing the formatting.

Why add suppressions to the FormatContext

The main reason for storing the suppressions on the context rather than on the state is that the context, unlike the FormatState, is language specific, and thus, it's possible to have a Language type parameter on the Suppressions type.

The one thing I dislike about adding it to the context is that the context is changed AFTER its creation.

Performance

The performance of the two approaches is roughly the same.

Future improvement

It would be possible to improve Rome's overall performance if the workspace provides a way to combine the visitors run for the linter and the visitor for computing the suppression comments.

Test Plan

cargo test

@MichaReiser MichaReiser temporarily deployed to aws July 19, 2022 17:00 Inactive
@MichaReiser
Copy link
Contributor Author

!bench_formatter

@github-actions
Copy link

github-actions bot commented Jul 19, 2022

@github-actions
Copy link

github-actions bot commented Jul 19, 2022

Parser conformance results on ubuntu-latest

js/262

Test result main count This PR count Difference
Total 45878 45878 0
Passed 44938 44938 0
Failed 940 940 0
Panics 0 0 0
Coverage 97.95% 97.95% 0.00%

jsx/babel

Test result main count This PR count Difference
Total 39 39 0
Passed 36 36 0
Failed 3 3 0
Panics 0 0 0
Coverage 92.31% 92.31% 0.00%

symbols/microsoft

Test result main count This PR count Difference
Total 5946 5946 0
Passed 395 395 0
Failed 5551 5551 0
Panics 0 0 0
Coverage 6.64% 6.64% 0.00%

ts/babel

Test result main count This PR count Difference
Total 588 588 0
Passed 519 519 0
Failed 69 69 0
Panics 0 0 0
Coverage 88.27% 88.27% 0.00%

ts/microsoft

Test result main count This PR count Difference
Total 16257 16257 0
Passed 12393 12393 0
Failed 3864 3864 0
Panics 0 0 0
Coverage 76.23% 76.23% 0.00%

@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Jul 19, 2022

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: e65085a
Status: ✅  Deploy successful!
Preview URL: https://aa671e40.tools-8rn.pages.dev
Branch Preview URL: https://refactor-formatter-suppressi.tools-8rn.pages.dev

View logs

@github-actions
Copy link

Formatter Benchmark Results

group                                    main                                   pr
-----                                    ----                                   --
formatter/checker.ts                     1.00    429.5±9.83ms     6.1 MB/sec    1.01   432.8±15.22ms     6.0 MB/sec
formatter/compiler.js                    1.00    279.7±9.25ms     3.7 MB/sec    1.04    291.4±9.94ms     3.6 MB/sec
formatter/d3.min.js                      1.00    229.4±5.56ms  1170.1 KB/sec    1.01   230.6±10.22ms  1164.2 KB/sec
formatter/dojo.js                        1.00     14.3±0.44ms     4.8 MB/sec    1.05     15.0±0.53ms     4.6 MB/sec
formatter/ios.d.ts                       1.00    269.1±7.32ms     6.9 MB/sec    1.06   285.0±10.19ms     6.5 MB/sec
formatter/jquery.min.js                  1.00     59.4±5.01ms  1425.0 KB/sec    1.02     60.3±3.10ms  1402.5 KB/sec
formatter/math.js                        1.00    471.0±8.91ms  1407.7 KB/sec    1.10   518.2±20.84ms  1279.5 KB/sec
formatter/parser.ts                      1.00      9.3±0.27ms     5.2 MB/sec    1.05      9.8±0.50ms     5.0 MB/sec
formatter/pixi.min.js                    1.00    255.9±8.16ms  1756.6 KB/sec    1.05   267.8±13.42ms  1678.2 KB/sec
formatter/react-dom.production.min.js    1.00     68.6±2.75ms  1717.8 KB/sec    1.07     73.2±3.99ms  1609.7 KB/sec
formatter/react.production.min.js        1.00      3.6±0.16ms  1760.8 KB/sec    1.08      3.9±0.15ms  1634.5 KB/sec
formatter/router.ts                      1.00      6.9±0.20ms     8.6 MB/sec    1.02      7.1±0.29ms     8.4 MB/sec
formatter/tex-chtml-full.js              1.00   599.4±12.51ms  1556.7 KB/sec    1.08   644.6±17.98ms  1447.7 KB/sec
formatter/three.min.js                   1.00    296.5±6.69ms  2027.8 KB/sec    1.08   320.8±18.43ms  1873.8 KB/sec
formatter/typescript.js                  1.00  1824.3±58.58ms     5.2 MB/sec    1.08  1970.9±107.96ms     4.8 MB/sec
formatter/vue.global.prod.js             1.00    104.5±9.56ms  1180.3 KB/sec    1.02    107.1±8.68ms  1152.3 KB/sec

@MichaReiser MichaReiser force-pushed the refactor/formatter-suppressions branch from 739f903 to 8d7816b Compare July 20, 2022 08:16
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 08:16 Inactive
@MichaReiser
Copy link
Contributor Author

!bench_formatter

@MichaReiser MichaReiser force-pushed the refactor/formatter-suppressions branch from 8d7816b to 90e6178 Compare July 20, 2022 08:53
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 08:53 Inactive
@github-actions
Copy link

Formatter Benchmark Results

group                                    main                                   pr
-----                                    ----                                   --
formatter/checker.ts                     1.00    454.4±4.12ms     5.7 MB/sec    1.02    461.5±4.13ms     5.6 MB/sec
formatter/compiler.js                    1.00    288.8±0.87ms     3.6 MB/sec    1.01    290.9±1.30ms     3.6 MB/sec
formatter/d3.min.js                      1.05    244.0±2.23ms  1100.1 KB/sec    1.00    233.5±1.95ms  1149.6 KB/sec
formatter/dojo.js                        1.00     15.8±0.11ms     4.3 MB/sec    1.00     15.8±0.05ms     4.3 MB/sec
formatter/ios.d.ts                       1.00    298.5±1.88ms     6.3 MB/sec    1.06    315.3±3.55ms     5.9 MB/sec
formatter/jquery.min.js                  1.01     62.5±0.52ms  1355.1 KB/sec    1.00     61.8±0.50ms  1369.1 KB/sec
formatter/math.js                        1.00    489.5±1.95ms  1354.5 KB/sec    1.02    498.0±3.00ms  1331.4 KB/sec
formatter/parser.ts                      1.00     10.3±0.02ms     4.7 MB/sec    1.03     10.5±0.02ms     4.6 MB/sec
formatter/pixi.min.js                    1.00    268.8±1.38ms  1671.7 KB/sec    1.00    268.0±1.43ms  1676.8 KB/sec
formatter/react-dom.production.min.js    1.00     76.7±0.37ms  1536.0 KB/sec    1.00     76.6±0.54ms  1538.8 KB/sec
formatter/react.production.min.js        1.00      3.8±0.01ms  1677.0 KB/sec    1.02      3.8±0.03ms  1645.3 KB/sec
formatter/router.ts                      1.00      7.6±0.02ms     7.8 MB/sec    1.01      7.7±0.01ms     7.7 MB/sec
formatter/tex-chtml-full.js              1.00    615.3±2.55ms  1516.6 KB/sec    1.01    621.6±1.82ms  1501.1 KB/sec
formatter/three.min.js                   1.02    315.9±4.15ms  1903.0 KB/sec    1.00    309.9±2.36ms  1940.1 KB/sec
formatter/typescript.js                  1.00  1905.7±11.08ms     5.0 MB/sec    1.02  1934.8±16.05ms     4.9 MB/sec
formatter/vue.global.prod.js             1.02    103.3±0.45ms  1194.8 KB/sec    1.00    101.4±1.30ms  1216.7 KB/sec

@MichaReiser MichaReiser force-pushed the refactor/formatter-suppressions branch from 90e6178 to 0ccd92f Compare July 20, 2022 09:07
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 09:08 Inactive
@MichaReiser MichaReiser force-pushed the refactor/formatter-suppressions branch from 0ccd92f to bf630fc Compare July 20, 2022 09:11
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 09:11 Inactive
@MichaReiser MichaReiser force-pushed the refactor/formatter-suppressions branch from bf630fc to f65c7a3 Compare July 20, 2022 09:17
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 09:17 Inactive
@MichaReiser MichaReiser marked this pull request as ready for review July 20, 2022 09:23
@MichaReiser MichaReiser requested a review from a team July 20, 2022 09:23
@MichaReiser MichaReiser requested a review from ematipico as a code owner July 20, 2022 09:23
This PR changes how the formatter extracts formatter suppression comments. The existing implementation used to traverse to a node's first token and then checking if it has any suppression comment when `context.is_suppressed()` was called.

This PR iterates over the CST before starting the actual formatting to extract all suppressed nodes and stores them in an index that can be queried during formatting.

The benefit of pre-computing the suppressions is that the logic can then be combined with the logic that extracts a nodes leading/trailing comments because that also requires iterating over all trivia pieces in the tree.

Adding this traversal on its own would regress performance by about 10% which we can offset if combining it with the suppression handling because the use of `first_token` of the existing implementation is expensive.

This change has the added benefit that suppressions are now a core formatter concept.

### Other changes
This PR further fixes a few places where the formatter didn't correctly check if a node has any suppression comment and if so, prevents it from changing the formatting.

### Why add suppressions to the `FormatContext`

I decided to introduce a new `FormatNodeContext` that replaces the existing `CommentsContext` and `CommentStyle` and now also stores the `suppressions`.

The main reason for storing the suppressions on the context rather than on the `state` is that the context, unlike the `FormatState`, is language specific, and thus, it's possible to have a `Language` type parameter on the `Suppressions` type.

The one thing I dislike about adding it to the context is that the context is changed AFTER its creation.

### Performance

The performance of the two approaches are roughly the same.

### Future improvement
It would be possible to improve Rome's overall performance if the workspace provides a way to combine the visitors run for the linter and the visitor for computing the suppression comments.
@MichaReiser MichaReiser force-pushed the refactor/formatter-suppressions branch from f65c7a3 to 90e72f7 Compare July 20, 2022 09:29
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 09:29 Inactive
@ematipico
Copy link
Contributor

This is mainly preparing for the migration to extract leading and trailing node comments with the re-work of the comments handling.

Can someone read somewhere what you plan to do? If this is a refactor per preparing something else, I think the reviewers should know what's coming next, so they have more context and everything is out in the open. We can add some comment in the original issue to explain what you have in mind.

@MichaReiser
Copy link
Contributor Author

This is mainly preparing for the migration to extract leading and trailing node comments with the re-work of the comments handling.

Can someone read somewhere what you plan to do? If this is a refactor per preparing something else, I think the reviewers should know what's coming next, so they have more context and everything is out in the open. We can add some comment in the original issue to explain what you have in mind.

The goal is to address #2768 by distinguishing between leading/node comments and dangling comments as described in this notion document.

Copy link
Contributor

@ematipico ematipico left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have mostly nitpicks

This PR further fixes a few places where the formatter didn't correctly check if a node has any suppression comment and if so, prevents it from changing the formatting.

Would it be possible to add new tests to make sure we demonstrate the bugs you found?

crates/rome_formatter/src/comments.rs Outdated Show resolved Hide resolved
crates/rome_formatter/src/comments.rs Outdated Show resolved Hide resolved
crates/rome_formatter/src/comments.rs Outdated Show resolved Hide resolved
crates/rome_formatter/src/comments.rs Outdated Show resolved Hide resolved
crates/rome_formatter/src/comments.rs Outdated Show resolved Hide resolved
crates/rome_formatter/src/comments.rs Outdated Show resolved Hide resolved
crates/rome_formatter/src/comments.rs Outdated Show resolved Hide resolved
crates/rome_formatter/src/lib.rs Outdated Show resolved Hide resolved
crates/rome_formatter/src/lib.rs Outdated Show resolved Hide resolved
crates/rome_formatter/src/comments.rs Outdated Show resolved Hide resolved
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 11:42 Inactive
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 12:10 Inactive
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 12:19 Inactive
@MichaReiser MichaReiser added the A-Formatter Area: formatter label Jul 20, 2022
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 14:44 Inactive
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 14:51 Inactive
@MichaReiser MichaReiser temporarily deployed to aws July 20, 2022 15:04 Inactive
@MichaReiser MichaReiser merged commit 3dc27f7 into main Jul 20, 2022
@MichaReiser MichaReiser deleted the refactor/formatter-suppressions branch July 20, 2022 15:16
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A-Formatter Area: formatter
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants