-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor partial parsing to cover more startup costs #3163
Comments
This was referenced Mar 16, 2021
We've narrowed the scope of this issue:
Will open another issue to cover:
|
Have opened #3217 for moving the save point. |
gshank
added a commit
that referenced
this issue
Apr 1, 2021
gshank
added a commit
that referenced
this issue
Apr 1, 2021
gshank
added a commit
that referenced
this issue
Apr 1, 2021
4 tasks
gshank
added a commit
that referenced
this issue
Apr 1, 2021
gshank
added a commit
that referenced
this issue
Apr 1, 2021
gshank
added a commit
that referenced
this issue
Apr 2, 2021
gshank
added a commit
that referenced
this issue
Apr 6, 2021
gshank
added a commit
that referenced
this issue
Apr 6, 2021
gshank
added a commit
that referenced
this issue
Apr 6, 2021
gshank
added a commit
that referenced
this issue
Apr 6, 2021
gshank
added a commit
that referenced
this issue
Apr 6, 2021
Use Manifest instead of ParseResult [#3163]
iknox-fa
pushed a commit
that referenced
this issue
Feb 8, 2022
automatic commit by git-black, original commits: 307d47e
iknox-fa
pushed a commit
that referenced
this issue
Feb 8, 2022
iknox-fa
pushed a commit
that referenced
this issue
Feb 8, 2022
iknox-fa
pushed a commit
that referenced
this issue
Feb 8, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The current implementation of partial parsing saves a pickle file during parsing, before the 'create_manifest' step in which the following occurs 1) patch_sources (where source tests are processed), 2) patch_nodes, 3) patch macros, 4) process_sources, 5) process_refs, and 6) process_docs. If we refactor parsing so that the parsing state is saved after these things happen, then startup costs when using partial parsing will be substantially reduced. In our large parsing test project, those steps take 70 seconds out of a total of 193 in a sample run. We wouldn't be able to eliminate all of that time, but eliminating most of it would result in almost a 1/3 reduction in time.
Because the current implementation does not have to handle re-processing sources, refs, and docs, we would need to create new code to handle doing those tasks for only the changed files.
At some point we may also look at whether a pickle file is the best alternative for saving and restoring state, but that is not currently part of this project.
The text was updated successfully, but these errors were encountered: