Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve startup and opening times #15759

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

dimven-adsk
Copy link
Contributor

@dimven-adsk dimven-adsk commented Jan 16, 2025

Purpose

Imagine you have 10 very large graphs in your recent files list. Every time the start page opens, it reads the entire contents of those files (plus any backup files), and then parses the content of each file twice! Then every time you open a new file, the whole process is repeated multiple times 😿

I propose we:

  1. Stream the files instead of reading their entire content.
  2. Skip the json validation (which forces us to pre-parse the file). It's not important why the file fails - Dynamo shouldn't be a json parser, just a json consumer. If the file is not a json or if the json content is malformed, you should use the right tool to figure that out.
  3. Read and store only the properties that are actually needed.
  4. Inline the description, author, etc. methods because they're not really needed.
  5. Implement a "quiet" observable collection for the recent files list. When you open a file, it is (1) removed from the list, (2), added to the front of the list, (3) if the max number of recents is exceeded, the list is truncated. These multiple actions force the start page to read the recents' properties multiple number of times.

The way the RecentFiles is used right now doesn't really necessitate an ObservableCollection and a list property would work just the same. However, in case that changes in the future, a quiet observable collection will preserve the flexibility.

image

Declarations

Check these if you believe they are true

  • The codebase is in a better state after this PR
  • Is documented according to the standards
  • The level of testing this PR includes is appropriate
  • User facing strings, if any, are extracted into *.resx files
  • All tests pass using the self-service CI.
  • Snapshot of UI changes, if any.
  • Changes to the API follow Semantic Versioning and are documented in the API Changes document.
  • This PR modifies some build requirements and the readme is updated
  • This PR contains no files larger than 50 MB

Release Notes

(FILL ME IN) Brief description of the fix / enhancement. Use N/A to indicate that the changes in this pull request do not apply to Release Notes. Mandatory section

Reviewers

(FILL ME IN) @mjkkirschner

(FILL ME IN, optional) Any additional notes to reviewers or testers.

FYIs

(FILL ME IN, Optional) Names of anyone else you wish to be notified of

@dimven-adsk dimven-adsk changed the title Improve Improve startup and opening times Jan 16, 2025
@mjkkirschner mjkkirschner self-assigned this Jan 17, 2025
@mjkkirschner mjkkirschner self-requested a review January 17, 2025 15:59

namespace Dynamo.Wpf.Utilities
{
public class QuietObservableCollection<T> : ObservableCollection<T>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have a similar implementation.
Please use or enhance the existing one it if necessary
https://github.com/DynamoDS/Dynamo/blob/master/src/DynamoUtilities/SmartObservableCollection.cs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, makes sense. Sorry, I did not find that one :D

@@ -465,51 +465,50 @@ private void RefreshFileList(ObservableCollection<StartPageListItem> files,
}
}

private Dictionary<string, object> DeserializeJsonFile(string filePath)
private static string[] jsonKeys = { "Description", "Thumbnail", "Author" };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

readonly ?

else
{
DynamoViewModel.Model.Logger.Log("File is not valid: " + ex.StackTrace);
while(jr.Read())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is this low level parsing faster then just doing a Json.deserialize to a new class containing only the 3 properties that we need ? Doing the latter would at least be cleaner and less error prone

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The advantage of doing it this way, is that we don't have to read the entire content of the dyn file upfront. I'll do some more testing and see if deseralizing into a new mini class will work better.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The properties { "Description", "Thumbnail", "Author" }; are not always in a fixed position. I've seen those scattered through the dyn file. Maybe @mjkkirschner can confirm.
Reading it token by token (or line by line) might be faster in some cases, but it does look messier.
I could go either way

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that it is messier. However you have to remember that this happens every time you open the home page and every time you open a new graph. Every ~100ms we can shave off from the overall process means a noticeably better user experience overall.

I did some quick tests and got some rough numbers:

file size (kb) JsonConvert.Deserialize (ms) JsonReader (ms)
800 28 40
1300 45 70
4300 170 450

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order of the json properties is not enforced or guaranteed. I've not yet looked deeply at this PR - is the logic dependent on the tokens being in a specific order?

We can enforce them using the order attribute but that won't help us with existing graphs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not dependent on the order. If those attributes were always first, it would make it faster I guess

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

example:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants