Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce number of Dataflow updates through dependency tree code #5508

Merged
merged 2 commits into from
Nov 20, 2023

Conversation

drewnoakes
Copy link
Contributor

The provision of transitive dependency tree nodes uses Dataflow to propagate project data from producer to consumer.

This change converts a TransformBlock into a TransformManyBlock, such that we can return zero items when there's no real change to propagate. This reduces the work done by consumers, who were previously wasting resources in response to such no-op updates.

Bug

Fixes: NuGet/Home#13019

Regression? Last working version: N/A

Description

PR Checklist

  • PR has a meaningful title

  • PR has a linked issue.

  • Described changes

  • Tests

    • Automated tests added
    • OR
    • Test exception
    • OR
    • N/A Dataflow is notoriously difficult to test and we hardly ever do so in the project system
  • Documentation

    • Documentation PR or issue filled
    • OR
    • N/A

The provision of transitive dependency tree nodes uses Dataflow to propagate project data from producer to consumer.

This change converts a TransformBlock into a TransformManyBlock, such that we can return zero items when there's no real change to propagate. This reduces the work done by consumers, who were previously wasting resources in response to such no-op updates.
nkolev92
nkolev92 previously approved these changes Nov 17, 2023
@nkolev92 nkolev92 added Approved for CI Community PRs created by someone not in the NuGet team and removed Approved for CI Community PRs created by someone not in the NuGet team labels Nov 17, 2023
@nkolev92
Copy link
Member

I forgot you have permissions :D

…plorer/AssetsFileDependenciesDataSource.cs

Co-authored-by: Donnie Goodson <[email protected]>
@drewnoakes drewnoakes dismissed stale reviews from donnie-msft and nkolev92 via 3dc5b39 November 17, 2023 21:47
@drewnoakes
Copy link
Contributor Author

/azp run

Copy link

No pipelines are associated with this pull request.

@drewnoakes drewnoakes merged commit d85c6ea into dev Nov 20, 2023
1 check passed
@drewnoakes drewnoakes deleted the dev-drnoakes-reduce-dataflow-activity branch November 20, 2023 01:06
Nigusu-Allehu pushed a commit that referenced this pull request Nov 21, 2023
The provision of transitive dependency tree nodes uses Dataflow to propagate project data from producer to consumer.

This change converts a `TransformBlock` into a `TransformManyBlock`, such that we can return zero items when there's no real change to propagate. This reduces the work done by consumers, who were previously wasting resources in response to such no-op updates.
drewnoakes added a commit that referenced this pull request Jul 12, 2024
#5508)"

This reverts commit d85c6ea.

Turns out that the search provider uses `GetLatestVersionAsync` on this
data source, meaning it must propagate a value for every input version.
I will fix the original issue in a different way, by updating the
consumer to ignore version-only changes instead.
drewnoakes added a commit that referenced this pull request Jul 12, 2024
This replaces the approach taken in #5508 in a way that doesn't interfere with Solution Explorer search.
drewnoakes added a commit that referenced this pull request Jul 13, 2024
… of dependencies tree (#5917)

* Revert "Reduce number of Dataflow updates through dependency tree code (#5508)"

This reverts commit d85c6ea.

Turns out that the search provider uses `GetLatestVersionAsync` on this
data source, meaning it must propagate a value for every input version.
I will fix the original issue in a different way, by updating the
consumer to ignore version-only changes instead.

* Avoid tree updates when the value is unchanged

This replaces the approach taken in #5508 in a way that doesn't interfere with Solution Explorer search.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Perf issue in AssetsFileDependenciesDataSource
3 participants