Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: garbage collect Stream when dropped #167

Merged
merged 10 commits into from
Jul 11, 2023

Conversation

aschran
Copy link
Contributor

@aschran aschran commented Jul 6, 2023

This fixes an issue where a Stream would never be removed until its connection is closed.

Resolves #166.

This fixes issue libp2p#166 where a `Stream` would never be removed
until its connection is closed.
Copy link
Member

@mxinden mxinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very well done @aschran. Thank you for reporting, debugging and fixing. I appreciate your work here.

I missed this in my review in #156 (comment).

Overall this looks good to me. Missing steps:

  • I would like to give @thomaseizinger a chance to review as well.
  • @aschran would you mind adding a changelog entry to CHANGELOG.md?

Again, well done. Thank you.

yamux/src/connection.rs Outdated Show resolved Hide resolved
yamux/src/connection.rs Outdated Show resolved Hide resolved
yamux/src/connection.rs Outdated Show resolved Hide resolved
@mxinden
Copy link
Member

mxinden commented Jul 7, 2023

This likely warrants a patch release (i.e. v0.11.1). I can get that done once we merged here.

@mxinden
Copy link
Member

mxinden commented Jul 7, 2023

@aschran @thomaseizinger I merged latest master and added a test that fails on master without this patch.

@aschran
Copy link
Contributor Author

aschran commented Jul 7, 2023

Committed your suggestions. Thanks for the review and confirmation + test! Will add the CHANGELOG entry shortly when I can get back to my computer.

Copy link
Contributor

@thomaseizinger thomaseizinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Ironically, the fact that all of Stream was shared bugged me already in the past because it is conceptually hard to understand. The fact that it now actually caused a bug annoys me that I didn't follow my gut feeling there :)

I made one suggestion on how I think we could make this a bit cleaner. Let me know if you are up for it, otherwise happy to merge this as is once CI passes!

CHANGELOG.md Outdated Show resolved Hide resolved
yamux/src/connection.rs Outdated Show resolved Hide resolved
yamux/src/connection.rs Outdated Show resolved Hide resolved
@@ -350,7 +353,7 @@ struct Active<T> {
socket: Fuse<frame::Io<T>>,
next_id: u32,

streams: IntMap<StreamId, Stream>,
streams: IntMap<StreamId, Arc<Mutex<stream::Shared>>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it would turn out cleaner if Shared would hold Arc<Mutex> internally? That would remove the boilerplate around locking and some noise from the type signatures.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would prefer to merge the fix and leave this for future work if that's ok with you, since it looks like a bit more involved of a change

aschran and others added 2 commits July 9, 2023 10:04
@thomaseizinger
Copy link
Contributor

Can you look at the CI failures? Otherwise LGTM!

@aschran
Copy link
Contributor Author

aschran commented Jul 10, 2023

Can you look at the CI failures? Otherwise LGTM!

I'll admit I'm not sure what's up with those. If I run cargo build on this branch locally it works fine for me. The errors in CI suggest it's not running on the same code I'm looking at? yamux/src/connection.rs is only 927 lines long and yet errors are showing lines 942-943? It doesn't seem I have access to trigger a re-run of CI myself.

@mxinden
Copy link
Member

mxinden commented Jul 10, 2023

Our CI first merges the target branch, here master, and then executes the tests.

We recently merged #153. While git does not identify it as a merge conflict, it does conflict with this pull request.

I pushed a merge resolving the non-git-merge-conflict conflicts. @thomaseizinger this moves the is_outbound method inline and duplicates the is_pending_ack on Shared.

@aschran
Copy link
Contributor Author

aschran commented Jul 10, 2023

Thanks for helping explain & fix Max :)

Copy link
Member

@mxinden mxinden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging here.

@thomaseizinger let me know in case you would like to see any follow-ups given the latest changes.

@aschran once more, thank you for the work. Let me know in case you would like me to backport this change to v0.11.

@mxinden mxinden changed the title Only save stream Shared state in Active<T> fix: garbage collect Stream when dropped Jul 11, 2023
@mxinden mxinden merged commit dcff3d5 into libp2p:master Jul 11, 2023
@aschran
Copy link
Contributor Author

aschran commented Jul 11, 2023

@mxinden yes, if you have time to backport that would be great! Appreciate the diligent review from you both.

@thomaseizinger
Copy link
Contributor

@thomaseizinger let me know in case you would like to see any follow-ups given the latest changes.

None that are critical for a release.

@mxinden mxinden mentioned this pull request Jul 20, 2023
@mxinden
Copy link
Member

mxinden commented Jul 20, 2023

@aschran released happened through #169. Let me know if that is good enough for you.

@aschran
Copy link
Contributor Author

aschran commented Jul 20, 2023

That works, thanks for letting me know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dropped Streams are never removed until connection closes?
3 participants