Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic trait for SecondaryMaps #51

Merged
merged 7 commits into from
May 30, 2023
Merged

Conversation

aborgna-q
Copy link
Collaborator

@aborgna-q aborgna-q commented May 25, 2023

This is a big breaking change that will give us more flexibility when managing secondary data on a portgraph.

It adds a SecondaryMap trait that encapsulates the common interfaces we use when managing node and port data.
We can then implement that trait on different structures like the old SecondaryMap (now renamed UnmanagedMap), bit vec for boolean storage, and any {Hash,BTree}{Set,Map}. This closes #46.

With this, we can add a Map generic parameter to DominatorTree and TopoSort that lets us use more efficient structures when we do not traverse the full graph. This closes #31.

Also:

  • Renames the old SecondaryMap struct as "UnmanagedMap".
  • Implements SecondaryMap for BitVec.
  • Uses the trait in Dominators and TopoSort to avoid wasting memory when doing partial traversals.

There's a TODO to implement SecondaryMap for sparse containers (hash and BTree), but those can be done in another non-breaking PR.

Note that unmanaged.rs is mostly the old contents of secondary.rs, but git didn't pick up the renaming.

@aborgna-q aborgna-q requested a review from lmondada May 25, 2023 13:03
@aborgna-q
Copy link
Collaborator Author

@zrho You may have comments on this.

@aborgna-q aborgna-q mentioned this pull request May 25, 2023
aborgna-q added 2 commits May 25, 2023 14:56
- Renames the old SecondaryMap struct as "UnmanagedMap"
- Implements SecondaryMap for BitVec
- Uses the trait in Dominators to avoid wasting memory when doing
  partial traversals.
@aborgna-q aborgna-q force-pushed the feature/secondarymap-trait branch from f06febf to a2fbad8 Compare May 25, 2023 13:56
@aborgna-q aborgna-q requested review from ss2165 and removed request for lmondada May 30, 2023 09:15
@aborgna-q aborgna-q added the breaking change This breaks the SemVer compatibility label May 30, 2023
Copy link
Member

@ss2165 ss2165 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe there's a bug in BitVec implementation but otherwise looks mostly ok

@@ -127,13 +140,13 @@ impl<'graph> TopoSort<'graph> {
// Mark all the candidate ports as visited, so we don't visit them again.
for node in candidate_nodes.iter() {
for port in graph.ports(*node, direction.reverse()) {
remaining_ports.set(port.index(), false);
visited_ports.set(port, true);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there an assumption here that the default value of Map is always false or is that guranteed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is part of the definition of SecondaryMap:

/// A map from keys to values that does not manage it's indices.
///
/// Querying a key that has not been set returns a default value.

(and default_value returns false for BitVec)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right but that doesn't stop me using a struct for Map that returns true for default_value right? Might need a debug assert at least?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the check to toposort. If the default value is true it initializes everything (and states so in the docs).

@zrho
Copy link
Contributor

zrho commented May 30, 2023

Looks good to me.

  1. The concept of capacity might be a bit confusing when the key is generic, especially when reducing capacity cuts things off; perhaps it could be explained that it is intended to be used with keys that are convertible with numbers?
  2. UnmanagedMap could perhaps be UnmanagedDenseMap to emphasize that it is a dense storage?

@aborgna-q
Copy link
Collaborator Author

  1. The concept of capacity might be a bit confusing when the key is generic, especially when reducing capacity cuts things off; perhaps it could be explained that it is intended to be used with keys that are convertible with numbers?

Yes, perhaps shrink_to should be left for the concrete impls instead of being part of the trait.

  1. UnmanagedMap could perhaps be UnmanagedDenseMap to emphasize that it is a dense storage?

👍

Copy link
Member

@ss2165 ss2165 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@aborgna-q aborgna-q merged commit 396ebd1 into main May 30, 2023
@aborgna-q aborgna-q deleted the feature/secondarymap-trait branch May 30, 2023 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change This breaks the SemVer compatibility
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support bit vectors as secondary maps. Add efficient toposort/dominators structures for partial traversals.
3 participants