Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scalability part I: Multiports #759

Merged
merged 122 commits into from
Nov 23, 2021
Merged

Scalability part I: Multiports #759

merged 122 commits into from
Nov 23, 2021

Conversation

edwardalee
Copy link
Collaborator

This branch does a major refactoring of the instance classes, ReactorInstance, ReactionInstance, PortInstance, etc., converting them all to Java, eliminating the MultiportInstance class, and consolidating so that a multiport results in exactly one PortInstance being created rather than one PortInstance per channel. This is a large PR with a lot of changes, but it is still a work in progress. Quite a bit more work needs to be done, most notably to handle banks more intelligently and create only one ReactorInstance per bank rather than one ReactorInstance per bank member. That additional work will occur in a new branch derived from this one.

edwardalee and others added 30 commits October 28, 2021 17:29
Soroosh129 and others added 15 commits November 22, 2021 14:34
This is based on a suggestion from Christian, since the Python runner script does not parse the output of the modified benchmark correctly.

The original benchmark has a couple of minor corrections that also appear in the one that uses the benchmark runner.
…guaFrancaShapeExtensions.xtend

Co-authored-by: Marten Lohstroh <[email protected]>
…guaFrancaShapeExtensions.xtend

Co-authored-by: Marten Lohstroh <[email protected]>
Copy link
Collaborator

@cmnrd cmnrd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! This is quite difficult to review carefully due to the many changes made, but I didn't notice any red flags.

* @author{Marten Lohstroh <[email protected]>}
* @author{Edward A. Lee <[email protected]>}
*/
public class ReactorInstance extends NamedInstance<Instantiation> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class is huge. A good rule of thumb seems to be to keep classes below 500 lines. Is there maybe a way to split this class up. Also several methods in this class are too long (The method on line 911 for instance). Generally, method bodies should fit on a single screen.

@cmnrd
Copy link
Collaborator

cmnrd commented Nov 23, 2021

BTW: There is something fishy going with the history in this branch. The commits between
2cbe84e and 631e261 look like they are duplicated from master. It looks as if they were rebased. Was this on purpose?

As for the effectiveness of the changes made, I did a quick performance test compiling the bank transaction benchmark with 1000 accounts. It looks like this branch improves execution time, but increases the memory footprint even further.
time -v reports 123.68s user time on master and 10.2GB memory usage. On scalability, it reports 62.45s user time and 12.2GB memory usage. I am not sure if the changes made in this branch are supposed to influence the size of generated C code, but it is 1.2GB large on both branches.

@edwardalee
Copy link
Collaborator Author

As for the fishy history, I have no idea... All I did was merge in master. Was this the wrong thing to do?

@edwardalee
Copy link
Collaborator Author

The interleaved connection in Banking foils the methods used in this PR, I think. It will require a bit more work to handle those efficiently.

@cmnrd
Copy link
Collaborator

cmnrd commented Nov 23, 2021

As for the fishy history, I have no idea... All I did was merge in master. Was this the wrong thing to do?

Merging master shouldn't be a problem. But it looks like what actually happened is not a merge, but a rebase (there is also no merge commit). I observed this happening a few times now (we also had this in one of Marten's branches), but I don't know what causes this. Consider for instance the commit ce52695 on master. It is the same as commit 631e261 on this branch, but they got different hashes. The two commits are duplicates. I don't know what is causing this though... How did you perform the merge? Directly with git merge master, or did you use some other tool?

@edwardalee
Copy link
Collaborator Author

I used git merge master, then resolved conflicts and committed them, then did either git rebase --continue or git rebase --abort (the latter apparently needed if the conflicts are resolved in favor of what's in master).

@cmnrd
Copy link
Collaborator

cmnrd commented Nov 23, 2021

Oh, that explains it. When you are merging, you should use git merge --continue instead. I don't know what precisely happens if you start a merge and then continue with a rebase, but apparently it aborts the merge and rebases all the changes instead (i.e. duplicating the master commits and putting them on top of the branch).

@edwardalee
Copy link
Collaborator Author

Ugh, ok. Does this need to be undone? If so, how?

@cmnrd
Copy link
Collaborator

cmnrd commented Nov 23, 2021

No, I don't think that we need to fix it. Nothing actually broke, we will only have a few duplicated commits when we merge this into master. For fixing it we would need to rewrite the history of this branch, but his would come at the high risk of breaking something else.

@lhstrh
Copy link
Member

lhstrh commented Nov 23, 2021

I want to be extra clear here to avoid further misunderstandings. When you do a git pull --rebase (--rebase is implicit if you have pull.rebase set to true in your configuration) and conflicts arise, then after resolving conflicts, you do a git rebase --continue. If you do a git merge, then after resolving conflicts you do a git merge --continue. Git tells also you this, but the merge vs. rebase difference in the suggested commands is easily missed.

@edwardalee: I'm glad we were able to pinpoint this, because it likely also explains some other issues that you have been experiencing with Git that really puzzled me.

@edwardalee edwardalee merged commit 419b25e into master Nov 23, 2021
@cmnrd cmnrd deleted the scalability branch January 18, 2022 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants