Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize performance of transfer #3091

Merged
merged 5 commits into from
May 31, 2022
Merged

Optimize performance of transfer #3091

merged 5 commits into from
May 31, 2022

Conversation

qinxuye
Copy link
Collaborator

@qinxuye qinxuye commented May 27, 2022

What do these changes do?

This PR tries to optimize performance of transfer, including:

  1. No more copy for memoryview when serializing
  2. Optimize actor call for transfer.

Performance on my laptop.

Master: {('127.0.0.1:64520', '127.0.0.1:64514'): ('137.24M', '284.03MB/s'), ('127.0.0.1:64514', '127.0.0.1:64520'): ('137.24M', '296.84MB/s')}
This branch: {('127.0.0.1:64601', '127.0.0.1:64595'): ('137.24M', '490.63MB/s'), ('127.0.0.1:64595', '127.0.0.1:64601'): ('137.24M', '513.19MB/s')}

Related issue number

Fixes #3090 .

Check code requirements

  • tests added / passed (if needed)
  • Ensure all linting tests pass, see here for how to run them

@qinxuye qinxuye added type: enhancement request to be backported Indicate that the PR need to be backported to stable branch mod: storage labels May 27, 2022
@qinxuye qinxuye added this to the v0.10.0a1 milestone May 27, 2022
@qinxuye qinxuye requested a review from a team as a code owner May 27, 2022 18:31
@qinxuye
Copy link
Collaborator Author

qinxuye commented May 28, 2022

The master cannot run storage asv_bench directly, so the ci will fail.

wjsi
wjsi approved these changes May 31, 2022
View reviewed changes
Copy link
Member

@wjsi wjsi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

hekaisheng
hekaisheng approved these changes May 31, 2022
View reviewed changes
Copy link
Contributor

@hekaisheng hekaisheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGMT

@hekaisheng hekaisheng merged commit 994aec1 into mars-project:master May 31, 2022
@qinxuye qinxuye deleted the enh/transfer branch June 1, 2022 04:46
qinxuye pushed a commit to qinxuye/mars that referenced this pull request Jun 1, 2022
Optimize performance of transfer (mars-project#3091)
fe297d1 
(cherry picked from commit 994aec1)
@qinxuye qinxuye mentioned this pull request Jun 1, 2022
Merged
wjsi pushed a commit that referenced this pull request Jun 1, 2022
[BACKPORT] Optimize performance of transfer (#3091) (#3105)
3eae650
@wjsi wjsi added backported already PR has been backported and removed to be backported Indicate that the PR need to be backported to stable branch labels Jun 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wjsi wjsi wjsi approved these changes

@hekaisheng hekaisheng hekaisheng approved these changes

Assignees
No one assigned
Labels
backported already PR has been backported mod: storage type: enhancement request
Projects
None yet
Milestone
v0.10.0a1
Development

Successfully merging this pull request may close these issues.

[BUG] Serialization pickled memoryview instead of treating it as buffer
3 participants
@qinxuye @wjsi @hekaisheng