Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement worker-side components of SyncSGD without backup worker #1130

Open
hjp615 opened this issue May 5, 2017 · 0 comments
Open

Implement worker-side components of SyncSGD without backup worker #1130

hjp615 opened this issue May 5, 2017 · 0 comments
Labels

Comments

@hjp615
Copy link
Contributor

hjp615 commented May 5, 2017

Until now, dolphin supports two models for parameter server : asynchronous model and SSP model.
They are faster than synchronous model because speed of synchronous model is limited by the longest training-time worker.

However, the paper Revisiting Distributed Synchronous SGD written by Google Brain team states that SyncSGD with backup worker can solve both convergence problem and speed efficiency problem. So, it would be nice if we implement SyncSGD with backup worker on dolphin and compare them each other.

Implementing worker-side components of SyncSGD without backup worker is a first step to implement SyncSGD with backup worker.

@hjp615 hjp615 added the SyncSGD label May 5, 2017
@hjp615 hjp615 changed the title Implement worker-side components for SyncSGD without backup worker Implement worker-side components of SyncSGD without backup worker May 5, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant