You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Until now, dolphin supports two models for parameter server : asynchronous model and SSP model.
They are faster than synchronous model because speed of synchronous model is limited by the longest training-time worker.
However, the paper Revisiting Distributed Synchronous SGD written by Google Brain team states that SyncSGD with backup worker can solve both convergence problem and speed efficiency problem. So, it would be nice if we implement SyncSGD with backup worker on dolphin and compare them each other.
Implementing worker-side components of SyncSGD without backup worker is a first step to implement SyncSGD with backup worker.
The text was updated successfully, but these errors were encountered:
hjp615
changed the title
Implement worker-side components for SyncSGD without backup worker
Implement worker-side components of SyncSGD without backup worker
May 5, 2017
Until now, dolphin supports two models for parameter server : asynchronous model and SSP model.
They are faster than synchronous model because speed of synchronous model is limited by the longest training-time worker.
However, the paper Revisiting Distributed Synchronous SGD written by Google Brain team states that SyncSGD with backup worker can solve both convergence problem and speed efficiency problem. So, it would be nice if we implement SyncSGD with backup worker on dolphin and compare them each other.
Implementing worker-side components of SyncSGD without backup worker is a first step to implement SyncSGD with backup worker.
The text was updated successfully, but these errors were encountered: