HybridBackend v0.6.0
Objectives:
- Communication-efficient training and evaluation at scale
- Easy to use with existing AI workflows
Features:
-
Data-Parallel Training and Evaluation
- Bucketized Gradients Aggregation using AllReduce
- Global Metric Operations
- Out-Of-Range Coordination -
Hybrid-Parallel Embedding Learning
- Bucketized Embedding Exchanging using AllToAllv
- Fusion and Quantization of AllToAllv
- Fusion of Partitioning and Stitching -
Usability
- Support of MonitoredSession and Estimator
- Declarative API for Model Definition -
Compatibility
- Support of NVIDIA TensorFlow and DeepRec -
Interoperability
- Inference Pipeline Needs No Change
- Support of SavedModel
- Support of Variable, XDL HashTable and PAI Embedding Variable
Bug Fixes:
[#46] Fixes rebatching in ParquetDataset.