DeepSpeed
Deep learning optimization library from Microsoft for training and running very large models efficiently across multiple GPUs.
- Actively maintained
- Well documented
- Permissive license
Overview
DeepSpeed implements memory- and communication-efficient techniques - like ZeRO optimizer state partitioning - that let teams train models far larger than would otherwise fit in available GPU memory, and speeds up training across multiple GPUs and machines. It integrates with PyTorch (also in this catalog) rather than replacing it.
DeepSpeed is a fit for teams training large models that push the limits of available hardware, a specialized optimization layer on top of PyTorch rather than a general-purpose framework - most teams reach for it specifically when standard single-GPU or naive multi-GPU training isn't enough.
Why it's in this catalog
Catalog rating: ★★★☆☆ (3/5)
Widely used large-scale model training optimization library.
- Mature
- Vendor-backed
- Categories
- AI & Machine Learning
- Languages
- Python
- License
- Apache-2.0
Related Projects
PyTorch
BSD-3-ClauseOpen source machine learning framework that accelerates the path from research prototyping to production deployment.
Actively maintained
- AI & Machine Learning
TensorFlow
Apache-2.0End-to-end open source platform for machine learning, from research to production.
Actively maintained
- AI & Machine Learning
Unsloth
Apache-2.0Library for fine-tuning large language models significantly faster and with less memory than standard training approaches.
Actively maintained
- AI & Machine Learning
Milvus
Apache-2.0Open source vector database built for scalable similarity search and AI applications.
Actively maintained
- AI & Machine Learning
- Database
Spotted an error? Suggest an edit on GitHub.