Accelerate documentation
Example Zoo
You are viewing v0.13.2 version. A newer version v1.13.0 is available.
Example Zoo
Below contains a non-exhuastive list of tutorials and scripts showcasing Accelerate
Official Accelerate Examples:
Basic Examples
These examples showcase the base features of Accelerate and are a great starting point
Feature Specific Examples
These examples showcase specific features that the Accelerate framework offers
- Automatic memory-aware gradient accumulation
- Checkpointing states
- Cross validation
- DeepSpeed
- Fully Sharded Data Parallelism
- Gradient accumulation
- Memory-aware batch size finder
- Metric Computation
- Using Trackers
Full Examples
These examples showcase every feature in Accelerate at once that was shown in “Feature Specific Examples”
- Complete NLP example
- Complete computer vision example
- Causal language model fine-tuning example
- Masked language model fine-tuning example
- Speech pretraining example
- Translation fine-tuning example
- Text classification fine-tuning example
- Semantic segmentation fine-tuning example
- Question answering fine-tuning example
- Beam search question answering fine-tuning example
- Multiple choice question answering fine-tuning example
- Named entity recognition fine-tuning example
- Image classification fine-tuning example
- Summarization fine-tuning example
Integration Examples
These are tutorials from libraries that integrate with 🤗 Accelerate:
Catalyst
DALLE2-pytorch
🤗 diffusers
fastai
- Distributed training from Jupyter Notebooks with fastai
- Basic distributed training examples with fastai