Course Outline

Introduction

  • Overview of deep learning scaling challenges
  • Overview of DeepSpeed and its features
  • DeepSpeed vs. other distributed deep learning libraries

Getting Started

  • Setting up the development environment
  • Installing PyTorch and DeepSpeed
  • Configuring DeepSpeed for distributed training

DeepSpeed Optimization Features

  • DeepSpeed training pipeline
  • ZeRO (memory optimization)
  • Activation checkpointing
  • Gradient checkpointing
  • Pipeline parallelism

Scaling Models with DeepSpeed

  • Basic scaling using DeepSpeed
  • Advanced scaling techniques
  • Performance considerations and best practices
  • Debugging and troubleshooting techniques

Advanced DeepSpeed Topics

  • Advanced optimization techniques
  • Using DeepSpeed with mixed precision training
  • DeepSpeed on different hardware (e.g. GPUs, TPUs)
  • DeepSpeed with multiple training nodes

Integrating DeepSpeed with PyTorch

  • Integrating DeepSpeed with PyTorch workflows
  • Using DeepSpeed with PyTorch Lightning

Troubleshooting

  • Debugging common DeepSpeed issues
  • Monitoring and logging

Summary and Next Steps

  • Recap of key concepts and features
  • Best practices for using DeepSpeed in production
  • Further resources for learning more about DeepSpeed

Requirements

  • Intermediate knowledge of deep learning principles
  • Experience with PyTorch or similar deep learning frameworks
  • Familiarity with Python programming

Audience

  • Data scientists
  • Machine learning engineers
  • Developers
 21 Hours

Delivery Options

Private Group Training

Our identity is rooted in delivering exactly what our clients need.

  • Pre-course call with your trainer
  • Customisation of the learning experience to achieve your goals -
    • Bespoke outlines
    • Practical hands-on exercises containing data / scenarios recognisable to the learners
  • Training scheduled on a date of your choice
  • Delivered online, onsite/classroom or hybrid by experts sharing real world experience

Private Group Prices RRP from £5700 online delivery, based on a group of 2 delegates, £1800 per additional delegate (excludes any certification / exam costs). We recommend a maximum group size of 12 for most learning events.

Contact us for an exact quote and to hear our latest promotions


Public Training

Please see our public courses

Provisonal Upcoming Courses (Contact Us For More Information)

Related Categories