TinyML and Efficient Deep Learning Computing | MIT 6.S965 Fall 2022

Have you found it difficult to deploy neural networks on mobile devices and IoT devices? Have you ever found it too slow to train neural networks? This course is a deep dive into efficient machine learning techniques that enable powerful deep learning applications on resource-constrained devices. Topics cover efficient inference techniques, including model compression, pruning, quantization, neural architecture search, and distillation; and efficient training techniques, including gradient compression and on-device transfer learning; followed by application-specific model optimization techniques for videos, point cloud, and NLP; and efficient quantum machine learning. Students will get hands-on experience implementing deep learning applications on microcontrollers, mobile phones, and quantum machines with an open-ended design project related to mobile AI.

• 课时1：Basics of Neural Networks (1小时1分31秒)

• 课时2：Training Deep Nets with PyTorch (18分54秒)

• 课时3：Pruning and Sparsity (1小时7分6秒)

• 课时4：Pruning and Sparsity (1小时7分45秒)

• 课时5：Pruning and Sparsity (1小时7分39秒)

• 课时6：Quantization (1小时11分45秒)

• 课时7：Quantization (1小时11分42秒)

• 课时8：Quantization (1小时11分15秒)

• 课时9：Quantization (1小时10分53秒)

• 课时10：Neural Architecture Search (1小时4分0秒)

• 课时11：Neural Architecture Search (1小时12分54秒)

• 课时12：Neural Architecture Search (1小时12分50秒)

• 课时13：Neural Architecture Search (1小时6分57秒)

• 课时14：Knowledge Distillation (1小时7分25秒)

• 课时15：Knowledge Distillation (1小时7分21秒)

• 课时16：MCUNet- Tiny Neural Network Design for Microcontrollers (1小时6分41秒)

• 课时17：MCUNet- Tiny Neural Network Design for Microcontrollers (1小时6分38秒)

• 课时18：Distributed Training and Gradient Compression (1小时1分19秒)

• 课时19：Distributed Training and Gradient Compression (1小时1分5秒)

• 课时20：Distributed Training and Gradient Compression (57分32秒)

• 课时21：Distributed Training and Gradient Compression (57分28秒)

• 课时22：On-Device Training and Transfer Learning (1小时15分57秒)

• 课时23：On-Device Training and Transfer Learning (1小时15分55秒)

• 课时24：On-Device Training and Transfer Learning (1小时6分16秒)

• 课时25：On-Device Training and Transfer Learning (1小时6分29秒)

• 课时26：TinyEngine - Efficient Training and Inference on Microcontrollers (1小时15分6秒)

• 课时27：TinyEngine - Efficient Training and Inference on Microcontrollers (1小时15分1秒)

• 课时28：Efficient Point Cloud Recognition (1小时14分53秒)

• 课时29：Efficient Point Cloud Recognition (1小时14分14秒)

• 课时30：Efficient Video Understanding and Generative Models (1小时37分46秒)

• 课时31：Efficient Video Understanding and Generative Models (1小时37分40秒)

• 课时32：Efficient Transformers (1小时18分8秒)

• 课时33：Efficient Transformers (1小时18分4秒)

• 课时34：Basics of Quantum Computing (38分38秒)

• 课时35：Basics of Quantum Computing (38分38秒)

• 课时36：Quantum Machine Learning (1小时9分1秒)

• 课时37：Noise-Robust Quantum Machine Learning (1小时8分57秒)

• 课时38：Course Summary (12分47秒)

• 课时39：Course Summary (12分49秒)

• 课时40：AI Model EfficiencyToolkit (45分15秒)

• 课时41：AI Model EfficiencyToolkit (45分12秒)