NeuralFlix

How to Compress Your BERT NLP Models For Very Efficient Inference

Presenter: Mark Kurtz

This video covers SOTA compression research that addresses common Transformer setbacks, including their large size and difficulty with efficient deployments. Specifically, we dig into how you can compress BERT models 10x for much more efficient inference, gaining a 9-29x CPU inference speedup along the way.

More ML Research in Action Videos

Sparse Training of Neural Networks Using AC/DC
How Well Do Sparse Models Transfer?
How to Achieve the Fastest CPU Inference Performance for Object Detection YOLO Models
Workshop: How to Optimize Deep Learning Models for Production
How to Compress Your BERT NLP Models For Very Efficient Inference
Sparsifying YOLOv5 for 10x Better Performance, 12x Smaller File Size, and Cheaper Deployment
Tissue vs. Silicon: The Future of Deep Learning Hardware
Pruning Deep Learning Models for Success in Production

Get more info about

How to Compress Your BERT NLP Models For Very Efficient Inference