Hello

Pruning Hugging Face BERT: Using Compound Sparsification for Faster CPU Inference with Better Accuracy

Pruning Hugging Face BERT: Apply both pruning and layer dropping sparsification methods to increase BERT performance anywhere from 3.3x to 14x on CPUs depending on accuracy constraints In this post, we go into detail on pruning Hugging Face BERT and describe how sparsification combined with the DeepSparse Engine improves BERT model performance on CPUs. We’ll… Read More Pruning Hugging Face BERT: Using Compound Sparsification for Faster CPU Inference with Better Accuracy

YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance and a Smaller Footprint

Prune and quantize YOLOv5 for a 10x increase in performance with 12x smaller model files. Neural Magic improves YOLOv5 model performance on CPUs by using state-of-the-art pruning and quantization techniques combined with the DeepSparse Engine. In this blog post, we’ll cover our general methodology and demonstrate how to: Leverage the Ultralytics YOLOv5 repository with SparseML’s sparsification… Read More YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance and a Smaller Footprint

New Tutorial: Sparsifying YOLOv3 Using Recipes

Sparsifying YOLOv3 (or any other model) involves removing redundant information from neural networks using algorithms such as pruning and quantization, among others. This sparsification process results in many benefits for deployment environments, including faster inference and smaller file sizes. Unfortunately, many have not realized the benefits due to the complicated process and number of hyperparameters… Read More New Tutorial: Sparsifying YOLOv3 Using Recipes

Neural Magic Appoints Brian Stevens as Chief Executive Officer

We are excited to announce that industry veteran Brian Stevens will be joining Neural Magic as Chief Executive Officer. Brian brings vast experience in open source, enterprise, and hyper-scale cloud to the team. Before joining Neural Magic, Brian was Vice President and CTO of Google Cloud and Executive Vice President and CTO of Red Hat.… Read More Neural Magic Appoints Brian Stevens as Chief Executive Officer

YOLOv3 on CPUs: Sparsifying to Achieve GPU-Level Performance

Use CPUs to decrease costs and increase deployment flexibility while still achieving GPU-class performance. In this post, we elaborate on how we used state-of-the-art pruning and quantization techniques to improve the performance of the YOLOv3 on CPUs. We’ll show that by leveraging the robust YOLO training framework from Ultralytics with SparseML’s sparsification recipes it is… Read More YOLOv3 on CPUs: Sparsifying to Achieve GPU-Level Performance

Neural Magic 1.4 Product Release

We are excited to announce the Neural Magic 1.4 product release. This milestone contains new product features, an improved user experience, and stability enhancements that will simplify the ability for our clients to achieve GPU-class performance on commodity CPUs. NEW – Introducing Sparsify BETA Experience driven tooling to simplify the process of analyzing and optimizing… Read More Neural Magic 1.4 Product Release

Neural Magic 1.2 Product Release

We are excited to announce the Neural Magic 1.2 product release. This product milestone contains new feature updates, an improved user experience, and stability enhancements that will simplify the ability for our clients to achieve price performance on commodity CPUs.  Neural Magic Inference Engine Enables clients to run mission critical deep learning models on commodity… Read More Neural Magic 1.2 Product Release