Neural Magic Joins MLCommons to Help Accelerate ML Innovation Through Transparency and Open Source

Neural Magic Joins MLCommons  Through research, benchmarks, and best practices, Neural Magic is committed to open standards that will guide machine learning (ML) along the path from a research field to a mature industry. In February of 2021, we open-sourced our model sparsification libraries and made our sparsity-aware inference engine freely available for community use.… Read More Neural Magic Joins MLCommons to Help Accelerate ML Innovation Through Transparency and Open Source

Video: Azure, AMD, and Neural Magic Raise the Bar for High-Performance Computing

Microsoft, AMD, and Neural Magic are raising the bar for high-performance computing. With a combination of HBv3 virtual machines and our sparsity-aware inference engine, we are able to run deep learning workloads on CPUs at speeds previously reserved only for GPUs. For example, together we deliver 5x inference speedup for BERT NLP models over other… Read More Video: Azure, AMD, and Neural Magic Raise the Bar for High-Performance Computing

Increasing Inference Performance with Sparsity and AMD Milan-X

In alignment with AMD’s latest launch, Neural Magic is pushing CPU-based neural network execution to new heights. Using only software and SOTA sparsification research, Neural Magic achieves a 3x relative speedup of inference performance for sparse BERT NLP and ResNet-50 image classification models, with a nearly 20-25% boost attributed to the L3 cache increase from… Read More Increasing Inference Performance with Sparsity and AMD Milan-X

Neural Magic CE 0.7, 0.8, and 0.9 Product Releases

The full technical release notes are always found within our GitHub release indexes linked from our Docs website or the specific Neural Magic repository. SparseZoo The latest additions to sparsezoo.neuralmagic.com! Sparse BERT mask language modeling models with example recipes for transferring to other downstream datasets  Pruned-Quantized BERT models on SQuAD (Question Answering)  YOLACT – for image segmentation  DeepSparse Engine Optimization… Read More Neural Magic CE 0.7, 0.8, and 0.9 Product Releases

Sparsify Hugging Face BERT for Better CPU Performance & Smaller File Size

Get Started: Sparsify Hugging Face BERT Using Your Data Check out our previous blog post to learn about compound sparsification and how it enables faster and smaller BERT models: Pruning Hugging Face BERT: Using Compound Sparsification for Faster CPU Inference with Better Accuracy. Ready to sparsify Hugging Face BERT? You can replicate the performance and… Read More Sparsify Hugging Face BERT for Better CPU Performance & Smaller File Size

Neural Magic Announces $30 Million Series A Funding Led by NEA

Neural Magic, the AI company building a software platform for deep learning inference, today announced a $30 million Series A funding round led by existing investor NEA with participation from Andreessen Horowitz, Amdocs, Comcast Ventures, Pillar VC, and Ridgeline Ventures. This financing brings the company’s total amount raised to $50 million. The new capital will… Read More Neural Magic Announces $30 Million Series A Funding Led by NEA

Pruning Hugging Face BERT: Using Compound Sparsification for Faster CPU Inference with Better Accuracy

Pruning Hugging Face BERT: Apply both pruning and layer dropping sparsification methods to increase BERT performance anywhere from 3.3x to 14x on CPUs depending on accuracy constraints In this post, we go into detail on pruning Hugging Face BERT and describe how sparsification combined with the DeepSparse Engine improves BERT model performance on CPUs. We’ll… Read More Pruning Hugging Face BERT: Using Compound Sparsification for Faster CPU Inference with Better Accuracy

YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance and a Smaller Footprint

For YOLOv3, read our previous blog: YOLOv3 on CPUs: Sparsifying to Achieve GPU-Level Performance Prune and quantize YOLOv5 for a 10x increase in performance with 12x smaller model files. Neural Magic improves YOLOv5 model performance on CPUs by using state-of-the-art pruning and quantization techniques combined with the DeepSparse Engine. In this blog post, we’ll cover our… Read More YOLOv5 on CPUs: Sparsifying to Achieve GPU-Level Performance and a Smaller Footprint

New Tutorial: Sparsifying YOLOv3 Using Recipes

Sparsifying YOLOv3 (or any other model) involves removing redundant information from neural networks using algorithms such as pruning and quantization, among others. This sparsification process results in many benefits for deployment environments, including faster inference and smaller file sizes. Unfortunately, many have not realized the benefits due to the complicated process and number of hyperparameters… Read More New Tutorial: Sparsifying YOLOv3 Using Recipes