Sparsifying YOLOv3 (or any other model) involves removing redundant information from neural networks using algorithms such as pruning and quantization, among others. This sparsification process results in many benefits for deployment environments, including faster inference and smaller file sizes. Unfortunately, many have not realized the benefits due to the complicated process and number of hyperparameters… Read More New Tutorial: Sparsifying YOLOv3 Using Recipes
Neural Magic Appoints Brian Stevens as Chief Executive Officer
We are excited to announce that industry veteran Brian Stevens will be joining Neural Magic as Chief Executive Officer. Brian brings vast experience in open source, enterprise, and hyper-scale cloud to the team. Before joining Neural Magic, Brian was Vice President and CTO of Google Cloud and Executive Vice President and CTO of Red Hat.… Read More Neural Magic Appoints Brian Stevens as Chief Executive Officer
YOLOv3 on CPUs: Sparsifying to Achieve GPU-Level Performance
Use CPUs to decrease costs and increase deployment flexibility while still achieving GPU-class performance. In this post, we elaborate on how we used state-of-the-art pruning and quantization techniques to improve the performance of the YOLOv3 on CPUs. We’ll show that by leveraging the robust YOLO training framework from Ultralytics with SparseML’s sparsification recipes it is… Read More YOLOv3 on CPUs: Sparsifying to Achieve GPU-Level Performance
How Neural Magic’s Deep Sparse Technology Works
To understand how Neural Magic’s Deep Sparse technology works, it’s important to quickly cover the journey of our founders. While mapping the neural connections in the brain at MIT, Neural Magic’s founders Nir Shavit and Alexander Matveev were frustrated with the many limitations imposed by GPUs. Along the way, they stopped to ask themselves a… Read More How Neural Magic’s Deep Sparse Technology Works
ResNet-50 on CPUs: Sparsifying for Better Performance on CPUs
In this post, we elaborate on how we measured, on commodity cloud hardware, the throughput and latency of five ResNet-50 v1 models optimized for CPU inference. By the end of the post, you should be able reproduce these benchmarks using tools available in the Neural Magic GitHub repo, ultimately achieving better performance for ResNet-50 on CPUs.… Read More ResNet-50 on CPUs: Sparsifying for Better Performance on CPUs
Accelerating Machine Learning Inference on CPU with VMware vSphere and Neural Magic
This blog was originally posted by Na Zhang on VMware’s Office of the CTO Blog. You can see the original copy here. Increasingly large deep learning (DL) models require a significant amount of computing, memory, and energy, all of which become a bottleneck in real-time inference where resources are limited. In this post, we detail our… Read More Accelerating Machine Learning Inference on CPU with VMware vSphere and Neural Magic
Sparsify is Open Sourced – Try it Now
Today, we are very excited to provide you with early access to Sparsify, our automated model optimization tool! As deep learning models continue to grow in size, deploying and running them performantly and accurately has required significant investments in flops and system resources. Take GPT-3 for example, with over 175 billion parameters, it takes nearly… Read More Sparsify is Open Sourced – Try it Now
Neural Magic 1.4 Product Release
We are excited to announce the Neural Magic 1.4 product release. This milestone contains new product features, an improved user experience, and stability enhancements that will simplify the ability for our clients to achieve GPU-class performance on commodity CPUs. NEW – Introducing Sparsify BETA Experience driven tooling to simplify the process of analyzing and optimizing… Read More Neural Magic 1.4 Product Release
Product Release Notes
Release 0.1.0 for the Community! February 4, 2021 As of February 2021, our products have been renamed, most have been open sourced and their release notes can be be found in GitHub! Sparsify SparseML (formerly Neural Magic ML Tooling) SparseZoo (formerly Neural Magic Model Repo) DeepSparse Engine (formerly Neural Magic Inference Engine) Release 1.4.0 January… Read More Product Release Notes
Using Sparse-Quantization in Inference: NeurIPS 2020
Did you know that most weights in a neural network are actually useless? In other words, most weights can be removed with little to no impact on the loss. But, how and why would you optimize a deep learning model in practice? Through a combination of pruning and quantization (or “sparse-quantization”) you can drastically improve… Read More Using Sparse-Quantization in Inference: NeurIPS 2020