Hello

ResNet-50 on CPUs: Sparsifying for Better Performance on CPUs

In this post, we elaborate on how we measured, on commodity cloud hardware, the throughput and latency of five ResNet-50 v1 models optimized for CPU inference. By the end of the post, you should be able reproduce these benchmarks using tools available in the Neural Magic GitHub repo, ultimately achieving better performance for ResNet-50 on CPUs.… Read More ResNet-50 on CPUs: Sparsifying for Better Performance on CPUs

Accelerating Machine Learning Inference on CPU with VMware vSphere and Neural Magic

This blog was originally posted by Na Zhang on VMware’s Office of the CTO Blog. You can see the original copy here. Increasingly large deep learning (DL) models require a significant amount of computing, memory, and energy, all of which become a bottleneck in real-time inference where resources are limited. In this post, we detail our… Read More Accelerating Machine Learning Inference on CPU with VMware vSphere and Neural Magic

Sparsify is Open Sourced – Try it Now

Today, we are very excited to provide you with early access to Sparsify, our automated model optimization tool! As deep learning models continue to grow in size, deploying and running them performantly and accurately has required significant investments in flops and system resources. Take GPT-3 for example, with over 175 billion parameters, it takes nearly… Read More Sparsify is Open Sourced – Try it Now

Product Release Notes

Release 0.1.0 for the Community! February 4, 2021 As of February 2021, our products have been renamed, most have been open sourced and their release notes can be be found in GitHub! Sparsify SparseML (formerly Neural Magic ML Tooling) SparseZoo (formerly Neural Magic Model Repo) DeepSparse Engine (formerly Neural Magic Inference Engine) Release 1.4.0 January… Read More Product Release Notes

Neural Magic at NeurIPS 2020

Are you attending this year’s virtual NeurIPS conference? The Neural Magic team would love to meet you.  Who is Neural Magic?  After years of research at MIT, our team concluded that throwing teraflops at dense models is not sustainable. So we’ve taken the best of known research on model compression (unstructured pruning and quantization, in… Read More Neural Magic at NeurIPS 2020

Neural Magic End-to-End Demo Videos

Neural Magic delivers best-in-class deep learning performance on commodity CPUs. We do this via: Model optimization techniques like pruning and quantization Smart algorithms that utilize CPU memory more effectively. To help visualize the power of Neural Magic, we recorded three short end-to-end video guides on how to install our software, prepare and run a model… Read More Neural Magic End-to-End Demo Videos

Part 3: Gradual Magnitude Pruning (GMP) Hyperparameters

TL;DR: To facilitate the GMP process when pruning a network, several hyperparameters must be defined. These include general hyperparameters such as learning rate, pruning update frequency, and pruning schedule function in addition to the sparsity per layer. All hyperparameters affect end level recovery, loss, and performance. Reading time: 5 minutes, 5 seconds Welcome to Part… Read More Part 3: Gradual Magnitude Pruning (GMP) Hyperparameters

Profile: Jeannie Finks

Jeannie Finks, Head of Customer Success at Neural Magic, has over 25 years of experience spanning customer success, digital strategy & implementation, and technical program leadership. Prior to joining Neural Magic Jeannie held numerous hands-on customer success roles at Acquia, a SaaS company whose enterprise products, services, and technical support focus on the open-source CMS… Read More Profile: Jeannie Finks

Challenging Memory Requirements and Performance Standards in ML

Everything we know about memory requirements in machine learning may be wrong.  Today, when data scientists process deep learning models using a “throughput computing” device like a GPU, TPU, or similar hardware accelerator, they’re likely faced with a decision to shrink their model or input size to fit within the device’s memory limitations. Training a… Read More Challenging Memory Requirements and Performance Standards in ML