Neural MagicInference Engine

Intelligent, no-hardware AI inference engine. Deep learning performance, simplified.

GPU-class performance on everyday CPUs. The future is now.

Get Started

GPU-Class Performance with Software Flexibility

Costly, complex, memory limited, resource and deployment constrained. That’s deep learning on specialized hardware accelerators. Sound familiar?

Neural Magic Inference Engine software lets data science teams use ubiquitous CPU resources to achieve machine learning performance breakthroughs, at scale, no expensive hardware required.

Improve prediction accuracy while lowering your deep learning costs. Neural Magic turns CPUs you already own into high performance machine learning resources.

Simplify Performance
Start quickly with Neural Magic optimized versions of popular image classification and object detection models. Alternatively, put our recalibration tools to work on your own models to maximize performance.
Reduce Costs
Our proprietary algorithms deliver GPU-class performance, without the expense. Deploy models on cheaper and more commonly available CPU servers. Scale model pipelines on a single, flexible CPU instance configuration.
Run Anywhere
Integrate seamlessly into existing deployment pipelines. Deploy in containers or virtual machines, and manage with Kubernetes like any software application. Run anywhere: on premise, in the cloud, or at the edge.
01

Pick a model

Start with a trained model and dataset. Or select one of the many Neural Magic performance-tuned models in our model repo.
02

Accelerate

Using Neural Magic’s state-of-the-art recalibration tools, tune the accuracy and performance of your model using industry leading techniques. Our recalibration tools make sparsification simple to implement, while delivering the performance you need.
03

Deploy

Deploy in the cloud, on premise, at the edge. Plug quickly into your own model serving environment via API or deploy Neural Magic’s runtime stack.
04

Scale

Manage Neural Magic runtime scale-out via Kubernetes. Respond quickly to changing traffic demands, monitor model performance, and reduce infrastructure costs - all while exceeding user expectations.

Neural Magic In the News

Neural Magic allows any company to deploy a deep learning model without the need for specialized hardware. It not only lowers the costs of deep learning but also makes AI more widely accessible.
MIT Technology Review »
An inference engine that data scientists can run on computers running CPUs, rather than specialized chips like GPUs or TPUs. Greatly reduce the cost associated with machine learning projects by allowing data scientists to use commodity hardware.
TechCrunch »
Neural Magic proves that high performance execution of deep learning models is … a systems engineering problem that can be solved with the right algorithms in software.
VentureBeat »

Start Using the Neural Magic Inference Engine Now

Try the Neural Magic Inference Engine for yourself. See our breakthrough performance in minutes.

Our team is here to help you get started.

See a Live Demo Get Started