Software-Delivered AI - Neural Magic

Software-delivered AI Inference

Forget special hardware. Get GPU-class performance on CPUs with our sparsity-aware inference runtime.

LEARN MORE

4-Core CPU (n2-highcpu-8) | DeepSparse 1.1.0 | 99% Accuracy | Replicate Now

T4 NVIDIA GPU | TensorFlow 20.06-py3 NGC | 100% Accuracy | NVIDIA Numbers

Try it Now

COPIED

PICK A USE CASE

Natural Language Processing (NLP)

Question Answering

Text Classification

Token Classification

Computer Vision

Object Detection

Image Classification

Explore all use cases →

1. Benchmark Your Use Case

2. Train With Your Data

3. Deploy To Your Infrastructure

Best CPU Performance, Guaranteed

Install DeepSparse, our sparsity-aware inference runtime, and benchmark a sparse-quantized version of the Hugging Face BERT-base model.

Throughput speedup: 7x
Accuracy recovery: 99%

See SparseZoo for other sparse models and recipes you can benchmark and prototype from. Click here to see our Sparse Question Answering guide.

Apply Your Data With a Few Lines of Code

Install SparseML, our open-source library, to transfer learn our sparse-quantized model to your dataset using a few lines of code.

The example given in the terminal uses a public dataset. Click here to apply to your own data via transfer learning.

Best CPU Performance, Guaranteed

Install DeepSparse, our sparsity-aware inference runtime, and benchmark a sparse-quantized version of the Hugging Face BERT-base model.

Throughput speedup: 8.1x
Accuracy recovery: 99%

See SparseZoo for other sparse models and recipes you can benchmark and prototype from. Click here to see our Sparse Text Classification: Sentiment Analysis guide.

Apply Your Data With a Few Lines of Code

Install SparseML, our open-source library, to transfer learn our sparse-quantized model to your dataset using a few lines of code.

The example given in the terminal uses a public dataset. Click here to apply to your own data via transfer learning.

Apply Your Data With a Few Lines of Code

Install SparseML, our open-source library, to transfer learn our sparse-quantized model to your dataset using a few lines of code.

The example given in the terminal uses a public dataset. Click here to apply to your own data via transfer learning.

Best CPU Performance, Guaranteed

Install DeepSparse, our sparsity-aware inference runtime, and benchmark a sparse-quantized version of the YOLOv5s model to achieve a 12x speedup over PyTorch CPU.

See SparseZoo for other sparse models and recipes you can benchmark and prototype from. Click here to see our YOLOv3 and YOLOv5 benchmarking example in GitHub.

Best CPU Performance, Guaranteed

Install DeepSparse, our sparsity-aware inference runtime, and benchmark a sparse-quantized version of the ResNet-50 model to achieve a 7x speedup over ONNX Runtime CPU with 99% of the baseline accuracy.

See SparseZoo for other sparse models and recipes you can benchmark and prototype from. Click here to see our YOLOv3 and YOLOv5 benchmarking example in GitHub.