Neural Magic 1.1 Product Release


We are excited to announce the Neural Magic 1.1 product release. This product milestone contains new feature updates, an improved user experience, and stability enhancements that will simplify the ability for our clients to achieve GPU-class performance on commodity CPUs. 

Neural Magic Inference Engine

Enables clients to run mission critical deep learning models on commodity CPUs to reduce cost per inferences and generate price-performant deployments. This feature set includes the inference engine, ONNX conversion tooling, model server if needed, and is focused on model deployment and scaling machine learning pipelines.

  • Support for the AVX2 instruction set and tested AMD AVX2 chipsets
  • Ability to run U-Net convolutional network for image segmentation
  • Addition of model performance diagnostics mode during runtime execution
  • SPLIT operator support
  • Improved open-source TF2ONNX converter in support of newer TensorFlow versions 
  • Simplified packaging to improve user journey from evaluation to test

Neural Magic ML Tooling

Enables data scientists to optimize their model for performance without having to sacrifice accuracy required for business outcomes. This feature set includes model pruning APIs and command line interfaceCLIs as well as transfer learning APIs and CLIs, simplifying the process of achieving performance on deep learning models with Neural Magic.

  • Pruning command-line interface (CLI) for ease of use and rapid prototyping
  • Transfer learning command-line interface (CLI) for ease-of-use and rapid prototyping
  • Pruning for Success best practices guide and getting started documentation 
  • ONNX API for model and pruning analysis, as well as model conversions
  • PyTorch API improvements for pruning and transfer learning analysis
  • New notebook experience to demo installation and benchmarking process

Neural Magic Model Repo

Simplify time to value and reduce skill burden to build performant deep learning models by having a collection of pre-trained, performance-optimized deep learning models to prototype from. The repository consists of popular image classification and object detection models and is constantly growing.

Performant model additions:

  • Inception-v3
  • ResNet-101 v1
  • ResNet-152 v1
  • VGG-11
  • VGG-19

Contact us for more information. Or if you are interested in trying our software, please request product access.