Nov 06, 2019
Author(s)
Neural Magic, the “No-Hardware AI” company, is announcing a $15 million seed investment led by Comcast Ventures, and including NEA, Andreessen Horowitz, Pillar VC and Amdocs. With this investment, Neural Magic launches the early access program for its Inference Engine, a software-only solution to processing deep learning workloads on general purpose CPUs, eliminating the need for specialized AI hardware. This fundraise follows a $5 million pre-seed round, bringing the company’s total amount raised to $20 million.
Founded by an award-winning team from MIT, Neural Magic eliminates the need for specialized AI accelerators, such as GPUs or TPUs, to process deep learning models. Today’s data science teams often rely on costly and memory-limited hardware accelerators despite their many drawbacks. Most teams find that they have to sacrifice accuracy in favor of performance gains, due to the high cost and memory constraints of GPUs. To solve this problem, Neural Magic uses a novel software algorithm on commodity CPU hardware to deliver equivalent or better speeds on a variety of networks, from traditional ResNet and MobileNet to state-of-the-art EfficientNets. With Neural Magic, data scientists can achieve similar or better prediction performance results to specialized chips at significantly lower costs.
“Neural Magic proves that high performance execution of deep learning models is not just about FLOPS, it’s a systems engineering problem that can be solved with the right algorithms in software,” said Neural Magic’s co-founder and CEO Nir Shavit. “Our vision is to enable data science teams to take advantage of the ubiquitous computing platforms they already own to run deep learning models at GPU speeds -- in a flexible and containerized way that only commodity CPUs can deliver.”
First use cases for Neural Magic’s Inference Engine are real-time recommendation systems and computer vision applications. Most recommendation engines rely on very large models that can be difficult for memory-constrained GPUs to process effectively in production environments. To contrast, Neural Magic’s software runs recommendation systems at accelerator speeds on commodity CPUs, which have more than enough memory to handle these large models. As a result, eCommerce sites can return more accurate recommendations to customers in a fraction of the time.
For computer vision applications, Neural Magic’s software allows using existing, in-house CPUs or less expensive cloud CPU resources to run tasks like image classification and object detection, all at GPU speeds. Apart from cost savings, Neural Magic enables execution on larger images and video streams, and adds the flexibility of delivering containerized applications to clients without requiring them to add hardware to their deployment infrastructure.
“Neural Magic’s ability to turn commodity CPU processors into a lightning fast AI acceleration platform unlocks incredible opportunities for deep learning usage,” said Gil Beyda, Managing Director at Comcast Ventures and lead investor in Neural Magic. “Neural Magic is well down the path of using software to replace high-cost, specialized AI hardware. Software wins because it unlocks the true potential of deep learning to build novel applications and address some of the industry’s biggest challenges.”
Neural Magic will use the round to build the team, and is hiring for machine learning engineers, software engineers, and sales and marketing roles. To join the early access program, sign up today via the company’s website. Visit Neural Magic at NeurIPS on December 8-14 at Booth #9 .
