Announcing Community Support for GPU Inference Serving Over the past five years, Neural Magic has focused on accelerating inference of deep learning models on CPUs. To achieve this, we did two things: Many of the techniques we used to accelerate CPUs to make them more efficient can also help GPUs in their processing of LLMs.… Read More Blog

Announcing Community Support for GPU Inference Serving Over the past five years, Neural Magic has fo...

03.05.2024

[Major Product News] Neural Magic Announces GPU Support for LLM Inference! Over the past several months, our team has been focused on expanding our capabilities to enable LLM inference on GPUs! A few weeks ago, we released our announcement of nm-vllm, our fork of vLLM, with a focus on incorporating the latest LLM optimizations like… Read More Blog

[Major Product News] Neural Magic Announces GPU Support for LLM Inference! Over the past several mon...

03.20.2024

Introducing YOLOv8—the latest object detection, segmentation, and classification architecture to hit the computer vision scene! Developed by Ultralytics, the authors behind the wildly popular YOLOv3 and YOLOv5 models, YOLOv8 takes object detection to the next level with its anchor-free design. But it's not just about cutting-edge accuracy. YOLOv8 is designed for real-world deployment, with a… Read More Blog

Introducing YOLOv8—the latest object detection, segmentation, and classification architecture to h...

01.18.2023