Challenges
Large models are inefficient.
Many of the top models across NLP and computer vision domains are difficult and expensive to use in a real-world deployment. While they are accurate, they are computationally intensive, which can require inflexible hardware accelerators in production.
Small models are less accurate.
Smaller models, while faster and more efficient, deliver less accuracy on real-world data.
How it Works
SparseML enables you to create inference-optimized models using state-of-the-art pruning and quantization algorithms.
Models trained with SparseML in PyTorch can then be exported and deployed with nm-vllm and DeepSparse.
Product Overview
Boost model performance with the same accuracy by introducing sparsity.
Fine tune pre-sparsified versions of common models like BERT, YOLOv5, and ResNet-50 onto your datasets.
Use with your existing SOTA software like PyTorch, Ultralytics, and Hugging Face.
Gain visibility around model experiment tracking through TensorBoard and Weights & Biases.
Export your sparse models to ONNX format for deployment with DeepSparse.
Get started with just a few lines of code.
Modify any training graph and easily get to a functioning implementation.
Inspect, modify, and enhance software that’s maintained with SOTA algorithms.
Optimize
Accelerate Inference
Deploy models optimized by SparseML with GPU-class performance on DeepSparse.
Use Common Models
Convenient integrations for optimizing PyTorch, Ultralytics, and Hugging Face models.
Leverage the Latest
Adopt state-of-the-art (SOTA) compression algorithms to make inference efficient.
Learning & Impact
Products
Oct 17, 2024
We Ran Over Half a Million Evaluations on Quantized LLMs: Here's What We Found
Open Source
Oct 14, 2024
Introducing Machete, a Mixed-Input GEMM Kernel Optimized for NVIDIA Hopper GPUs
Open Source
Aug 14, 2024
LLM Compressor is Here: Faster Inference with vLLM
