KServe
Kubernetes-native platform for serving machine learning models at scale, standardizing model deployment across frameworks.
- Actively maintained
- Foundation-backed
- Well documented
- Permissive license
Overview
KServe provides a standard Kubernetes custom resource for deploying trained models - from scikit-learn, PyTorch, TensorFlow, and others already in this catalog - with built-in autoscaling (including scale-to-zero), canary rollouts, and a consistent inference API regardless of which framework trained the model. It's a CNCF project built specifically for general-purpose ML model serving rather than LLMs specifically.
KServe is a fit for platform teams standardizing how models get deployed across many data science teams and frameworks, a broader-scope serving platform than vLLM or SGLang (also in this catalog), which focus specifically on LLM inference.
Why it's in this catalog
Catalog rating: ★★★☆☆ (3/5)
Standardized Kubernetes model-serving layer, complements Kubeflow/MLflow.
- Mature
- Foundation-governed
- Categories
- AI & Machine Learning
- Languages
- Python, Go
- License
- Apache-2.0
Related Projects
Kubeflow
Apache-2.0Machine learning platform for Kubernetes that bundles pipelines, model training, hyperparameter tuning, and serving into one toolkit.
Actively maintained
- AI & Machine Learning
MLflow
Apache-2.0Open-source platform for managing the machine learning lifecycle: experiment tracking, model packaging, versioning, and deployment.
Actively maintained
- AI & Machine Learning
vLLM
Apache-2.0High-throughput, memory-efficient inference and serving engine for large language models, built for running LLMs in production at scale rather than on a single local machine.
Actively maintained
- AI & Machine Learning
llama.cpp
MITHigh-performance C/C++ implementation for running LLM inference locally on consumer hardware, including CPUs, with minimal dependencies.
Actively maintained
- AI & Machine Learning
Spotted an error? Suggest an edit on GitHub.