Search open source projects

High-throughput, memory-efficient inference and serving engine for large language models, built for running LLMs in production at scale rather than on a single local machine.

• AI & Machine Learning

High-performance C/C++ implementation for running LLM inference locally on consumer hardware, including CPUs, with minimal dependencies.

• AI & Machine Learning

Fast serving framework for large language models and vision-language models, with a structured generation language for complex LLM programs.

• AI & Machine Learning

Kubernetes-native platform for serving machine learning models at scale, standardizing model deployment across frameworks.

• AI & Machine Learning

Machine learning platform for Kubernetes that bundles pipelines, model training, hyperparameter tuning, and serving into one toolkit.

• AI & Machine Learning

Deep learning optimization library from Microsoft for training and running very large models efficiently across multiple GPUs.

• AI & Machine Learning

Framework for developing applications powered by large language models, agents and retrieval-augmented generation.

• AI & Machine Learning

Open-source platform for managing the machine learning lifecycle: experiment tracking, model packaging, versioning, and deployment.

• AI & Machine Learning

Run large language models locally with a simple command-line interface and API.

• AI & Machine Learning

Open-source tool for testing and evaluating LLM prompts, comparing outputs across models and catching quality regressions before deployment.

• AI & Machine Learning

Library for fine-tuning large language models significantly faster and with less memory than standard training approaches.

• AI & Machine Learning

Unified programming model for defining batch and stream data processing pipelines that can run portably across multiple execution engines.

• Analytics

Open-source access-control library implementing ACL, RBAC, and ABAC authorization models, with production-ready ports for Go, Java, Python, Node.js, and more.

• Authentication & Security

High-performance proxy built for cloud-native applications, serving as the foundational data plane for many modern service meshes and API gateways.

• DevOps

Library for getting reliable, structured data output from LLMs using function-calling APIs and Pydantic models for validation.

• AI & Machine Learning

Data framework for connecting large language models to your own data - documents, databases, APIs - for retrieval-augmented applications.

• AI & Machine Learning

Relationship-based authorization engine inspired by Google Zanzibar, for modeling and enforcing fine-grained access control as a standalone service.

• Authentication & Security

Open source systems monitoring and alerting toolkit with a dimensional data model and powerful query language.

• Monitoring

Microsoft's open-source SDK for integrating large language models into applications, with support for plugins, memory, and planning.

• AI & Machine Learning

Self-hosted, open-source headless CMS that generates REST and GraphQL APIs from visually modeled content types.

• CMS