Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across SGLang, vLLM, TRT-LLM, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

An Agent Development Kit (ADK) allowing for seamless creation of A2A-compatible agents written in Go

A2A agent server enabling Google Calendar scheduling, retrieval, and automation

A command-line tool to scaffold and manage enterprise-ready AI Agents powered by the A2A (Agent-to-Agent) protocol

An Agent Development Kit (ADK) allowing for seamless creation of A2A-compatible agents written in Rust

An SDK written in Go for the Inference Gateway

An agentic command-line assistant that writes code, understands project context, and uses tools to perform real tasks.

An SDK written in Rust for the Inference Gateway

The UI for the inference-gateway, providing a user-friendly interface to interact with and visualize inference results and manage models

Extensive documentation of the inference-gateway

An SDK written in Typescript for the Inference Gateway

This project provides a Kubernetes Operator for managing the lifecycle of the inference-gateway and its related components. It simplifies deployment, configuration, and scaling of the gateway within Kubernetes clusters, enabling seamless integration of inference workflows.

Architectural Reference and System Flow Documentation for Distributed LLM Inference. Mapping the control-plane integration between the Kubernetes Gateway API Inference Extension and the LLM-D Inference Scheduler data-plane.

Inference Ops exercise for my class

Core Elixir primitives for building reliable self-hosted inference clients, provider adapters, transport boundaries, and operational controls for private AI runtimes across local, edge, and dedicated infrastructure.

An Agent Development Kit (ADK) allowing for seamless creation of A2A-compatible agents written in TypeScript

An SDK written in Python for the Inference Gateway