Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.llmgrid.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
What is LLMGrid?

LLMGrid is an enterprise‑ready AI gateway and orchestration platform that provides a single control plane for using large language models, tools, and agents. It centralizes access, governance, routing, safety, and observability—without requiring application rewrites. LLMGrid exposes an OpenAI‑compatible API, so existing SDKs and frameworks work by simply pointing to the LLMGrid proxy.
Image

​
Core Capabilities

​
OpenAI‑Compatible Proxy

  • Drop‑in replacement for OpenAI SDKs
  • Minimal code changes (update base_url and API key)
  • Supports chat, embeddings, streaming, tools, and function calling

​
Model & Traffic Routing

  • Route requests across models and providers
  • Configure fallbacks and retry strategies
  • Use aliases to keep application code stable during model changes
  • Optimize for availability, latency, or efficiency

​
Governance & Safety

  • Virtual Keys for scoped access and limits
  • Guardrails for input/output and tool enforcement
  • Budgets and rate limits to prevent overuse
  • Tags for routing, attribution, and segmentation

​
Agents, Tools & Retrieval

  • Register Agents with skills and capabilities
  • Attach Search Tools for live retrieval and grounding
  • Manage Vector Stores for RAG workflows
  • Secure tool execution with pre‑execution checks

​
Observability & Analytics

  • Request and audit logs
  • Usage and cost analytics by model, key, team, tag, or agent
  • Cache analytics and health checks
  • End‑to‑end visibility for debugging and audits

​
Performance & Efficiency

  • Response Caching (Redis‑backed) to reduce latency and repeat calls
  • Semantic caching for similarity‑based reuse
  • Centralized cost tracking and discounts

​
How LLMGrid Fits In

LLMGrid sits between your applications and AI capabilities: Your App ↓ LLMGrid (Auth • Routing • Guardrails • Observability) ↓ Models • Tools • Vector Stores • Search This architecture lets teams evolve models and controls independently of application code.

​
Who Should Use LLMGrid?

  • Platform teams needing centralized governance
  • Developers shipping AI features quickly
  • Security & compliance teams enforcing policies
  • FinOps teams monitoring usage and cost
  • Enterprises running multi‑model, multi‑tool AI workloads

​
Getting Started

  1. Create a Virtual Key
  2. Point your OpenAI SDK to the LLMGrid proxy
  3. Configure models and routing
  4. Add guardrails, budgets, and observability
  5. Iterate safely as usage grows
  • API Reference – OpenAI‑compatible endpoints
  • Models – Configure available models
  • Router Settings – Control routing and fallbacks
  • Guardrails – Enforce safety and compliance
  • Usage & Logs – Observe and analyze traffic
  • Security & Compliance – Enterprise controls and governance