Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.llmgrid.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

The Model Management area is where administrators configure the models available through the platform proxy. It includes tooling to:
  • Register new models and set how they are exposed to users
  • Manage provider credentials securely
  • Configure pass-through endpoints for external APIs
  • Monitor model health and run checks on demand
  • Tune retry behavior for common failure types
  • Create model group aliases to simplify API usage
  • Reload pricing metadata used for usage analytics

​
Tabs

​
All Models

The All Models tab lists every configured model for the current tenant and selected team context.

​
Team context and view

  • Current Team: controls which team context is being viewed (for example, a personal context vs. a shared team).
  • View: controls how the list is filtered (for example, showing models available to the current team).
A helper note may appear indicating how to access models that are not tied to a team, depending on how your tenant is configured.

​
Search and filters

  • Search model names to quickly locate a model.
  • Filters to narrow results by attributes.
  • Reset Filters to clear applied filters.

​
Typical columns

Depending on tenant configuration, the table can include:
  • Model ID
  • Model Information (public-facing model name)
  • Credentials (credential set used)
  • Created By / Updated At
  • Team ID
  • Model Access Group
  • Status
  • Actions (manage or remove)

​
Add Model

The Add Model tab is used to register a model so it can be routed and accessed via the proxy.

​
Primary fields

  • Provider (required)
    Select the provider backing the model.
  • Provider Model Name(s) (required)
    The provider-specific model identifier(s) the proxy will call.
  • Model Mappings (required)
    Maps a Public Model Name to the underlying provider model name.
    This lets you expose a stable name to users while changing provider details behind the scenes.
  • Mode (optional)
    A selectable mode that can influence how requests are made and/or how health checks are performed (depending on provider configuration).

​
Credentials

You can either:
  • Select Existing Credentials, or
  • Enter new provider credentials in the form (for example, an API key)
Avoid embedding secrets in documentation, tickets, or screenshots. Use named credentials and rotate keys regularly.

​
Access and governance

  • Team-BYOK Model (toggle, optional)
    When enabled, the model can be used in bring-your-own-key patterns depending on tenant policy.
  • Model Access Group (optional)
    Assign the model to an access group to simplify availability rules.

​
Additional settings

  • Additional Model Info Settings (expandable)
    Optional model metadata and configuration fields.

​
Validation and creation

  • Test Connect verifies connectivity and configuration.
  • Add Model saves and registers the model.

​
Add Auto Router (within Add Model)

The Add Auto Router sub-tab creates an intelligent routing configuration that can automatically select a target model based on routing rules.

​
Router configuration

  • Auto Router Name (required)
    A unique name for the router.
  • Routes Configuration
    Use Add Route to define one or more routes.

​
Defaults

  • Default Model (required)
    The model used when no route matches.
  • Embedding Model (optional)
    Used when routing needs embeddings (for example, semantic matching use cases).
  • Model Access Group (optional)
    Restricts router usage to an access group.

​
JSON preview

A JSON Preview section may be available to view the router config before saving.

​
Validation and creation

  • Test Connect validates configuration.
  • Add Auto Router saves the router.

​
LLM Credentials

The LLM Credentials tab stores named credential sets used by models.

​
Credential list

A table typically includes:
  • Credential Name
  • Provider
  • Actions (edit, delete)

​
Add Credential

Select Add Credential to create a new credential set. Common fields include:
  • Credential Name (required)
    Friendly name to reference in model configs.
  • Provider (required)
  • API Base (optional, provider-dependent)
    Base URL for the provider API.
  • Organization / Account identifiers (optional, provider-dependent)
  • API Key / Secret (required, provider-dependent)
Use Add Credential to save, or Cancel to exit.

​
Pass-Through Endpoints

The Pass-Through Endpoints tab configures routes that forward requests to external APIs through the proxy. This is useful for custom services, image generation APIs, or any external endpoint you want to standardize behind a single gateway.

​
Add Pass-Through Endpoint

Select Add Pass-Through Endpoint to open the configuration form.
Route configuration
  • Path Prefix (required)
    The prefix that becomes part of the proxied path (examples are shown in the UI).
  • Target URL (required)
    The external URL requests are forwarded to.
  • Include Subpaths (toggle)
    When enabled, subpaths are forwarded to the target (commonly recommended for REST-style APIs).
Headers
  • Authentication Headers (required)
    Add required headers sent with every request (common examples include Authorization or x-api-key).
  • Add Header
    Add additional headers as needed.
Security
A Security toggle may be available to require a valid platform key to call the pass-through endpoint.
Guardrails (optional)
Pass-through endpoints can be configured with guardrails to enforce policy checks on requests and/or responses.
Field-level targeting
A field targeting helper may be displayed, including examples such as:
  • query (single field)
  • documents[*].text (all text fields in a documents array)
  • messages[*].content (message contents)
When field targeting is left empty, the entire request/response is evaluated by the guardrail.
Billing / cost tracking (optional)
An optional section may exist to configure cost tracking for the pass-through endpoint.
Documentation should describe this feature without listing specific pricing.
Save
  • Add Pass-Through Endpoint saves the configuration.
  • Cancel exits without saving.

​
Health Status

The Health Status tab runs health checks on configured models to verify that they are reachable and operating correctly.

​
Table fields

Common columns include:
  • Model ID
  • Model Name
  • Health Status
  • Error Details
  • Last Check
  • Last Success
  • Actions (run a check for a single model)

​
Run checks

  • Run All Checks initiates health checks for all listed models.
  • Row-level actions allow checking a specific model.
Use this tab when onboarding new models, after credential rotation, or when investigating incidents.

​
Model Retry Settings

The Model Retry Settings tab configures retry behavior for failure types. Settings can be scoped to a specific model and fall back to global defaults when not set.

​
Retry policy scope

  • Retry Policy Scope: choose the model to configure.

​
Common error types shown

The UI may display configurable retry counts for:
  • Bad Request
  • Authentication Error
  • Timeout
  • Rate Limit
  • Content Policy Violation
  • Internal Server Error
Select Save to apply changes.
Recommended practice: keep retries conservative for authentication and bad request errors, and use retries mainly for timeouts and transient rate limits.

​
Model Group Alias

The Model Group Alias tab lets you create user-friendly aliases that map to an underlying model group. This simplifies API calls and supports smoother migrations when you change underlying model groups.

​
Add new alias

  • Alias Name: the friendly name users call
  • Target Model Group: the model group the alias points to
  • Add Alias: saves the alias mapping

​
Manage existing aliases

A list shows saved aliases with actions to update or remove.

​
Configuration example

A configuration preview may be displayed to demonstrate how aliases appear in configuration files.

​
Price Data Reload

The Price Data Reload tab manages internal pricing metadata used for analytics and reporting.

​
Actions

  • Reload Price Data: refreshes pricing metadata immediately.
  • Set Up Periodic Reload: schedules automatic refresh.

​
Status

A status panel may show:
  • Whether periodic reload is scheduled
  • When the last run occurred
Use this after adding new models or when analytics requires updated pricing metadata.
  1. Add credentials in LLM Credentials
  2. Add models in Add Model and validate with Test Connect
  3. Optionally configure Auto Router for intelligent model selection
  4. Configure external services using Pass-Through Endpoints
  5. Verify availability in Health Status
  6. Tune resilience using Model Retry Settings
  7. Create stable names using Model Group Alias
  8. Refresh analytics metadata in Price Data Reload
  • Virtual Keys – Control which models and routes keys can access
  • Teams / Organizations – Apply layered governance boundaries
  • Usage Dashboard – Monitor model and key activity
  • Request Logs – Debug model requests and failures