Overview

Olyx sends model requests through configured endpoints or provider plugins and records the results in SQLite. Use terminal commands for individual requests or run olyx serve for an application.

For a first request, follow the quickstart.

Models and routes

A model alias is a name in your configuration, such as fast. It points to an ordered list of providers. Each provider specifies the actual model to call.

Olyx tries the next provider on connection failures, HTTP 429, or provider server errors. Authentication errors and invalid requests stop the route. Once streamed output begins, Olyx does not switch providers.

Connect Hugging Face, OpenAI, or a local compatible server using the provider guide. For another API, implement a provider plugin.

Requests and traces

A request selects a configured alias. Olyx records its input, response, provider attempts, errors, and latency in a trace with a unique ID. Embedding traces retain the input and response summary; vector values are returned to your application without being duplicated in SQLite.

olyx route --input request.json
olyx debug TRACE_ID
olyx replay TRACE_ID

Replace TRACE_ID with the ID returned by route. Debug reads the saved trace. Replay sends a new request using the current configuration and keeps the original trace. See debug, replay, and storage for details.

Application requests

export OLYX_SERVER_API_KEY=your-application-secret
olyx serve --local --config olyx.json

The HTTP server shares routes and trace storage with terminal commands. It exposes Chat Completions, embeddings, and model aliases. Applications authenticate with OLYX_SERVER_API_KEY; provider credentials authorize calls to upstream models. See application integration for client and container setup.

Application responsibilities

Your application executes returned tool calls. For RAG, it also owns document ingestion, vector storage, and retrieval; Olyx provides the embeddings endpoint.

Local inference servers load and run model weights. Olyx connects to their HTTP endpoint or a plugin; it does not load model files itself.

For content checks, use the separate Olyx Guardrails gem at the boundaries your Ruby application chooses.