Skip to content
Olyx

olyx serve

Start a model endpoint for an application using configured routes and local SQLite traces.

olyx serve [--local] [--config FILE] [--host IP] [--port PORT] [--max-concurrency N]

Start an HTTP server using configured model aliases and ordered fallbacks; successful and failed attempts are recorded in the same SQLite database used by route, replay, and debug.

Option Default Meaning
--local Off Restrict the listener to loopback addresses
--config FILE OLYX_CONFIG or ./olyx.json Configuration to load at startup
--host IP 127.0.0.1 Bind address; IPv4 or IPv6 literal
--port PORT 8787 Listen port; 0 selects an available port
--max-concurrency N 32 Maximum active chat and embedding requests; excess requests return 429

Startup errors go to stderr and exit with status 1. The resolved listen URL is written to stderr.

POST /v1/chat/completions accepts a Chat Completions request. Set stream: true for SSE. POST /v1/embeddings generates document and query vectors. GET /v1/models lists aliases. /healthz checks process liveness and /readyz checks local storage. Configuration changes require a restart.

Start and query the server

Section titled “Start and query the server”
Terminal window
export OLYX_SERVER_API_KEY=your-application-secret
olyx serve --local --config olyx.json

From a second terminal, set the same application secret, then query the endpoint:

Terminal window
export OLYX_SERVER_API_KEY=your-application-secret
curl http://127.0.0.1:8787/v1/models \
  -H "Authorization: Bearer $OLYX_SERVER_API_KEY"


curl http://127.0.0.1:8787/v1/chat/completions \
  -H "Authorization: Bearer $OLYX_SERVER_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"fast","messages":[{"role":"user","content":"Hello"}]}'

Health probes do not require that key:

Terminal window
curl http://127.0.0.1:8787/healthz
curl http://127.0.0.1:8787/readyz

Readiness checks SQLite access; it does not verify provider credentials or model access.

See Connect an application for client examples, authentication, Docker Compose, persistence, streaming errors, and shutdown behavior.