Creating a Provider Plugin

Olyx owns aliases, ordered fallback, request limits, HTTP serving, trace storage, replay, and diagnostics. It sends HTTP requests in the OpenAI-compatible format. A provider plugin is a local program that you add to the configuration. It reads an Olyx request, calls another service or model engine, and returns the response.

Prerequisites

Install Olyx and place olyx on PATH. The walkthrough uses Python 3 and its standard library. A real adapter can use any language capable of reading stdin and writing newline-delimited JSON. No model service or provider credential is needed for the walkthrough.

Creating a Minimal Adapter

Work in a new directory. Save the following as echo_adapter.py:

import json
import sys


def emit(event):
    print(json.dumps(event), flush=True)


emit({
    "type": "hello",
    "protocol_version": 1,
    "capabilities": ["chat", "streaming"]
})

envelope = json.loads(sys.stdin.readline())
if envelope["protocol_version"] != 1 or envelope["operation"] != "chat":
    emit({"type": "error", "code": "invalid_request"})
    raise SystemExit()

request = envelope["request"]
text = request["messages"][-1].get("content")
if not isinstance(text, str):
    emit({"type": "error", "code": "invalid_request"})
    raise SystemExit()

output = {
    "choices": [{
        "index": 0,
        "message": {"role": "assistant", "content": text},
        "finish_reason": "stop"
    }]
}
if request.get("stream"):
    emit({"type": "snapshot", "output": output})
emit({"type": "result", "output": output})

This example returns the last text message unchanged. It demonstrates how a plugin communicates with Olyx; it does not run a model or report token usage. It does not advertise tools, images, or embeddings.

Registering the Executable

Generate echo.json in the same directory:

python3 - <<'PYTHON'
import json
from pathlib import Path
import sys

configuration = {
    "models": {
        "echo": [{
            "protocol": "plugin",
            "model": "echo-v1",
            "plugin": {
                "command": sys.executable,
                "args": [str(Path("echo_adapter.py").resolve())],
                "capabilities": ["chat", "streaming"]
            }
        }]
    }
}
with Path("echo.json").open("x") as output:
    json.dump(configuration, output, indent=2)
PYTHON

The configuration resolves the interpreter and script to absolute paths. Olyx launches them directly rather than through a shell. Exclusive file creation prevents replacing an existing configuration. There is no api_key_env because this adapter does not use credentials.

Sending a Request

Save request.json:

{
  "model": "echo",
  "messages": [{"role": "user", "content": "Hello from a plugin."}]
}

olyx check echo.json
olyx route --config echo.json --input request.json

The response contains Hello from a plugin. and a generated trace ID. Olyx creates data/traces.db beside echo.json and records the attempt. Replace TRACE_ID with the returned value:

olyx debug TRACE_ID --config echo.json
olyx replay TRACE_ID --config echo.json

Replay starts a new adapter process and creates a new trace. The original trace remains unchanged. This echo adapter runs entirely locally; replaying a real service adapter can make another upstream request.

To stream the same request, add "stream": true to request.json and route it again. The terminal receives a snapshot followed by a completed event. The adapter provides cumulative output; Olyx creates the CLI and HTTP envelopes.

Testing the Adapter

From a CLI source checkout, run the conformance kit with absolute configuration and binary paths as needed:

python3 scripts/test-plugin \
  --binary target/debug/olyx \
  --config /path/to/echo.json \
  --alias echo \
  --report echo-contract.json

This adapter declares chat and streaming, so its checks cover configuration, chat/trace/replay, and streaming. The repository’s richer echo fixture also supports tools and embeddings. See Testing Olyx and Provider Plugins for that complete account-free example, reports, and optional live checks.

Configuring a Service Adapter

{
  "models": {
    "custom": [{
      "protocol": "plugin",
      "model": "my-model",
      "api_key_env": "MY_PROVIDER_KEY",
      "plugin": {
        "command": "/absolute/path/to/adapter",
        "args": [],
        "capabilities": ["chat", "streaming", "tools", "embeddings"],
        "env": [],
        "timeout_ms": 60000
      }
    }]
  }
}

command must be absolute. Arguments are passed directly; no shell is used.

For a Python adapter, set command to your interpreter and pass the absolute script path in args. Omit api_key_env for an account-free local adapter. Do not set base_url for a plugin. List only capabilities the adapter implements. An unsupported requested capability fails before launching the executable. Embedding replicas must retain the same model identity, protocol, and approved response identifiers, just as with HTTP providers.

Use ordinary route, serve, debug, replay, and comparison commands. Plugins are also invoked during replay, which may make new provider calls. The CLI returns tool calls; your application executes tools.

Version 1 protocol

One subprocess handles one invocation. It reads one newline-terminated JSON object from stdin, then stdin closes:

{"protocol_version":1,"operation":"chat","model":"my-model","request":{"model":"my-model","messages":[{"role":"user","content":"Hello"}],"max_tokens":128}}

Operation is chat or embeddings. Request fields use the common OpenAI schema; model is already resolved from the alias. Respect stream and token controls. Translate fields you support and reject unsupported options rather than dropping them. The core applies its normal body/output limits and rejects native_request.

Stdout is exclusively newline-terminated JSON events. First send:

{"type":"hello","protocol_version":1,"capabilities":["chat","streaming","tools","embeddings"]}

Both the configuration and the hello event must list the capabilities needed by the request. For chat without streaming, return a Chat Completions response:

{"type":"result","output":{"choices":[{"index":0,"message":{"role":"assistant","content":"Hello"},"finish_reason":"stop"}],"usage":{"prompt_tokens":2,"completion_tokens":1,"total_tokens":3}}}

For streaming chat, emit cumulative snapshot events using the same output shape, followed by result. The last snapshot must equal the final result; include final finish reasons and usage before completing. Each snapshot is validated and sent to the CLI/HTTP client. Do not emit OpenAI SSE on plugin stdout: Olyx creates the client-facing SSE or NDJSON envelope.

For embeddings, send one result with the standard model/data/usage shape:

{"type":"result","output":{"model":"my-model","data":[{"index":0,"embedding":[0.1,0.2]}],"usage":{"prompt_tokens":2,"total_tokens":2}}}

The core validates vector dimensions, ordering, finite values, batch size, usage, and model identity. Embedding traces store summaries; replay reexecutes the recorded input. Match requested dimensions and encoding_format.

For failure, send an error after hello:

{"type":"error","code":"unavailable"}

Codes are unavailable, rate_limited, authentication_failed, or invalid_request. Unknown codes become plugin_failed. Do not put provider messages or credentials on stdout. Only unavailable/rate_limited permit ordered fallback, and only before a chat snapshot was emitted. A timeout, malformed event, version mismatch, nonzero exit, missing result, result mismatch, or extra event after result fails the attempt. Core traces retain the stable error and any partial chat output.

The executable must exit successfully after its result. Each event is limited to 1 MiB; total chat output is limited to 8 MiB. Embeddings use the configured max_embedding_response_bytes total limit with the same per-event limit. Default timeout is 60 seconds; configure 100–300000 ms. If the request is cancelled or times out, Olyx stops the plugin process. Plugins must stop any additional processes they start.

Execution and credentials

Installed plugins are trusted executable code, not sandboxed extensions. Olyx does not install or discover executables automatically. A child receives PATH, system/temp variables, the selected api_key_env value, and only additional variables listed in plugin.env. Other credentials are not passed to the child. Olyx does not include credentials in the JSON sent to the plugin. Plugin stderr is discarded to avoid leaking raw provider errors into output. Version 1 diagnostics use fixed codes; it provides no automatic credential refresh, plugin signing, sandboxing, or upstream certification.

Account-free development and tests

examples/plugins/echo.py is a runnable adapter fixture. It echoes text, returns synthetic vectors and tool calls, and never contacts a provider. It is a test fixture, not a model. Use it as a starting point for a fixture mode in a real adapter, keeping SDK calls behind a separately tested translation layer.

./scripts/test-providers
python3 scripts/smoke-plugins.py target/debug/olyx

The core contract smoke checks CLI and HTTP chat, streaming, tools, embedding batches, trace persistence, replay immutability, capabilities, version errors, malformed output, timeouts, child failures, environment isolation, eligible fallback, and no fallback after partial output. It requires no provider account.

Test an independently developed adapter using its fixture configuration:

./scripts/test-plugin --config /path/to/plugin-fixture.json --alias custom \
  --report plugin-contract.json

The kit tests declared chat/streaming/tools/embeddings capabilities through real CLI/HTTP paths. It launches the configured executable; use your adapter’s fixture mode to avoid network calls and provider charges. Its sanitized report establishes plugin contract behavior, not real service compatibility. Live provider testing can be performed independently by the adapter’s maintainer or a contributor.