Skip to main content
Pullfrog can route LLM calls to any endpoint that speaks the OpenAI chat-completions API — a LiteLLM proxy, a Cloudflare AI Gateway, Alibaba DashScope (Qwen), a self-hosted vLLM server, or any compatible gateway. Useful when your model isn’t in the catalog, when you front several providers behind one gateway, or when you run your own inference.
OpenAI-compatible support is BYOK only. Inference is billed by your provider to your account; Pullfrog never proxies this traffic.

​
Setup

Pick the model in the dropdown, then supply the endpoint, key, model ID, and that model’s two token limits. Only the key is sensitive — but there are two ways to supply the set, and which one you want depends on how your org pays for models.

​
1. Select Custom › OpenAI-compatible in the model dropdown

In pullfrog.com/console/<account>/<repo>, open the model dropdown and pick Custom › OpenAI-compatible. Like Bedrock and Vertex, this entry has no per-model variants — the actual model is supplied separately via OPENAI_COMPATIBLE_MODEL (see step 3), because the endpoint, not Pullfrog, owns the model catalog.

​
2. Supply the five values

Either store all five in the console — open Model costs → Custom → Set up and fill the form. Pullfrog injects them at run time, so you never touch pullfrog.yml and can skip step 3 entirely. This is the path to use on a Router-funded org: the server can only honor a stored openai-compatible/byok default when it can see the credentials itself (see the Router note below). Or put OPENAI_COMPATIBLE_API_KEY in a GitHub Actions secret and the other four straight into your workflow env: (step 3). Only the key is sensitive — the base URL, model ID, and token limits are plain config and don’t need encrypting, though storing them in Pullfrog is equally fine. Values in your workflow env: win over Pullfrog-stored ones, so you can store the set once for the org and override a single value per repo.

​
3. Wire the env vars in your workflow (only if you chose the workflow path)

All five must be present, or the run fails fast with a setup error naming what’s missing. The two limits must parse as positive numbers — 128k is rejected.

​
Why the token limits are required

Because your endpoint owns the model catalog, Pullfrog has no metadata about the model’s limits and cannot discover them. Undeclared, they don’t degrade gracefully — they break two things at once:
  • Completions get capped at 32000 tokens, and any model with a smaller cap rejects the request outright: max_tokens is too large: 32000. This model supports at most 16384 completion tokens. GPT-4o and GPT-4o-mini cap at 16384, many open models at 4096 or 8192.
  • Auto-compaction switches off entirely. Compaction keys off the context limit, so with none declared a long session never compacts — it grows until your endpoint refuses it on context length. Nothing fails until the session is already too big, which is why this is required rather than merely recommended.
Set OPENAI_COMPATIBLE_CONTEXT to the model’s real window. Understating it makes the agent compact earlier than necessary; overstating it defers compaction until your endpoint refuses the request.
Put the key in OPENAI_COMPATIBLE_API_KEY, never inside OPENAI_COMPATIBLE_BASE_URL. The base URL is treated as non-sensitive and is not masked in logs — credentials embedded in it would leak.
On a Router-funded org, a stored openai-compatible/byok default only takes effect if the credentials are saved as Pullfrog secrets (server-visible). If they live solely in your workflow env: / GitHub Actions secrets, the server can’t see them, so a no-card run falls back to the Router default model instead of your endpoint. BYOK orgs are unaffected.
The model ID follows your gateway’s own convention. On Cloudflare AI Gateway’s unified endpoint that’s <provider>/<model> — e.g. workers-ai/@cf/meta/llama-3.1-8b-instruct (docs). On DashScope it’s the bare model ID, e.g. qwen3-coder-plus. On a LiteLLM proxy it’s the model_name you declared in the proxy’s model_list.

​
LiteLLM

A LiteLLM proxy is an OpenAI-compatible endpoint, so it needs no LiteLLM-specific configuration — point the base URL at the proxy’s /v1 path and use a virtual key:
OPENAI_COMPATIBLE_MODEL is the model_name from your proxy’s model_list, not the upstream provider’s ID — LiteLLM’s own routing, failover, and spend tracking then apply as configured. The same shape covers models the proxy fronts from self-hosted vLLM or Ollama.

​
How it works

Pullfrog materializes the endpoint as an @ai-sdk/openai-compatible provider at run time and points it at OPENAI_COMPATIBLE_BASE_URL with your key. The run targets openai-compatible/<OPENAI_COMPATIBLE_MODEL> through the OpenCode agent. Because the model ID is whatever your endpoint serves, there’s nothing to catalog or version — the same single entry covers every OpenAI-compatible gateway and model.