AI Gateway

In plain words

AI models from several companies through one account. Your app asks a question and gets an answer. What it uses is paid from your balance, once any AI your plan includes is used up.

For example: A shop turns a few bullet points into a product description.

Chat

curl https://harakumo.com/api/ai/chat \
  -H "Authorization: Bearer $HARAKUMO_API_KEY" -H "content-type: application/json" \
  -d '{ "model": "harakumo-1", "system": "Answer in one sentence.",
        "messages": [{ "role": "user", "content": "What is an edge function?" }], "max_tokens": 300 }'
# → { "output", "modelUsed", "provider", "finishReason", "truncated", "tokensIn", "tokensOut", "cost", "latencyMs" }

const res = await fetch('https://harakumo.com/api/ai/chat', {
  method: 'POST',
  headers: { authorization: 'Bearer ' + process.env.HARAKUMO_API_KEY, 'content-type': 'application/json' },
  body: JSON.stringify({ model: 'claude-sonnet-5', stream: true, messages: [{ role: 'user', content: 'Tell me a story.' }] }),
});
// Events: "meta" (the model answering), "delta" ({ text }), then "done"
// ({ finishReason, truncated, tokensIn, tokensOut, cost }) or "error".
for await (const chunk of res.body) process.stdout.write(new TextDecoder().decode(chunk));

const answer = await hk.ai.chat({
  model: 'harakumo-1', system: 'Answer in one sentence.',
  messages: [{ role: 'user', content: 'What is an edge function?' }],
});
console.log(answer.output, answer.modelUsed, answer.cost);

for await (const ev of hk.ai.stream({ prompt: 'Tell me a story.' })) {
  if (ev.event === 'delta') process.stdout.write(ev.data.text);
  if (ev.event === 'done') console.log('\n', ev.data.cost);
}

OpenAI-compatible

Point any OpenAI client at https://harakumo.com/api/ai/v1 with a Harakumo key. The request and response shapes are the same (streaming included), model is any Harakumo model id, and each reply carries a harakumo field with the provider, cost and any fallback taken. Refusals come in OpenAI's error shape, { "error": { "message", "type", "code" } }, with the codes below.

import OpenAI from 'openai';

const ai = new OpenAI({ apiKey: process.env.HARAKUMO_API_KEY, baseURL: 'https://harakumo.com/api/ai/v1' });
const r = await ai.chat.completions.create({ model: 'harakumo-1', messages: [{ role: 'user', content: 'Hi' }] });
console.log(r.choices[0].message.content);

Request fields

An answer that stopped at max_tokens has truncated: true and a note on how to get the rest.

Embeddings

POST /api/ai/embeddings { input, model } returns vectors from bge-small (384 dimensions), bge-base (768, the default) or bge-large (1,024), up to 100 inputs per call. They pair with Vector DB collections of the same size.

Price, limits and logs

• Hobby, Pro, Team and Scale include no AI usage: every request is pay as you go from the Balance (on a paid plan, the Balance of the workspace that pays for the plan, for every workspace it covers, under that workspace's spending limit; on Hobby, the workspace's own), at each model's per-token price: the model maker's list price, plus up to 5% on third-party models to cover buying that usage (edge models never carry it). On Hobby, vendor models need the owner's verified email. Workspaces that already have included AI keep it ($10 a month on a Pro subscription from before these plans and $100 a month on Enterprise), one amount per subscription shared by the workspaces it covers, used before the Balance. Team's $20 monthly usage credit pays for use past the plan, not for AI or domains, and what is not used does not carry over. GET /api/ai/chat lists the exact price of every model.
• Rate limit per workspace: 60 a minute on hobby, 600 a minute on pro, 2,000 a minute on team, 6,000 a minute on scale; chat and embeddings have separate buckets. Over it: 429 ai_rate_limited with Retry-After.
• Input up to 200,000 characters per request (413 over it).
• GET /api/ai/requests lists recent calls (model, tokens, cost, speed — never the prompt or answer), kept 30 days, with this month's gateway spend by model and project. Billing → Balance has every AI charge this month by model, embeddings, memory and vectors included.
• A key limited to one project calls chat, the OpenAI-compatible URL, the playground, embeddings and the model list for its own project: leave projectId out, or name that project (naming another is refused). The spend is the workspace's, as with any key, and a read-only key cannot spend. GET /api/ai/requests covers the whole workspace and needs a workspace key.

How a request is paid for

Before the model is called, Harakumo sets an estimate aside, sized from your input and max_tokens (for harakumo-1, at the price of the dearest model its tier may use). When the answer is back you are charged the exact cost of the model that answered, and the rest is released. The dashboard playground shows that estimate before you press Run.

• A stream that ends early, because you disconnected or the model stopped, is charged only for what was delivered; a failure arrives as an error event with code stream_error.
• Auto-reload, when the owner has turned it on, is tried before a 402 for credit. Requests made with an API key set it off only if the owner allowed that (Billing → Limits & alerts).
• If a request is lost on Harakumo's side before it is settled, the estimate set aside for it is settled automatically within about half an hour: at what it had delivered, or given back.

When a request is refused

A refused or failed request costs nothing, with one exception: a model that spends its whole token budget without answering has used those tokens, and they are charged.

Reference

const answer = await hk.ai.chat({ model: 'harakumo-1', prompt: 'Summarize edge functions.' });
console.log(answer.output, answer.modelUsed, answer.cost);

for await (const ev of hk.ai.stream({ messages: [{ role: 'user', content: 'Write a haiku' }] })) {
  if (ev.event === 'delta') process.stdout.write(ev.data.text);
}

const { models, limits } = await hk.ai.models();                  // what this workspace can use, with prices
const { requests, month } = await hk.ai.requests({ limit: 20 });  // recent calls and this month's spend

// Embeddings for Vector DB (bge-small 384 · bge-base 768 · bge-large 1024):
const { embeddings, dimensions } = await hk.ai.embed(['fresh mango', 'ripe banana'], 'bge-base');
// hk.ai.generate({ prompt }) is the older name of a one-prompt chat.

harakumo ai "Write a haiku about the cloud" --model harakumo-1

POST /api/ai/chat                   { "model", "messages" | "prompt", "system"?, "max_tokens"?, "temperature"?, "tier"?, "stream"?, "projectId"? }
GET  /api/ai/chat                   # the models and limits this workspace can use
POST /api/ai/v1/chat/completions    # OpenAI-compatible (base URL https://harakumo.com/api/ai/v1)
POST /api/ai/playground             { "model", "prompt" }          # the one-shot alias
POST /api/ai/embeddings             { "input": ["…"], "model"?: "bge-base", "projectId"? }   → { embeddings, dimensions }
GET  /api/ai/requests?limit=&projectId=    # recent calls and this month's spend