Skip to content
All guides

Customer-support chatbot

Most support questions are answered from a short policy and the conversation so far. A fast, inexpensive model with streaming feels instant to your customers, and you only pay for the tokens each reply uses.

Recommended models

Chosen from models whose abilities we have verified for this job. Cost is for one reply in a 5-message conversation (about 2K tokens in, 250 out), at current prices.

ModelInput / 1MOutput / 1MContextPer request
Our pickDeepSeek V4.1 Flash
deepseek/deepseek-v4.1-flash
$0.375$1.501.05M$0.000938
Also strongGLM 5.3 Flash
z-ai/glm-5.3-flash
$0.1875$0.6251.05M$0.000438
Lowest costgpt-oss-20b
openai/gpt-oss-20b
$0.0625$0.5625128K$0.000234

How it works

  1. 1Write a system prompt that reads like a job description: who the bot works for, how long answers should be, and when to hand over to a human.
  2. 2Send the whole conversation each time (system prompt, then alternating user and assistant messages). The model has no memory between requests.
  3. 3Stream the reply (stream: true) so text appears word by word instead of after a pause.
  4. 4Trim or summarize old messages when conversations get long, so each request stays cheap.

Copy, paste, run

Set AMR_API_KEY to a key from your dashboard, install the OpenAI SDK (pip install openai or npm install openai) and run it. Swap the model for any other in the table.

import os

from openai import OpenAI

client = OpenAI(base_url="https://api.aimodelsrouter.ai/v1", api_key=os.environ["AMR_API_KEY"])
MODEL = "deepseek/deepseek-v4.1-flash"

# The system prompt is your bot's job description. Keep it short and specific.
history = [{
    "role": "system",
    "content": "You are the support assistant for Acme Shop. Answer in 1-3 short sentences. "
               "If you are not sure, say so and offer to connect the customer with a human.",
}]


def reply(message: str) -> str:
    history.append({"role": "user", "content": message})
    answer = ""
    with client.chat.completions.create(model=MODEL, messages=history, stream=True) as stream:
        for chunk in stream:  # tokens arrive as they are generated
            if chunk.choices and chunk.choices[0].delta.content:
                print(chunk.choices[0].delta.content, end="", flush=True)
                answer += chunk.choices[0].delta.content
    print()
    history.append({"role": "assistant", "content": answer})  # remember the conversation
    return answer


reply("Hi, I think I was charged twice for order #1042.")
reply("What should I do next?")

Tested 2026-10-28: both versions were run against the live API with each model recommended at the time, and their answers checked.

Tips

  • Put facts the bot must not get wrong (prices, opening hours, refund policy) in the system prompt rather than relying on the model's general knowledge.
  • Set a per-key credit limit on the key your website uses, so a bug or abuse can never spend more than you planned.
  • Log the conversation ID next to the X-Generation-Id response header to trace any reply in your Activity page.
All 32 models that fit this job