Build an AI Chatbot

Every team is building with AI right now, which makes "I built a chatbot" one of the highest-signal things you can put in a portfolio — if you can explain how it works. A chatbot that streams responses looks like magic and is genuinely useful, but the interesting engineering isn't the model. It's the boundary between your browser and a paid API, the mechanics of streaming over HTTP, and the handful of guardrails that separate a weekend demo from something you'd let a real user touch.

This walkthrough builds that. It assumes you've done something like the Weather App from the project library — you've called an API, awaited a response, and rendered it. If async code and fetch feel familiar, you're ready.

What you're building

A single-page chat app. A user types a message, hits send, and watches the assistant's reply appear word by word — the way ChatGPT and Claude do it. Under the hood:

• A React front end holds the conversation in state and renders the streaming reply.
• A server route (your backend) holds your API key, forwards the conversation to an LLM, and pipes the streamed answer back to the browser.
• The two talk over a streaming HTTP response, not a single JSON blob.

Stack: TypeScript and React (via Next.js, so your UI and your API route live in one project), and an LLM API — this guide uses Claude via the Anthropic SDK, but the shape is identical for any provider. You'll deploy it at the end so it has a real URL.

Milestone 0 — Scaffold and the boundary that shapes everything

Start a Next.js app with TypeScript:

npx create-next-app@latest ai-chatbot --typescript --app
cd ai-chatbot
npm install @anthropic-ai/sdk

Now get an API key from your provider's console and put it in a .env.local file:

# .env.local
ANTHROPIC_API_KEY=sk-ant-...

Why this is Milestone 0 and not a footnote: the single most important architectural decision in this project is that the browser never sees your API key. If the browser called the LLM directly, anyone could open dev tools, copy your key, and spend your money. So every request flows: browser → your server → LLM → your server → browser. Your server is the trusted middleman. Hold onto that mental model — the next three milestones are just filling it in.

Milestone 1 — A server route that answers once

Before streaming, get a plain request/response working. Create an API route at app/api/chat/route.ts:

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic(); // reads ANTHROPIC_API_KEY from the environment

export async function POST(req: Request) {
  const { messages } = await req.json();

  const response = await client.messages.create({
    model: "claude-sonnet-5",
    max_tokens: 1024,
    system: "You are a concise, friendly assistant.",
    messages,
  });

  const text = response.content[0].type === "text" ? response.content[0].text : "";
  return Response.json({ text });
}

Test it without a UI at all — that's a debugging habit worth building now:

curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Say hi in five words."}]}'

What to understand before moving on:

• messages is the whole conversation, not just the latest turn. LLMs are stateless — the model remembers nothing between calls. "Memory" is an illusion you create by sending the entire history every time. That array of { role, content } objects is the conversation. Internalize this; it explains almost every "why did the bot forget?" bug you'll ever hit.
• system is separate from the messages. The system prompt sets behavior and rules; the messages are the dialogue. Keeping them separate is deliberate — users supply messages, you supply the system prompt, and mixing them is how prompt-injection bugs start.
• max_tokens caps the reply length — and every token costs money. This is a budget lever, not a formality.

Milestone 2 — Make it stream

A two-second wait staring at nothing feels broken. Streaming fixes that: the model emits tokens as it generates them, and you forward each one to the browser the instant it arrives. Rewrite the route to stream:

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

export async function POST(req: Request) {
  const { messages } = await req.json();

  const stream = new ReadableStream({
    async start(controller) {
      const llmStream = await client.messages.stream({
        model: "claude-sonnet-5",
        max_tokens: 1024,
        system: "You are a concise, friendly assistant.",
        messages,
      });

      for await (const event of llmStream) {
        if (
          event.type === "content_block_delta" &&
          event.delta.type === "text_delta"
        ) {
          controller.enqueue(new TextEncoder().encode(event.delta.text));
        }
      }
      controller.close();
    },
  });

  return new Response(stream, {
    headers: { "Content-Type": "text/plain; charset=utf-8" },
  });
}

This is the milestone people copy-paste and never understand. Don't. Here's the whole thing in plain terms:

• A ReadableStream is a pipe your server writes into and the browser reads out of, chunk by chunk, before the response is "finished." A normal Response.json() sends everything at once; a stream sends it in pieces.
• The for await loop consumes events from the LLM as they arrive. The model doesn't hand you a finished paragraph — it hands you a sequence of tiny text_delta events, each a few characters.
• controller.enqueue(...) pushes each chunk into the pipe. TextEncoder turns the string into the bytes an HTTP stream carries.
• controller.close() signals "done" so the browser knows the reply is complete.

Milestone 3 — The chat UI

Now the front end. A chat page needs three pieces of state: the message history, the current input, and the reply-in-progress. Create app/page.tsx:

"use client";

import { useState } from "react";

interface Message {
  role: "user" | "assistant";
  content: string;
}

export default function Chat() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState("");

  async function send() {
    if (!input.trim()) return;

    const next: Message[] = [...messages, { role: "user", content: input }];
    setMessages(next);
    setInput("");

    const res = await fetch("/api/chat", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ messages: next }),
    });

    if (!res.ok || !res.body) {
      setMessages([...next, { role: "assistant", content: "Something went wrong. Try again." }]);
      return;
    }

    // Add an empty assistant message, then fill it as chunks arrive.
    setMessages([...next, { role: "assistant", content: "" }]);
    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let acc = "";

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      acc += decoder.decode(value, { stream: true });
      setMessages([...next, { role: "assistant", content: acc }]);
    }
  }

  return (
    <main style={{ maxWidth: 640, margin: "0 auto", padding: 24 }}>
      {messages.map((m, i) => (
        <p key={i}><strong>{m.role}:</strong> {m.content}</p>
      ))}
      <input
        value={input}
        onChange={(e) => setInput(e.target.value)}
        onKeyDown={(e) => e.key === "Enter" && send()}
        placeholder="Ask something…"
      />
      <button onClick={send}>Send</button>
    </main>
  );
}

The one line that trips everyone up is sending next, not messages, to the API. React state updates are asynchronous — right after setMessages(next), the messages variable in this function still holds the old value. If you send messages, you drop the user's newest turn and the bot replies to the wrong thing. Understanding why — closures capture the value at render time — is worth more than the fix itself, because this bug reappears in every React app you build.

The read loop mirrors the server exactly: pull a chunk, decode it, append it, re-render. The reply grows on screen because you call setMessages on every chunk with the accumulated text.

Milestone 4 — Make it real

A demo becomes a product at the edges. Add these one at a time, and make sure you can explain what each one defends against:

• Input validation. Never trust the request body. Validate messages on the server (a schema library like Zod is the standard) before touching the LLM. Reject empty arrays, cap the message count and length. Defends against: malformed requests, and users pasting a novel to run up your bill.
• Error and loading states. A disabled send button while a reply streams; a visible error when the route fails. Defends against: double-sends and the "is it broken or just slow?" confusion.
• Rate limiting. Cap requests per user/IP per minute. Defends against: one script turning your API budget into a crater. This is the difference between a toy and something with a public URL.
• Cost awareness. Log token usage per request. Know what a conversation costs before a surprise invoice teaches you. Defends against: the most common way hobby AI projects die.

Milestone 5 — Ship it

A project without a URL is invisible. Deploy to Vercel (the zero-config path for Next.js) or any host that runs Node:

1. Push the repo to GitHub.
2. Import it into your host and set ANTHROPIC_API_KEY as an environment variable in the host's dashboard — not in the repo.
3. Deploy, open the live URL, and send a message.

Confirm the key is a server-side env var and never shipped to the browser: open dev tools → Network → your /api/chat request. You should see the conversation going out and text streaming back — and no key anywhere. That check is the whole security model of this app, made visible.

Stuck? Study a production build

When something won't click, reading real code helps more than another tutorial. These are worth studying — after you've built your own version, so you can see what they added and why, not just copy their answer:

• vercel/ai-chatbot — a full production Next.js chatbot. Read its route handlers and streaming setup.
• Vercel AI SDK — the streaming utilities that hide the ReadableStream plumbing you just wrote by hand. Now that you've done it manually, the abstraction will make sense instead of hiding the mechanics.
• Anthropic TypeScript SDK — the streaming examples map straight onto Milestone 2.

The order matters: build first, read second. Reading a finished implementation before you've struggled with the problem teaches you what the code is, not why it's that way.

Build it with Cartara

Here's the honest part. You could get a chatbot working by pasting each block above and never understanding the ReadableStream, the next-vs-messages bug, or why the key stays on the server. It would run. You'd also be unable to change it, debug it, or answer a single follow-up question about it — and increasingly, an AI assistant will have written large parts of it for you, which makes that gap invisible until it bites.

That gap is exactly what Cartara exists to close. As you build:

• Point Cartara at your repo. It watches the code that lands — including anything an AI assistant generated — and turns each meaningful change into a checkpoint for understanding, not just a commit.
• Explain your changes back. When you (or your AI pair) add streaming or rate limiting, Cartara prompts you on the reasoning and trade-offs — the same questions a reviewer or interviewer would ask.
• See your comprehension, not just your output. You get a real signal on which parts of your own project you actually understand versus which you shipped on faith — so you can shore up the weak spots before they show up in a review.

That's the loop this whole library is built around: build something real, then prove to yourself you understand it. The chatbot is the project. The understanding is the point.