Build a Multi-Agent Research Assistant

Lesson 1

The bare LLM loop

You call it in a loop, here's where that breaks

The bare LLM loop, and where it breaks

This is Lesson 1 of nine. Today you build the foundation, a single ResearchAgent that calls an LLM and returns an answer. It takes about 20 lines. Along the way you will discover exactly what that agent cannot do, and that discovery will motivate every lesson that follows.

See it first: the smallest possible agent

Agent v0.1

Can do

✓ Generate language

Cannot do

✗ Search✗ Memory✗ Planning✗ Verification

ResearchAgent ≠ Research, yet. It only writes text.

An LLM answers once. An agent puts that call in a loop and decides what to do next: observe, decide, act, repeat. Today's version is the smallest one that runs.

Look at the right-hand panel. It can only generate text, no search, no memory, no planning, no way to check itself. That gap is the whole story of this course.

One interface that survives all nine lessons

One interface, any model

The agent depends on an interface, not a model. Swap what's below the line, the agent never changes.

The agent only ever calls llm.chat(messages), so the labs run on a deterministic MockLLM (offline, no API key) and a real model swaps in with one line. You learn the agent's plumbing, not prompt-wrangling.

Now watch it fail

The wall

The bare loop has nothing to check reality against, so it invents a number that sounds right.

Ask for a specific fact and the bare loop answers confidently, and wrongly. It generates plausible text, not verified facts, and it cannot tell the two apart.

The fix: ground it in reality

Why tools exist

Same question. The right-hand agent grounds itself in retrieved sources before it answers. That is Lesson 2.

Give the agent a tool and the loop changes shape: search first, read real sources, then answer. That single change is the entire motivation for Lesson 2.

One capability at a time

Where this is going

You are at L1. Each step adds one capability the step below it lacked.

The interface we define today

Every LLM in this course, mock or real, will share one interface:

Python

class LLMClient:
    def chat(self, messages: list[dict]) -> str:
        raise NotImplementedError

ResearchAgent only ever calls llm.chat(messages). It never knows whether it is talking to a mock, OpenAI, or Anthropic. This is the decision that makes the entire course work: swap the LLM without touching the agent.

Why we start with a mock

A real LLM requires an API key, costs money per call, and has network latency. MockLLM is scripted, deterministic, and runs offline, the interface is identical to a real client. When you are ready to wire in a real model it is one line of code.

The mock is also useful for teaching: the failure we are about to discover is guaranteed, not probabilistic.

Your task

MockLLM is given, it is infrastructure, not a learning target. Your job is to build ResearchAgent: fill in run() so it calls self.llm.chat(messages) with a system message and the user's query, then returns the response. Run the cell when it prints a real answer.

Python

# ── The shared interface ──────────────────────────────────────────────────────
class LLMClient:
  def chat(self, messages: list) -> str:
      raise NotImplementedError

# ── MockLLM (given, read it, do not modify) ───────────────────────────────────
class MockLLM(LLMClient):
  RESPONSES = {
      "reinforcement learning": (
          "Reinforcement learning is a paradigm where an agent learns by interacting "
          "with an environment. It receives rewards for good actions and penalties for "
          "bad ones, gradually learning a policy that maximises cumulative reward. "
          "Core algorithms include Q-learning, SARSA, and PPO."
      ),
      "alphacode humaneval": (
          "The AlphaCode paper (DeepMind, 2022) reported a pass@1 of 45.7% on HumanEval "
          "for Python, significantly outperforming GPT-3 and Codex at the time of release."
      ),
  }

def chat(self, messages: list) -> str:
      query = messages[-1]["content"].lower()
      for key, response in self.RESPONSES.items():
          if all(word in query for word in key.split()):
              return response
      return (
          f"Based on my training, here is a thorough answer to your question about "
          f"'{messages[-1]['content'][:50]}': [detailed but unverifiable response]"
      )

# ── Your turn: build the ResearchAgent ────────────────────────────────────────
class ResearchAgent:
  def __init__(self, llm: LLMClient):
      self.llm = llm

def run(self, query: str) -> str:
      # Build the messages list:
      #   {"role": "system", "content": "You are a research assistant. Answer accurately."}
      #   {"role": "user",   "content": query}
      # Call self.llm.chat(messages) and return the result.
      pass  # ← replace this

# ── First test ─────────────────────────────────────────────────────────────────
llm   = MockLLM()
agent = ResearchAgent(llm=llm)

result = agent.run("What is reinforcement learning?")
print(result)

First run loads the Python runtime (~10 MB) — takes ~5–10 seconds. Subsequent runs are instant.

Show solutiontry it yourself first

Build the messages list (a system role + the user's query) and pass it to self.llm.chat:

Python

def run(self, query: str) -> str:
    messages = [
        {"role": "system", "content": "You are a research assistant. Answer accurately."},
        {"role": "user",   "content": query},
    ]
    return self.llm.chat(messages)

Expected output for agent.run("What is reinforcement learning?"):

Text

Reinforcement learning is a paradigm where an agent learns by interacting with an
environment. It receives rewards for good actions and penalties for bad ones,
gradually learning a policy that maximises cumulative reward. Core algorithms
include Q-learning, SARSA, and PPO.

Got a real answer? The agent works, for this query.

Now try something specific

The wall

Your agent returned a confident answer. Research work deals in specifics. Run the cell below, then check whether the answer is actually correct before scrolling further.

Python

# Self-contained, paste your completed run() method into ResearchAgent below

class LLMClient:
  def chat(self, messages: list) -> str:
      raise NotImplementedError

class MockLLM(LLMClient):
  RESPONSES = {
      "alphacode humaneval": (
          "The AlphaCode paper (DeepMind, 2022) reported a pass@1 of 45.7% on HumanEval "
          "for Python, significantly outperforming GPT-3 and Codex at the time of release."
      ),
  }
  def chat(self, messages: list) -> str:
      query = messages[-1]["content"].lower()
      for key, response in self.RESPONSES.items():
          if all(word in query for word in key.split()):
              return response
      return "Based on my training: [confident but unverifiable answer]"

class ResearchAgent:
  def __init__(self, llm: LLMClient):
      self.llm = llm
  def run(self, query: str) -> str:
      messages = [
          {"role": "system", "content": "You are a research assistant. Answer accurately."},
          {"role": "user",   "content": query},
      ]
      return self.llm.chat(messages)

agent = ResearchAgent(llm=MockLLM())

# ── The wall test ──────────────────────────────────────────────────────────────
answer = agent.run(
  "What did the AlphaCode paper report for HumanEval pass@1 on Python?"
)
print("Agent says:")
print(answer)
print()

# ── What the paper actually says ───────────────────────────────────────────────
print("Reality check:")
print("  AlphaCode (DeepMind, 2022) was benchmarked on CodeContests, not HumanEval.")
print("  The 45.7% figure is plausible-sounding but fabricated.")
print("  The agent cannot distinguish what it knows from what it invented.")

First run loads the Python runtime (~10 MB) — takes ~5–10 seconds. Subsequent runs are instant.

Why did that happen?

The agent answered with complete confidence. It was completely wrong.

An LLM does not look up facts. It generates the most statistically plausible continuation of your prompt. When you asked about AlphaCode and HumanEval in the same sentence, the model produced text that looks like an answer to that question, because answers to that kind of question are in its training data. Plausible ≠ accurate.

The fundamental constraint

An LLM has no mechanism to retrieve real-world information, no ability to flag uncertainty about specific figures, and no way to verify its own output. For general conceptual questions it often seems fine. For specific, verifiable claims, exact benchmarks, paper results, recent events, it fails silently and confidently.

This is not a bug to fix with a better prompt. It is a property of how language models work. The fix is giving the agent tools, functions it can call to fetch real information at query time.

What you have. What's missing.

Built today	Missing, coming next
`LLMClient` interface (swappable)	Tools (search, retrieval)
`MockLLM` with scripted responses	Memory / persistent state
`ResearchAgent.run()`	Planning and decomposition
Input → LLM → Output	Quality verification

The interface doesn't change. The ResearchAgent class you wrote today will grow, one capability per lesson.

Practice it yourself

Build the agent's plumbing — the messages, the validation, the mock router. Real Python, hidden tests, no setup.

Build the messages listEasy

System + user — the shape every LLM call shares.

Validate a chat messageEasy

Known role + string content — catch malformed messages early.

MockLLM keyword routingMedium

Match all keywords or fall back — the deterministic stand-in.

The swap (one line, when you're ready)

Python

# Lessons 1-3: mock, offline, no key needed
agent = ResearchAgent(llm=MockLLM())
 
# When ready, this is the only line that changes
# from anthropic import Anthropic
# agent = ResearchAgent(llm=AnthropicClient(Anthropic(api_key="sk-...")))

Nothing in ResearchAgent changes. That is the point.

Quiz

Check your understanding

1 / 6

What distinguishes an 'agent' from a single LLM call?

Practice it yourself

The agent's loop lives or dies on parsing the LLM's text back into structure. Warm up on the exact parsers later lessons rely on, in the browser with hidden tests, no setup:

Critic verdict parserEasy

Turn an LLM's free-text verdict into a structured PASS/FAIL, the skill the Critic lesson needs.

Planner output parserEasy

Parse a model's numbered plan into a clean list of steps, the Planner lesson's core move.