How to Build a Voice Agent from Claude Code with MCP

Yes, you can build a voice agent from Claude Code, and you do not have to write a speech pipeline to do it. You connect Claude Code to a voice agent platform over MCP, and then one chat runs the whole job. Claude writes the agent’s instructions and creates it. You talk to it in your browser. Claude places a real phone call through your SIP trunk, or sends the agent into a Google Meet or Zoom call. When the call ends, Claude pulls the transcript and the rubric scores back into the chat, finds the turn where the call went wrong, and changes one thing in the instructions.

That is the loop: build, test, call, score, refine. Across the teams we work with, the first version of a voice agent is rarely the hard part. The tenth version is, because each one means listening to calls, finding the bad moment, and changing a sentence without breaking the rest. A coding agent is good at exactly that kind of work. This post shows the setup for each client, then walks through the loop with prompts you can paste, what Claude does with each one, and what comes back.

Two ways to get a voice agent out of Claude Code

There are two very different requests you can make. The first is “write me a voice agent.” Claude will happily do it: a speech-to-text service, a language model, a text-to-speech voice, some code for turn-taking, a phone integration, a place to host it. In an afternoon you have something that talks. Then you own it, on every call, including the night the phone provider changes something or the model starts talking over people.

The second request is “use this platform to build me a voice agent.” Here Claude does not write the voice stack at all. It operates a platform where the voice stack already runs, through that platform’s MCP server. Claude writes what only you can provide, which is the agent’s instructions and how a good call is scored, and the platform handles the seven jobs that keep an agent on the phone: the real-time voice loop, a page to test it in the browser, dialing through SIP, joining Google Meet and Zoom, recordings and transcripts, rubric scoring, and versions you can roll back to.

Both are reasonable. If the voice agent is part of your product, with its own business rules and latency budget, building on a framework can be the right call, and I compared the main options in LiveKit vs Vapi vs Retell. If the agent’s job is a conversation your team would otherwise have, like calling demo requests or running a first-round screen, the second path gets you to a real call the same day. I wrote about that decision in more depth in should you build your voice agent or buy one. The rest of this post is the second path.

Set up the voice agent MCP server in Claude Code

Tough Tongue AI runs a hosted MCP server at https://api.toughtongueai.com/api/public/mcp. There is nothing to run locally and no API key to paste. You add the URL, your client opens a browser window, you sign in and approve access, and the client keeps the token. You can see or revoke that access later in Developer settings.

In Claude Code, one command adds the server. Then run /mcp inside a session to finish the sign-in.

claude mcp add --transport http ttai https://api.toughtongueai.com/api/public/mcp

If you also want the open-source skills, which teach Claude how to write a good agent and when to ask before acting, install the plugin instead. It registers the same server and adds the skills in one step.

/plugin marketplace add tough-tongue/toughtongue-skills
/plugin install toughtongue@toughtongue-skills

codex mcp add ttai --url https://api.toughtongueai.com/api/public/mcp
codex mcp login ttai

{
  "mcpServers": {
    "ttai": { "url": "https://api.toughtongueai.com/api/public/mcp" }
  }
}

npx plugins add tough-tongue/toughtongue-skills

That last command installs the skills and the server into each coding agent it finds, including Claude Code, Codex, Cursor, GitHub Copilot CLI, and VS Code. In claude.ai or Claude Desktop, go to Settings > Connectors > Add custom connector and paste the same URL, or add the plugin under Settings > Plugins with the repo URL https://github.com/tough-tongue/toughtongue-skills (plugins need a Pro, Max, Team, or Enterprise plan). In ChatGPT, install the official ToughTongue AI app. For a CI job or a server with no browser, send a personal access token as a bearer header instead of signing in. The agents page has every client in one place.

The first prompt

Start with something boring. It tells you the connection works and gives Claude the context it needs for everything after.

Get me started with Tough Tongue AI. Which workspace am I in, and what does my plan allow?

Claude calls read_guide, which returns the server’s own usage guide, and then get_workspace_info, which returns you, the organizations you belong to, your plan’s limits, and your minute balance. If you work in a team workspace, Claude picks up its organization id here and passes it on every later call. What you see is a short answer: your name, the workspace, whether phone calls are enabled on your plan, and how many minutes you have.

The rest of the tools fall into four groups, and they line up with the loop. Claude reads a schema and the workspace before it writes anything. It builds with create_scenario and update_scenario (a scenario is how Tough Tongue AI stores an agent: its instructions, voice, tools, and rubric). It tests and runs with create_token for a browser link and schedule_bot for calls and meetings. And it scores with list_resources and get_resource, which read sessions with their transcripts, rubric scores, and the fields the agent pulled out of the call.

Two of these deserve care. schedule_bot reaches real people, so the server marks it as an action to confirm, and Claude should state the exact phone number or meeting link and wait for your yes. update_scenario changes the live agent. Both are worth leaving on “ask before running” in your client.

Walkthrough: build an AI phone agent with Claude

Here is a realistic example from the kind of work we see most: an education business with inbound demo requests that go cold because nobody calls them back fast enough. The agent, Maya, calls each new request for a coding bootcamp, finds out when they want to start and what they can spend, and books a call with a human advisor. Each step below is a prompt you can paste, what Claude does with it, and what you get back.

Step 1: Describe the agent

Create a voice agent that calls new demo requests for Northside Code Academy,
a 12-week web development bootcamp. It should confirm they asked for a demo,
qualify budget and start date, and book a call with an advisor on my calendar.
Score each call on whether it qualified both and booked a call.

Claude first calls get_schema("scenario"), which returns the full list of fields an agent can have, and then create_scenario. The instructions it writes look something like this:

You are Maya, an admissions advisor at Northside Code Academy, calling
{{ lead_name }}, who requested a demo of the 12-week web development bootcamp.
Say in your first sentence that you are an AI assistant.

Goals, in order:
1. Confirm they asked for a demo and that now is a good time.
2. Ask about their budget for the program.
3. Ask when they want to start.
4. If they can start within three months, book a call with an advisor.
5. Thank them and end the call.

Keep answers short. Only quote prices from the program page.

And the rest of the scenario, trimmed:

{
  "name": "Bootcamp demo request caller",
  "ai_instructions": "You are Maya, an admissions advisor at ...",
  "rubrik": "Score each from 1 to 10 and quote the line behind it: confirmed interest and timing; qualified start date; qualified budget; booked an advisor call.",
  "session_analysis": {
    "is_auto_analysis": true,
    "enable_extraction": true,
    "extraction_vars": [
      { "name": "start_date", "description": "When the lead wants to start", "type": "date" },
      { "name": "budget_range", "description": "The budget the lead gave, in their words", "type": "text" },
      { "name": "call_booked", "description": "Whether an advisor call was booked", "type": "boolean" }
    ]
  },
  "strategy": { "max_duration_seconds": 480 },
  "tools_config": {
    "tools": { "end_session": { "should_register": true, "add_to_system_prompt": true } }
  },
  "custom_function_ids": ["<your booking function>"]
}

A few of those fields are worth knowing. {{ lead_name }} is a variable filled in per call, so one agent can greet every lead by name. The rubric is plain text, and it is what every call gets scored against. The extraction fields are pulled out of each transcript after the call, so call_booked and start_date land as data you can filter on later. The tools_config matters more than it looks: if Claude leaves it out, the agent gets a broad default set of tools, which makes sense for a browser session and not for a phone call. Claude replies with the new agent’s id and a summary of what it set.

The calendar part needs one honest note. There is no built-in calendar tool. Booking works through a custom function, an HTTP call to your booking system that you set up once in the web app, or through an MCP server attached to the agent. Claude can link a function you already have by its id, but it cannot create one over MCP.

Step 2: Read what it wrote

This is the step people skip, and it is the one that decides how the first call goes. Claude writes instructions that read well on a page. Spoken out loud, a numbered list of goals turns into an agent that marches through questions. Look at goal 2 above. On a call, it comes out like this:

Maya: Great, thanks for confirming. What’s your budget for the program?

Priya: Um, I just wanted to see the curriculum first.

Nothing in the instructions is wrong. The order is wrong for a person who filled in a form five minutes ago and has not decided anything yet. You could fix it now, but I would leave it and run the test. A real call shows you the problem with a timestamp on it, and that makes for a much smaller, more confident edit than guessing.

Step 3: Test it in the browser

Give me a link to try Maya in my browser. I'll play a lead named Priya.

Claude calls create_token with the agent’s id and gets back a link that opens the agent in an embedded page, valid for anywhere from 1 to 168 hours. Anyone holding that link can talk to the agent without signing in, so Claude should ask before it makes one. You can also open the agent from your library in the web app.

Talk to it the way a real lead would, not the way you hope one will. Interrupt it mid-sentence. Ask for the price before it asks you anything. Say “I’m just looking.” Go quiet for a while. Five minutes of this catches most problems that would otherwise reach a real person, and it costs a few minutes of your plan instead of a lead.

Step 4: Place a real call, or send it into a meeting

Once the browser test holds up, put it on a real phone line. The first call should be to your own phone.

Call me at +1 415 555 0142 as a test lead named Priya. Use my outbound trunk.

Claude calls list_resources for your SIP trunks, picks the outbound one, and tells you exactly what it is about to do: this agent, this number, this trunk, now. When you say yes, it calls schedule_bot with kind: "phone", your number in E.164 format, and {"lead_name": "Priya"} as a variable. It gets back a call id, and your phone rings. This is a Claude Code phone agent in the plain sense: the call starts from your terminal and runs on the platform.

The same tool scales past one number. Give it a list of entries and it becomes a batch the dialer works through, with office hours in the lead’s time zone and up to five retries with delays you set. Each entry can carry its own variables, so every lead hears their own name and the program they asked about.

If the conversation belongs in a meeting instead, the prompt barely changes:

Send Maya to https://meet.google.com/abc-defg-hij now as "Maya (AI admissions)".

That is schedule_bot with kind: "meeting", a Google Meet or Zoom link, and a bot name that says it is an AI. The agent joins as a named participant and talks, and Claude gets back a bot id it can follow to the session once the call ends. I covered how meeting bots get into private calls, and what changes on Zoom, in build a voice AI agent that joins Google Meet and Zoom. If the agent is for sales meetings specifically, how to build an AI meeting agent covers the context it needs.

Step 5: Pull the transcript and scores into the chat

Pull the transcript and scores from that call. Where did Maya lose Priya?

Claude reads the call with get_resource to find its session id, then reads the session with the transcript, the rubric evaluation, and the extracted fields included. If the analysis has not run yet, it can start it with analyze_session and wait. Then it answers in the chat, and this is where the loop starts to pay off, because the answer points at a moment instead of a vibe.

In our example, the first call scores 8 out of 10 on confirming interest and timing, 7 on qualifying the start date, 3 on qualifying the budget, and 2 on booking an advisor call. Claude quotes the line behind the 3. At 0:41, before Maya knew anything about why Priya was looking, she asked about budget, and Priya pulled back. Everything after that was Maya trying to recover.

Step 6: Refine the instructions

This is the part of the loop that makes the agent good, and it is where how you ask matters most. Compare two requests:

Weak: Make Maya better at qualifying.

Strong: At 0:41 Maya asked about budget before any context and Priya pulled back. Change only the order of the qualifying questions so she asks about goals first and budget last, with a reason. Keep everything else. Save the current version first.

The weak request invites Claude to rewrite the whole prompt, and you lose track of what changed and why the next call is different. The strong one names the evidence, limits the change, and asks for a version you can go back to. With it, Claude calls update_scenario with only the changed instructions and a version label, which archives the current agent before applying the edit. The change it proposes is small:

- 2. Ask about their budget for the program.
- 3. Ask when they want to start.
+ 2. Ask what made them look at bootcamps now, and what they want to be
+    doing in a year. Listen for a job goal.
+ 3. Ask when they want to start, and tie it to the goal they gave.
+ 4. Only then ask about budget, with a reason: "So I point you to the
+    right plan, is there a range you had in mind?"
+ If they say they only want the curriculum, offer to send it, then ask
+ one goal question before moving on.

On the next call, the same moment sounds like this:

Maya: Happy to send the curriculum. Before I do, what got you looking at bootcamps now?

Priya: I’m in support at a software company. I’d like to be in a junior developer role by next summer.

Maya: That’s a common path for our students. So I point you to the right cohort, is there a budget range you had in mind?

Then you go around again. Call yourself, or call a few friendly leads, and ask Claude to read the new sessions. Because every version is saved, you can run a fair comparison: schedule_bot takes a specific saved version, so you can call five leads on the old version and five on the new one. After a week, a prompt like “Compare the scores on this agent before and after the budget change” reads the sessions in bulk and tells you whether the edit held. If it did not, rolling back is one more request.

Two habits make this loop work. Change one thing per pass, so you know what moved the score. And keep the rubric stable while you tune the instructions, or you will not know whether the agent got better or the grading got easier. Edits apply to new sessions only, so a call that is already running keeps the old instructions.

How other voice agent MCP servers compare

Tough Tongue AI is not the only voice platform with an MCP server. ElevenLabs, Vapi, and Retell each run one, and they expose quite different tools. Here is what each one’s own documentation says, as of November 2026. Check the linked pages before you rely on any of it, since these servers change often.

When you compare MCP servers for voice AI, three questions matter more than the tool count. Can the server reach the channel your agent works in: the phone, a meeting, or a page in your app? Do results come back into the chat in a form Claude can act on, like scores with quotes and timestamps rather than only a recording link? And how does sign-in work for unattended jobs? Vapi’s own FAQ, for example, sends headless automation to its REST API with an API key, because OAuth needs a person at a browser.

Honest limits

Building from a chat removes a lot of clicking. It does not remove the parts of running a voice agent that were never about clicking, and it is worth being clear about those before your first real call.

You still need a phone line. Real calls go out through a phone line on your account: your own SIP trunk from a provider like Twilio, Plivo, Exotel, Telnyx, or Vonage, or a number you get in the app. Inbound calls are answered on your own numbers the same way. That setup happens once in the web app. Claude can read your trunks over MCP and pick the right one, but it cannot create one.

Minutes apply. Connecting the MCP server is free on every plan, including the free one with 25 one-time minutes. Every test, call, and meeting spends minutes like any other session. Phone calls need a SIP-enabled plan and balance, and meeting bots need a paid plan. At the time of writing, Starter is $20 a month with 150 minutes and includes Google Meet and Zoom deployment, and Pro is $99 a month with 900 minutes that roll over.

A coding agent can write a weak prompt. Claude is good at structure and less good at how a sentence sounds on a phone. Its first drafts tend to be long, list-shaped, and polite in a way that makes calls stiff. Read every set of instructions it writes before the agent talks to anyone, and keep spoken turns to one or two sentences.

Some pieces live outside the chat. Booking on a calendar needs a custom function or an attached MCP server, and neither can be created over MCP. You set it up once, and then Claude can link it to any agent. The meeting bot joins Google Meet and Zoom today, and Microsoft Teams for the bot comes later.

The agent reaches real people. schedule_bot dials real phones and joins real meetings. The server tells Claude to confirm the exact number or link before acting, and the skills do the same, but your client’s approval prompts are the last check. Do not let Claude dial a list it inferred from a spreadsheet you did not review, and do not turn off confirmation to save a click.

The loop is the work

Most of the effort in a good voice agent goes into the loop, not the build. Somebody has to listen to calls, find the moment that went wrong, make one careful change, and check that it helped without breaking something else. That used to mean a person with a dashboard open and a lot of patience. Now it can mean a chat where you point at the evidence and Claude does the reading and the typing.

If you already work in Claude Code, Codex, or Cursor, the fastest way to see this is to connect the server, build one agent for a conversation your team has every week, and call yourself with it. You will know after a few passes around the loop whether the agent is good enough to call a real lead. Tough Tongue AI is free to start, and if you want to talk through your use case first, you can book 15 minutes with me.