All Stack Labs blog

Before a voice agent answers your phones

Mon, 12 Oct 2026 12:00:00 +0000

All Stack Labs, 12 October 2026

Many calls to businesses never reach a person. Invoca, which tracks calls for its business customers, found that only 56% of callers reach a person. Quo, a business phone provider, looked at 16.7 million missed calls and found that about 69% did not get a callback within 48 hours.

A voice agent closes that gap. It picks up every call, answers questions, books the appointment and writes it all back to your systems. Getting one to do that reliably, on real calls, is harder than the demos suggest.

Fast enough, and patient enough

People are fast. A study of conversations in ten languages found the average gap before an answer to a yes or no question was about 200 milliseconds. No phone agent matches that yet. Twilio publishes a target of about 1.1 seconds end to end, and calls it a starting benchmark rather than the best possible.

Speed is only half of turn-taking. The agent also has to know when the caller has finished. LiveKit describes an older model that decided a caller was done after each digit of a phone number. Its newer model cut false interruptions overall by 39%. An agent that interrupts a caller reading out a card number makes them start again.

Where voice agents fail

Sierra, which builds customer service agents, tested voice agents on real tasks with realistic audio: background noise, varied accents and callers who talk over the agent. By April 2026, the best agent on its leaderboard completed 67% of tasks. Their example failure will be familiar to anyone who has run a phone line: street noise and an unfamiliar accent throw off the transcription, and the identity check fails.

The quiet failures are worse, because they look like success. Roark, which tests voice agents, describes a scheduling agent that checked availability in UTC instead of the caller's local time. As they put it, "the agent sounded fine, the tool call succeeded, and the appointment was wrong." Common causes:

Time zones. The caller, the server and the calendar can each be in a different one.
Stale availability. The agent offers a slot someone booked a minute ago.
Invented answers. A price or a policy the agent was never given, said with confidence.
No way out. A caller who needs a person and cannot reach one.
Half-written records. The call went well, and nothing reached the CRM.

The rules that apply

This is not legal advice, and the details depend on your state and your industry. These are the questions to settle with your lawyer before launch:

Calling out is different from picking up. In February 2024 the FCC ruled that AI-generated voices count as artificial voices under the Telephone Consumer Protection Act. AI calls to consumers need prior express consent, and written consent for telemarketing. An AI callback to a missed caller is an outbound call.
Recording needs consent. Federal law needs one party to agree, but around a dozen states, including California, Florida, Illinois, Pennsylvania and Washington, require everyone on the call to agree. Most voice agents record and transcribe, so say so at the start.
Some states want the AI disclosed. Since January 2025, California has required health care providers that use AI to generate patient messages about clinical information to include a disclaimer, spoken at the start and the end of a call, along with how to reach a person. It does not cover scheduling or billing, or messages a licensed provider has reviewed. Utah requires disclosure when a consumer asks, and Texas requires it of state agencies and health care providers.
Health data needs a BAA. If the agent handles patient information, every vendor that stores it, recordings and transcripts included, is a business associate under HIPAA and needs a business associate agreement.

Test it before it takes a real call

A voice agent should pass a test suite before it reaches a customer, and keep passing it after every change. OpenAI's guide to evaluating real-time voice agents sets out the stages: scripted single turns first, then real noisy phone audio, then full simulated calls against mocked tools. Every failure in production becomes a new test. And keep a person listening: the same guide suggests reviewing 1 to 5% of sessions end to end.

Run each scenario more than once. Sierra's earlier research measured whether an agent succeeds on every repeat of the same task, and found reliability falls sharply as the repeats add up. A caller does not care that the agent usually gets it right.

Then track the numbers the business cares about: calls answered, tasks completed, transfers to a person, and bookings that landed correctly in the calendar.

What it takes

A voice agent that sounds good in a demo is easy to build. One that books the right slot in the right time zone, hands off when it should, follows the rules for your state and writes everything back to your CRM takes engineering. That is the part we build, and we have built thousands of agents and voice agents.

Sources

Build or buy: when your own tool wins

Mon, 12 Oct 2026 12:00:00 +0000

All Stack Labs, 12 October 2026

Most of the software a business runs should be bought. Email, payroll and accounting are solved problems, and building your own would waste money. But the stack keeps growing. Zylo, which sells SaaS management software, put median SaaS spend at $9,455 per employee in its 2026 index, and found companies leave 36% of their licences unused. In its 2025 index, companies with up to 500 employees ran 152 apps on average.

Somewhere in that pile is a tool your team bends its work around, or a spreadsheet doing a job no tool covers. That is where building pays.

Renting keeps getting more expensive

Subscriptions do not hold still. Vertice, another spend management vendor, put SaaS inflation at 16.4% in June 2026, the highest on its index. The biggest vendors are also adding AI to their plans and charging for it:

Microsoft raised Microsoft 365 prices from July 2026. Business Standard went from $12.50 to $14 per user a month.
Google added Gemini to Workspace in 2025 and moved Business Standard from $12 to $14.
Salesforce raised Enterprise and Unlimited list prices by an average of 6% in August 2025, and sells its AI agents on top, as add-ons or at $0.10 per action.

Per-seat pricing also taxes growth. Each new hire adds a seat on each per-seat tool.

Your spreadsheet is already a tool you built

When no product fits, the work lands in a spreadsheet. Spreadsheets suit analysis but are fragile as systems. Raymond Panko's review of the research put the average error rate at 3.9% of cells in lab studies, and detailed field audits found errors in 94% of the spreadsheets they examined. His other finding matters more: the people who build spreadsheets are highly overconfident that theirs are right.

Two well-known failures show the risk. In October 2020, Public Health England left 15,841 positive COVID cases out of its daily figures, which press reports traced to lab files loaded into an old Excel format with a row limit. And in January 2012, a JPMorgan model review described the risk calculation behind the trades later known as the London Whale as "done off spreadsheets using a manual process" and "error prone". The US Senate published it the following year.

If a spreadsheet moves data between systems, depends on copying numbers by hand or holds the only record of something important, it is already an internal tool. It just has no tests, no permissions and no history.

When to build

Martin Fowler's test still holds. Ask whether the function is a differentiator. If it is a utility, buy the package and adjust your process to fit it. If it is part of what makes you better than your competitors, you do not want the same software they have. Build when:

It is how you win. Quoting, scheduling or a client portal that customers choose you for.
You pay for a platform to use a few screens of it. Each price rise lands on every seat.
A spreadsheet is doing integration work. As Thoughtworks' Brandon Byars put it, you can't buy integration. If people copy data between systems, that job belongs in software with logs and alerts.
Your data or your rules are unusual. Regulated records, odd pricing, or approvals no product models well.

Buy when the function is a commodity, a good product exists and its roadmap is one you want.

The real cost of building

A tool is not finished at launch. A widely cited estimate from Robert Glass's Facts and Fallacies of Software Engineering puts maintenance at 40 to 80 percent of software costs, with 60 percent the average. Plan for it from the first day.

Build it so that work stays cheap: a standard stack, tests, documentation and everything in your own accounts. Then any capable engineer can extend it, and you are never tied to the people who wrote it. That is how we build internal tools, and why we hand them over.

The rule

Buy the commodities. Build the part that makes you different, and the part your spreadsheets are holding together. When you compare costs, count the seats you will add over the next three years, not the ones you pay for today.

Sources

What an MCP server is, and when you need one

Mon, 12 Oct 2026 12:00:00 +0000

All Stack Labs, 12 October 2026

Anthropic released the Model Context Protocol in November 2024 as an open standard for connecting AI assistants to the systems where data lives. A year later it had been adopted across the industry and handed to the Agentic AI Foundation, a fund under the Linux Foundation co-founded by Anthropic, Block and OpenAI. By then its official SDKs were being downloaded more than 97 million times a month.

If your customers use Claude, ChatGPT, Gemini, Copilot or Cursor, MCP is how those tools can work inside your product on their behalf. An MCP server is the piece you build to let them in.

What it is

An MCP server sits in front of your product and tells an agent what it can do there. It offers three kinds of things:

Tools. Actions the model can take, like creating an invoice or searching orders.
Resources. Data the model can read, like a document or a customer record.
Prompts. Ready-made workflows a user can pick, like "summarise this account".

A server can run locally, as a program on the user's machine, or remotely, over HTTP. Remote is what most products want: nothing for the customer to install, and you control the deployment. The latest revision of the spec, from July 2026, made the protocol core stateless, so a remote MCP server now runs like any other web service, behind your usual load balancers.

Who already ships one

Many of the tools your customers use every day have official servers. Stripe, GitHub, Atlassian, Linear and Notion all publish one. On the client side, MCP works in Claude, in ChatGPT, in Google's Gemini CLI, in Microsoft Copilot and VS Code, and in Cursor.

The permissions are the product

Most of the work is in the permissions. For remote servers, the spec builds on OAuth 2.1. The server publishes how clients should authorise against it, clients must use PKCE, and the server must check that every token was issued for it and must never pass a client's token on to another service.

The best servers go further. Stripe's server asks a person to confirm actions like refunds and outbound payments, and from 31 October 2026 it stops accepting full-access secret keys or any restricted key that is not tagged for agents. That is the pattern to copy: narrow scopes, separate credentials for agents, and a human step for anything that moves money or deletes data.

What can go wrong

Tool poisoning. Researchers at Invariant Labs showed instructions hidden in a tool's description, invisible to users but visible to the model. A server can also change its descriptions after a user has approved it.
Prompt injection through data. In another demonstration, a public GitHub issue led an agent to leak data from private repositories. The server code was not at fault. The agent trusted what it read.
Fake servers. In September 2025, researchers found a package posing as a Postmark MCP server that quietly copied every email it sent to an attacker.
Scopes that are too broad. The spec's own security guidance warns against offering every possible scope and handing clients all of them. OWASP has published a beta MCP Top 10 that includes scope creep, tool poisoning and prompt injection.

The common thread: your server may be running next to untrusted ones, in the same conversation. Enforce permissions on your side, on every call, and never rely on the model to behave.

When your product needs one

Your customers already work in AI tools. If they paste your data into ChatGPT or Claude today, they will want an agent to fetch it and act on it directly.
Your product has work worth delegating. Creating, updating, searching, reporting. Anything a user does in ten clicks is a candidate for one request.
Your competitors are shipping theirs. When an agent picks a tool to finish a task, it can only pick products it can reach.
You have an API, but it was built for developers. Agents need tools shaped around tasks, with descriptions they can follow.

You probably do not need one yet if your product has no API, or if nothing in it is safe to delegate without a person checking each step.

How we build them

Start with the tasks, not the endpoints: what should an agent be able to do for your users, and what should it never do. Design a small set of typed tools around those tasks, write descriptions a model can follow, and put every permission check on the server. Then test it the way agents will use it, across the clients your customers have, and publish it under a domain you control so nobody can pass off a copy as yours.

Start with the permissions

AI tools are becoming one of the ways your customers use software. An MCP server decides what those tools can do in yours, so design its permissions first and build the tools around them.

Sources

Why your AI search gives wrong answers

Mon, 12 Oct 2026 12:00:00 +0000

All Stack Labs, 12 October 2026

In February 2024, a tribunal in British Columbia ordered Air Canada to pay a passenger $812.02. The airline's website chatbot had told him he could claim a bereavement fare within 90 days of buying his ticket. The bereavement page that the chatbot linked to said the opposite. Air Canada argued the chatbot was responsible for its own words. The tribunal called that "a remarkable submission" and found the airline had not taken reasonable care to make its chatbot accurate.

The sum was small, but the ruling was clear: when an AI answers from your documents, its answers are yours. And the research says the cause of a wrong answer is often upstream of the model.

Where it breaks

A 2024 study of three retrieval systems, two of them in use, listed seven ways they fail. Three happen before the model writes anything:

The answer is not there. Your documents do not cover the question, and the system answers anyway instead of saying so.
It ranks too low. The right passage exists, but other passages outrank it.
It gets cut. The passage is retrieved, then dropped to fit the prompt.

The other four happen in the model. It misses an answer buried among noise or contradictions, or it gets the format, the level of detail or the completeness wrong.

More context can make it worse

The instinct is to hand the model more. Google Research found that when the retrieved text was not enough to answer the question, one model's rate of wrong answers rose from 10.2% with no retrieval at all to 66.1%. Partial evidence made it less willing to say it did not know.

Other studies point the same way. In one, models followed incorrect retrieved text over their own correct knowledge more than 60% of the time. In another, answers placed in the middle of a long prompt were found less often than ones at the start or the end. And at 32,000 tokens of context, 11 of the 13 models in one benchmark dropped below half of their short-prompt scores. A bigger context window is not a fix on its own.

Semantic search misses exact lookups

Vector search is good at meaning and poor at exact strings. In Microsoft's tests on customer data, vector search scored 11.7 on keyword-style queries where keyword search scored 79.2. Think part numbers, policy codes, product SKUs and people's names, which is a large share of what customers and staff actually ask.

Searching both ways is the starting point. On the same customer datasets, keyword and vector search combined, with a reranker on top, scored 60.1 against 43.8 for vectors alone.

What fixes it

Hybrid search and a reranker. Anthropic added a short line of context to each chunk before indexing it, then combined vector search, keyword search and reranking. Failed retrievals in their tests fell by 67%, from 5.7% to 1.9%.
Clean documents. Retire old versions, remove duplicates and put effective dates on policies. A model will repeat an outdated page as confidently as a current one.
A tested way to say "I don't know". When the answer is not in your documents, the right response is to say so and hand off to a person. Treat that as a feature and measure it.
Chunks that fit the questions. NVIDIA found page-sized chunks gave the most consistent accuracy across datasets, while short factual questions did best with chunks of 256 to 512 tokens.
No retrieval when you don't need it. If the whole knowledge base is under about 200,000 tokens, roughly 500 pages, Anthropic suggests putting all of it in the prompt.

Measure it, then keep measuring

Start with a set of real questions your users ask, each paired with the passage that answers it. Anthropic's engineering team suggests 20 to 50 cases drawn from real failures as a first set. Add to it every time something new goes wrong.

Score retrieval and answering separately. Context recall tells you whether the right passage came back. Faithfulness tells you whether every claim in the answer is supported by what came back. Open source tools such as Ragas compute both. OpenAI's evaluation guide works through an example that targets context recall of at least 0.85, and recommends running the evals on every change.

Once that set exists, a model upgrade, a new batch of documents or a prompt edit stops being a guess. You can see what got better and what broke.

Where to start

When your AI search is wrong, look at retrieval and the documents behind it before you look at the model, and measure retrieval before and after every change. That is how we build retrieval for clients, and it is what lets you put the system in front of customers.

Sources

Zapier or your own automation platform

Mon, 13 Jul 2026 12:00:00 +0000

All Stack Labs, 13 July 2026

Zapier made automation mainstream, and it is good for getting started. Its pricing, though, punishes growth. This is why we build most client automation on a platform the client owns.

The per-task trap

Zapier and Make both meter usage. Every step a workflow runs counts against your plan. That is fine at a few hundred tasks a month, but automation is supposed to scale. When a workflow that syncs orders or routes leads starts running thousands of times a month, you pay more because it is doing its job. Teams end up rationing automation to protect the invoice, which defeats the point.

What owning the platform changes

Run your automation on your own platform, on infrastructure in your name, and four things change:

Flat cost. You pay for a server, not per task. Volume barely moves the bill until you outgrow it.
Your data stays home. Credentials, customer records and payloads never leave your environment, which matters for privacy and compliance.
No feature gates. Custom code, your own credentials and unlimited workflows are not locked behind a higher tier.
You own it. The workflows, the logic and the integrations are yours, portable and exportable rather than trapped in a vendor account.

Where Zapier still wins

If you need a handful of simple automations and no engineering time, hosted Zapier is the pragmatic choice: no maintenance and a huge library of ready-made connectors. Owning the platform pays off once you have real volume, sensitive data, or workflows complex enough that per-task pricing hurts. A fair rule of thumb: low volume and simple, stay hosted; scaling, sensitive or complex, own it.

The catch: someone has to run it

An owned platform is not free of effort. A production setup needs hardening, backups, monitoring, error handling and updates. Done badly, it is fragile. Done properly, it runs untouched for months. That second part is most of the value, and it is what we build and hand over, documented so your team can own it.

Which one to pick

Hosted tools rent you convenience. Owning the platform buys you the cost curve, the data and the workflows themselves. For anything you plan to lean on as you grow, owning it wins.