How Yandex cut GPU costs for its AI agent by 6.6 times — and what it means for small business

Yandex has described how, over the course of a year, it built a research agent within Alice AI — the “Research” mode, conceptually similar to OpenAI's Deep Research. The publication was written by the team lead and a product manager. It breaks down the engineering mechanics: how the agent works internally, why the model itself wasn't retrained, and why the cost of a single query dropped 6.6 times in three months.

At first glance, this looks like a technical story for developers. But it contains three things relevant to any business owner using AI tools for customer interaction, content, or analytics.

First: the quality of an AI's answer is often determined not by the model but by the harness built around it — which changes how you should choose and configure services. Second: the cost of AI services for end users depends not only on how “smart” the model is but on the engineering surrounding it, and that engineering can get cheaper within a matter of months. Third: the reliability of long-running AI queries, where an answer takes minutes rather than seconds to prepare, is not a minor detail — it determines whether a service holds up under a surge of users or fails.

What the agent is and why it exists at all

The “Research” mode in Alice AI is a tool for complex queries that can't be answered with a single search. According to Yandex, the agent builds a plan for solving the task, runs hundreds of search queries, visits websites (including ones with dynamic JavaScript content), writes and executes Python code for calculations, and works with uploaded files. The example given in the publication is a request to “plan a two-week road trip to Dagestan with kids”: the agent assembles the route itself, checks the data, and compiles it into a coherent answer.

The first version took 30 minutes to respond, sometimes longer. That's a fundamentally different class of task compared to an ordinary chatbot that produces an answer in seconds. It was precisely because of this long response time that the team had to build separate infrastructure: an agent transport system (ATS) and tooling for multi-step queries involving hundreds of calls to different services.

For businesses, what matters here isn't the technical internals but the class of tasks such a tool solves: not “write me a text” but “gather and verify information from dozens of sources and give a structured conclusion.” That's closer to the work of an analyst or junior marketer than to a typical chatbot.

The main takeaway: the harness matters more than the model

The central point of the publication is stated concisely: answer quality is often determined not by the language model itself but by what's built around it. In the industry this is called the harness — the combination of orchestration, memory and context handling, verification, retries, code execution isolation, and a system for observing what's happening inside. There's a separate term, scaffolding — predefined templates and schemas the agent follows. The harness is what runs during operation; the scaffolding is what's laid down at the design stage.

Yandex's team illustrates this with concrete examples. A relevance classifier that filters out junk fragments from search results before handing them to the model added 9 percentage points on the BrowseComp benchmark. Training the agent to work with files — PDFs, spreadsheets, images — added 8.5 percentage points on the GAIA benchmark. A separate “advanced reasoning” tool once added 5 points, but as newer versions of the base model came out, it started getting in the way and was dropped.

This leads to a practical conclusion. The publication states it directly, quoting the team: “99% of the work is done by the model itself; we don't need a complex framework around it” — echoing Richard Sutton's well-known “Bitter Lesson”: add-ons designed around the weaknesses of a specific model become obsolete with every new release. That's why the harness needs not only to be expanded but regularly trimmed, with the value of each component re-checked after every model update.

For a business owner, this means: if you're comparing two AI services by the name of the model inside them (“built on GPT” versus “built on another model”), that's not the main criterion. Differences in answer quality more often come from how the service processed your query, filtered sources, and verified the result — not from the model version.

How they're saving on GPUs — and why it shows up in subscription pricing

A separate section of the publication is devoted to money. Every extra token in the context and every unnecessary model call burns expensive computing time on GPUs — which in turn limits how many users a service can serve at once.

Yandex lists three steps that reduced consumption:

• processing of search fragments was handed off to a trained classifier instead of running them through the large model itself
• the agent now calls tools only when doing so actually advances the research, rather than by default
• generation of the research plan was shifted to a lighter model without losing quality

As a result, the cost of a single query dropped 6.6 times in three months. That's not an abstract number for a report: this kind of savings determines whether a service can sustain a free tier, lower subscription prices, or survive a sharp surge in users without breaking down.

A telling episode from the publication is May 9, when Yandex launched a feature for searching information about relatives who fought in World War II through open archives (the OBD “Memorial,” “Feat of the People,” and “Memory of the People” databases). The surge in users was so large that the team urgently needed extra computing capacity, which a neighboring team lent them for a single day. Infrastructure built with checkpoints — saving the agent's state after each step — meant users didn't lose progress even while servers were overloaded.

For businesses planning to use AI agents during peak periods — sales, holidays, demand spikes — this episode shows that reliability of long-running AI queries isn't guaranteed by default. It's worth asking your service provider what happens to your query if it takes several minutes to run and something fails on the provider's end in the meantime.

What this means for small businesses in Russia and the CIS

The publication doesn't call for any direct technical action from a shop owner or service provider — you won't be building a harness yourself. But there are several practical takeaways from Yandex's story.

First: if you use AI agents for research tasks — competitor analysis, gathering supplier information, preparing reference material for clients, sifting through a mass of reviews — don't judge a service solely on “how smart the AI is.” Check whether it actually works with real files and websites rather than just text in a chat window: the difference in answer quality on a complex, multi-step query (“find all fabric suppliers that deliver to my region and compare prices”) comes down to the infrastructure around the model, not a flashy name.

Second: don't be put off by long response times. If a service is upfront that a piece of research will take several minutes, that's not a sign something is broken — it's a sign the agent is genuinely checking sources rather than returning the first thing it finds. A quick answer to a complex question about a market, competitors, or legal nuances is often less reliable than one that took longer to assemble.

Third: factor peak loads into your plans if you're building a product or service on top of a third-party AI agent — for example, a website chatbot with “smart search” over a catalog. The May 9 surge shows that even a large company with substantial infrastructure can run into a shortage of computing capacity during a spike in interest. If you're a small business connecting a third-party AI service to your website or messenger, ask the provider what happens to your customers during an overload — is the conversation lost, or preserved?

Fourth: watch pricing on such services with optimism rather than suspicion. Yandex's publication shows that the cost of processing a single query in complex AI agents can fall quickly — by several times within a few months — simply through engineering optimization, without the base model becoming more expensive or worse. That means subscriptions to such tools may get cheaper over time or expand their free limits, rather than only rising in price.

Fifth: if you have an in-house marketer or analyst who has to manually gather information from dozens of sources — comparing competitor prices, monitoring reviews, sourcing suppliers — it's worth testing a deep-research mode in whatever AI services you have access to on a real work task this week. The time cost of such a test is minimal, and if the task matches the tool's capabilities, the payoff can free up several hours of manual work a week.

Bottom line

Yandex's publication is interesting less for its description of a specific product than for its direct acknowledgment of what determines the quality of an AI agent's answers in practice: not the size or “intelligence” of the model, but the engineering harness around it — filters, verification steps, tools for working with files and websites, and a mechanism for preserving progress through failures. This harness simultaneously lowers the cost of processing a query — in Yandex's case, by 6.6 times over three months — and determines whether the service holds up under a sharp rise in users.

For businesses, this carries two practical implications: evaluate AI services by how well they handle your real, complex tasks rather than by the name of the model inside, and when negotiating with AI tool providers, raise the question of resilience under peak loads — especially if the service is tied to your website or sales.