Agentic Search:
When the Agent Decides What to Search

Ashish Tiwari
Principal Solutions Architect (Search AI Specialist)

Agenda

01Why search breaks
02The autonomy ladder
03Built natively in Elastic
04Live demo
05What it costs, and where to start

“something to keep my coffee hot on a long drive”

Same index. Same Elasticsearch. Only one thing changed: who decides what to search for.

Agentic search

Understands the need: a travel mug or thermos. Rewrites the query, picks the category, then runs hybrid search with reranking.

Keyword search

Matches the words: coffee beans, coffee makers, anything that says “coffee”. It did exactly what we asked.

Why search breaks

Users describe needs, not queries

Traditional search assumes the user knows what to ask and where to look. Four real queries from the demo catalog show where that breaks.

Needs, not keywords

“keep my coffee hot” means a thermos, not coffee.

Hidden constraints

“nike running shoes under 100 dollars” is a brand, a category and a price filter.

Messy input

“samsng phone”, “sasta laal lipstick”: typos and Hinglish.

Wrong place to look

“can I return perfume?” is a policy question, not a product search.

The idea in one line

“The agent decides what to search. Elasticsearch still decides what's relevant.”

The design principle for the rest of this talk

The autonomy ladder

“Agentic” is a ladder, not a switch

Each rung hands one more decision from the user to the model. Pick the rung per use case.

L1 Hybrid

BM25 + vectors + rerank. The engine decides relevance.

L2 Understand

The LLM picks filters, fixes typos, rewrites.

L3 Route

The LLM picks where to search, retries on zero hits.

L4 Agent

The agent picks which searches and how many, then combines.

Give each job to the right component

The LLM is good at judgement and bad at retrieval. Elasticsearch is the opposite. Keep them in their lanes.

LLM for judgement: intent, filters, rewrite, route, as structured JSON.

Elasticsearch for retrieval: hybrid search, RRF, reranking, aggregations.

Workflows for control: deterministic steps, guardrails, a trace for every run.

Agent Builder for reach: the same pipeline as an API, a chat agent, MCP and A2A.

Built natively in Elastic

One API call, one workflow, zero glue code

Architecture: UI calls one Agent Builder tool that runs a Kibana workflow: understand, plan, route, respond, using EIS and Elasticsearch

The LLM fills a schema. It never writes query DSL.

1 · ai.prompt output (JSON schema)

{
  "intent": "product_search",
  "search_text": "running shoes",
  "categories": ["mens-shoes", "womens-shoes"],
  "brands": ["Nike"],
  "price_max": 100,
  "sort_by": "relevance",
  "reasoning": "Nike running shoes, $100 cap"
}

Grounded: the prompt carries the real category and brand list from the index, so “samsng” becomes Samsung.

2 · stored search template (mustache)

"filter": [
  { "bool": { "should": [
    {{#brands}}{ "term": { "brand": "{{.}}" } },{{/brands}}
    {{^brands}}{ "match_all": {} },{{/brands}}
    { "match_none": {} } ] } },
  {{#price_max}}
  { "range": { "price": { "lte": {{price_max}} } } },
  {{/price_max}}
  { "match_all": {} }
]

The query is written and tested by us. The model only supplies parameters, so a bad answer can't produce a bad query.

Hybrid retrieval, reranking, self-correction

Template params feed BM25 and semantic retrieval, fused by RRF, reranked by Jina v3; if zero hits, relax filters and retry

Six Elastic building blocks, no extra services

semantic_text on EIS

Jina v5 embeddings created at ingest. No model hosting.

Retrievers

rrf fuses BM25 and semantic results in one request.

Reranker

text_similarity_reranker with Jina reranker v3.

Search templates

Tested query shapes. The LLM fills parameters only.

Workflows

ai.prompt, switch, if, data.set, workflow.output in YAML.

Agent Builder

Workflow tools, ES|QL tools, agents, MCP and A2A.

Live demo

Demo run-sheet

ShopSphere: 194 real products with images, 20 help-centre articles, one Elastic Serverless project.

1 · Classic vs agentic

“keep my coffee hot on a long drive”, “red lipstick sasta”

2 · Understand

“nike running shoes under 100 dollars”, “samsng phone”, “cheapest laptop in stock”

3 · Route + self-correct

“best rated laptop brand?”, “can I return perfume?”, “capital of France”, “gucci running shoes”

4 · The agent decides

“Goa beach trip, $150 total. What should I pack?”

Still Elasticsearch JSON, plus the reason why

What the API returns

{
  "took": 41,
  "hits": {
    "total": { "value": 7 },
    "hits": [
      { "_score": 0.93,
        "_source": { "title": "Nike Air Jordan 1…",
                     "brand": "Nike", "price": 89.99 } }
    ]
  },
  "agentic": {
    "route": "product_search",
    "rewritten_query": "running shoes",
    "relaxed_fallback": false,
    "execution_id": "…"
  }
}

Why that matters

  • Your front-end doesn't change: same took, hits, aggregations
  • Explainable: the agentic block says what was decided and why
  • Traceable: every run has an execution trace in Kibana, step by step
  • Reusable: the same tool serves the UI, the agent, and MCP / A2A clients

L4: the agent decides what, and how often, to search

Agent Builder agent splits a trip goal into four searches, calls the agentic search tool for each, then combines results within budget

What it costs, and where to start

Trade-offs, and how we handled them

Concern What we did in the demo
Latency One LLM call per search, a small fast model on EIS, reasoning-level: none, catalog list built into the prompt
Wrong answers JSON schema output, index vocabulary in the prompt, stored templates, zero-hit fallback
Debuggability Every run has a workflow execution trace; the response carries the decisions
Cost Agent (L4) only for goals; single searches stay at L2/L3
Lock-in Standard ES JSON out; the tool is exposed over MCP and A2A

Start on Monday

You don't need a new stack. Put a thin agentic layer in front of the index you already have.

1 · Understand

One workflow: ai.prompt → search template on your existing index.

2 · Route

Add a switch for your second index and a zero-hit fallback.

3 · Expose

Wrap it as an Agent Builder tool. Let agents call it over MCP.

Thank you!

Questions? The demo project: workflows, templates, agent and UI.

Resources

0:00-0:30. Introduce yourself. One line: today we let the agent decide what to search, and keep Elasticsearch deciding what's relevant.

0:30-1:00. Don't read it. "Problem, mental model, how it's built, show it, what it costs."

1:00-3:00. Hook. Optionally flip to the UI "Classic vs Agentic" tab and run this live. Say: "Nothing is broken. The engine matched words. I asked in words, but I meant a need."

3:00-5:00. India angle: "Our users type in Hinglish, abbreviate and misspell brands. Query understanding isn't nice-to-have for us, it's table stakes." We spent 20 years teaching users to speak search engine. Agentic search flips it.

5:00-5:30. Pause. Let it land.

5:30-8:00. L0 is plain keyword search: the user decides everything. "Most teams jump straight to L4 because it demos well. In production most value sits at L2 and L3: deterministic, fast, debuggable. The L4 agent should CALL L2/L3 as a tool, not replace it."

8:00-9:00. This is the architecture in one slide.

9:00-11:00. Walk left to right. "The UI makes one call. Everything else runs inside Elastic. No LangChain service, no Lambda." Per request: one LLM call + one search (two if the zero-hit retry fires).

11:00-13:00. The most important design choice in the talk. "That's the difference between a demo and something that passes a bank's security review."

13:00-14:30. semantic_text on EIS: no model hosting, multilingual. RRF fuses. Reranker makes the top 12 the best 12. Self-correction is the "agentic" part here: the workflow notices zero hits and changes strategy.

14:30-15:00. Quick recap before the demo.

15:00-25:00. Switch to the UI (localhost:8080). ~2.5 min per column. In tab 1, open the JSON drawer once: Response, Search-template params, Workflow execution trace. Fallback: if the network fails, use screenshots / Kibana workflow execution history.

Show this right after demo step 2, or skip if time is short. Values shown are illustrative.

Demo step 4. "I didn't write this plan. The agent did. And every search it ran is the same guardrailed pipeline from L2/L3."

25:00-27:00. Be honest: L4 agent responses take several seconds; L2/L3 search is one model call. Run `python3 setup.py --timing` beforehand and quote your real numbers here.

27:00-28:00. Call to action. Then thank you + Q&A.