What Is Site Search? How It Works, Core Metrics and Capability Framework

Summary: Site search retrieves only the content a website or app owns, built from a four-stage pipeline of indexing, query understanding, retrieval and ranking, and feedback, and its performance is measured by zero results rate, click-through rate and conversion rate.

1. What Site Search Is and Where It Stops

Site search is the retrieval capability deployed inside a website, app or enterprise system, with its scope limited to content that organization owns. What it retrieves may be products, articles, documents, orders, tickets or knowledge base entries, but they all share one trait: site search answers only whether something exists here, never whether it exists somewhere on the web.

Compared with a general search engine that crawls the whole web, the difference shows up in four basic constraints.

Dimension	Site search	General search engine
Data source	Own business and content repositories, highly structured	Crawled from the whole web, sources uncontrollable
Retrieval goal	Help the user finish one concrete action: place an order, find a document, file a ticket	Help the user understand a topic
Ranking basis	Relevance plus business goals such as inventory, margin, freshness and permissions	Relevance plus page authority
Cost of failure	The user leaves immediately, and it was mostly high-intent traffic	The user switches engines and searches again
Controllability	Indexing, tokenization and ranking rules are all tunable	Only page-side optimization is possible

This table says something important: site search is not a smaller search engine, it is a decision system tightly coupled with the business database. For one and the same query, inventory status, price band, user tier and content permissions all change what should be returned.

2. How It Works: The Four-Stage Pipeline

A single query travels from the input box to the results page through four stages connected in series.

Index construction: offline, every piece of content is cut into tokens and an inverted index is built, meaning a dictionary plus posting lists.
Query understanding: turning natural language into retrieval conditions, covering tokenization, error correction, synonym normalization, intent classification and query rewriting.
Retrieval and ranking: the retrieval layer maximizes breadth through inverted indexes plus vectors, while the ranking layer maximizes precision through BM25 plus business weighting plus learning to rank.
Presentation and feedback: results pages, faceted filters and tracking data close the optimization loop.

According to Elastic's official documentation, How full-text search works, text analysis applies ordered transformations including stemming, lowercasing and stop-word removal, and Elasticsearch's default relevance algorithm is Okapi BM25, which scores by term frequency, document frequency and document length (How full-text search works). Apache Lucene's official API documentation ships BM25Similarity as the standard similarity implementation, with default parameters k1 = 1.2 and b = 0.75 (BM25Similarity). BM25 remains the mainstream baseline for term matching today: explainable and free of training. Semantic vector retrieval compensates for its weakness on synonyms and cross-language queries, so the two should run in parallel as hybrid retrieval and then be re-ranked together.

Baymard Institute, the Danish user experience research firm, reports in its ecommerce search UX benchmark for 2026, covering more than 170 sites and apps and more than 10,000 manual ratings, that 56% of sites fail to support users' search needs adequately. Broken down by query type, non-product queries such as return policies, sizing notes and delivery times are the worst served, with 66% of sites showing problems (Ecommerce Search UX). That confirms query understanding as the place where site search breaks most easily and repays investment most generously.

3. The Modern Site Search Capability Map, Taking Tengence Search as the Example

Definitions and principles are the skeleton; what actually shapes the experience is platform capability. Taking Tengence Search as the example, here are the three core capability groups.

3.1 Intent Recognition: Teaching the System to Understand Users

Query understanding is intent recognition at heart. Tengence Search ships multiple intent recognition capabilities in three modes, built-in model, self-trained model and custom model, so business teams can adopt them gradually as their data accumulates:

Capability	Problem it solves	Supported modes
Tokenization	Word segmentation, plus traditional and simplified Chinese and full-width normalization	Built-in / self-trained / custom
Synonyms	Mapping "sofa", "couch" and local variants onto one concept	Built-in / self-trained / custom
Error correction	Fixing pinyin input, visually similar characters and adjacent-key typos	Built-in / self-trained / custom
Named entity recognition	Spotting brands, models and specifications as key entities	Built-in / self-trained / custom
Category prediction	Deciding which category tree node a query lands on	Built-in / self-trained / custom
Term weighting	Separating head terms from modifiers so effort is not spread evenly	Built-in / self-trained / custom
Standardization	Normalizing queries into one consistent form	Standard implementation
Query autocomplete	Guiding intent while typing and cutting the spelling burden	Built-in / self-trained / custom

Term weighting and category prediction are routinely underestimated: the former decides that in a query like "red dress" the dress is the subject, and the latter decides whether the results page lands in the right product domain.

3.2 Retrieval and Ranking: Breadth First, Precision Second

Retrieval owns finding everything relevant; ranking owns putting the best first. Tengence Search offers multi-layer retrieval and several tiers of ranking:

Capability	Role
Inverted retrieval	Exact term matching, where a model or code cannot vary by one character
Multi-route retrieval with fusion	Combining inverted, vector and category retrieval to raise coverage
AI semantic retrieval	Training semantic vector models to handle synonyms, cross-language cases and vague phrasing
First-stage scoring formula or model	Filtering huge candidate sets fast, with built-in, self-trained and custom options
Deep scoring formula or model	Computing precise scores for candidates, combining content understanding
Personalized reranking	Training ranking models against user tag systems so results differ per user
Automatic iteration	Algorithms retraining themselves on live performance data

On performance, Tengence Search sustains ultra-high traffic at hundreds of millions of daily page views, holds P95 response time at 80 ms, and keeps the entire pipeline updated in real time. Training semantic vectors on multimodal content and tuning multi-route fusion at the retrieval stage, then refining understanding through user tags at the ranking stage, is the typical adoption path for this solution.

3.3 Operational Intervention: Shipping Business Ideas Fast

Technology answers "is the result right"; operational intervention answers "is the result what the business needs now". Tengence Search puts weight on full business intervention that takes effect through configuration rather than a release:

Ranking intervention: pinning into fixed positions, boosting or demoting by weight, blocking by condition through allow and deny lists, direct-hit results and result diversification.
Operational term configuration: hot terms, placeholder terms, related searches and past searches, all available as built-in, self-trained or custom modes.
Experimentation and visualization: full-pipeline A/B testing, end-to-end visual configuration, real-time online debugging logs and expert-level deep parameter tuning.
Data immediacy: data source synchronization and real-time data push, so operational actions take effect instantly.

After adopting the platform, a well-known electronic component trading platform built up flexible operational intervention step by step, letting new products, promotions and compliance requirements reach search results quickly instead of queueing for engineering time every time.

4. Core Metrics and How to Measure Them

Response speed alone says little about site search quality. The seven metrics below cover the four questions of whether users bother to search, whether they find anything, whether they click and whether they buy, and every formula can be computed directly from search tracking.

Metric	Formula	Typical range	What a deviation means
Search usage rate	Sessions with a search divided by all sessions	Ecommerce 20%–40%	Search box placement or visibility is the problem
Zero results rate	Queries returning zero results divided by total queries	Excellent below 2%; typical 8%–15%	Index coverage gaps, missing synonyms, tokenization errors
No-click rate	Queries with results but no click divided by queries with results	Excellent below 20%	Ranking is irrelevant, or snippets do not earn the click
Above-the-fold click share	Clicks at position 10 or above divided by all clicks	70%–90%	Too high means top-heavy ranking and no long-tail exposure
Reformulation rate	Share of queries reformulated within one session	Excellent below 25%	The first result missed intent, or query understanding is weak
Search conversion rate	Orders from search sessions divided by search sessions	Usually 1.5 to 3 times the site average	Below average means high intent arrived but did not convert
P95 response latency	Query response latency at the ninety-fifth percentile	Excellent below 200 ms	Past one second, search exit rate climbs sharply

Two cautions. First, metrics must be read in slices, by device, language, new versus returning visitor and top-level category, because global averages hide local collapse. Second, zero results rate is the only directly actionable metric: sorting zero-result queries by frequency lets you read assortment gaps and vocabulary gaps in the customer's own words.

5. FAQ

Q: How do site search and site navigation, meaning the category tree, relate to each other?

A: They complement each other. Navigation suits users whose need is vague and who are still exploring; search suits users whose need is concrete and who already have a target term. Roughly half of users reach for search first when looking for products. Healthy sites usually have both share one attribute system, so that words search understands are also words the filters understand.

Q: What counts as an acceptable zero results rate?

A: There is no mandatory standard and categories differ sharply, with attribute-heavy categories naturally higher. In practice below 2% is treated as excellent, 8%–15% is the common band, and above 20% points to systemic problems in indexing or query understanding. Trend and segmentation matter more: once you split by language, device and category, any slice running clearly above average is a concrete repair list.

Q: With semantic vector search in place, is BM25 still needed?

A: Yes. Vector retrieval excels at synonyms, cross-language cases and vague phrasing, but on models, codes and exact nouns, the kind of query where one character off means wrong, term matching is still more accurate. The mainstream approach runs both in parallel as hybrid retrieval and re-ranks them together, rather than replacing BM25 with vectors.

Q: Does operational intervention mess up search results?

A: It depends on keeping it transparent. When every intervention, whether pinning, blocking or boosting, is traceable through visual configuration and real-time logs, and its impact is quantified with A/B tests, intervention is a controllable lever. What actually gets out of hand is manual result editing with no logs and no experiments. Every intervention should carry traceable conditions and an expiry date.

Q: Where should a small team with no algorithm engineering capacity start?

A: Start by completing your tracking so you can compute three baselines: zero results rate, no-click rate and reformulation rate. Then sort zero-result queries by frequency and batch-fill synonyms and spelling tolerance. Only then consider a ranking model. The first three steps need no algorithm team, yet they carry the most certain return.

Data Sources

Elastic official documentation: How full-text search works — text analysis, inverted index structure and the default relevance algorithm
Apache Lucene official API documentation: BM25Similarity — the standard BM25 implementation and the meaning of the k1 and b parameters
Baymard Institute: Ecommerce Search UX 2026, a benchmark across eight query types — search experience benchmark across more than 170 sites and apps and more than 10,000 ratings
Baymard Institute: The State of Mobile E-Commerce Search and Category Navigation — benchmark data on mobile zero-result pages and search experience
NIST: official TREC-3 proceedings, the volume containing Okapi at TREC-3 — the original source of the BM25 weighting function

About Tengence

Tengence Search is the Tengence product for enterprise site retrieval, covering the complete set of capabilities from intent recognition and retrieval and ranking to operational intervention. Tengence is the enterprise growth engine for the AI era, focused on public-traffic acquisition and private-domain conversion; its core products are Tengence GEO, the dual-engine GEO plus SEO offering, and Tengence Search.