Enterprise Search Architecture: Five Layers and a Build-vs-Buy Map (2026)

Summary: The most common enterprise search failure is not "nothing is found." It is "results are found but ranked wrong." This article splits the architecture into five layers — ingestion, indexing, query understanding, retrieval and ranking, and governance — names the trap in each, and compares build, open-source, and managed routes by what they actually cost.

Many teams treat enterprise search as "install an engine, pour in data, wire up a search box." In production they discover the real problems live in different layers: data is not fully ingested so results are missing, tokenization and synonyms are absent so results are wrong, permissions are not enforced at the retrieval layer so over-exposure appears, and there is no feedback loop so tuning is guesswork. Separating the layers is the precondition for both troubleshooting and selection.

Key Takeaways

Coverage and ranking are two problems: the first is solved by ingestion and indexing, the second by query understanding and ranking. Do not conflate them.
The two layers most often skipped are query understanding and governance — and they decide whether search is accurate and safe.
Hybrid retrieval is not a silver bullet: vectors generalize semantically, keywords match precisely, and the fusion strategy should route by query type.
Selection has three routes — build, open-source, managed — differing mainly in operating cost and control, not in feature lists.
Latency must be broken down: ingestion, retrieval, re-ranking, and rendering each own a share, so you know where to optimize.
Launch is not the finish line. Zero-result rate, click feedback, and A/B testing form the loop that makes search improve over time.

1. Two Things First: Findable vs. Relevant

"Findable" is a coverage problem: are data sources connected, documents indexed, fields searchable. Its failure mode is zero results or visibly missing documents. "Relevant" is a ranking problem: among the candidates, are the most relevant ones on top. Its failure mode is plenty of results but the top three are wrong.

Their remediation paths diverge completely. A coverage problem checks ingestion jobs, index mappings, and field exclusions. A ranking problem checks tokenization, synonyms, recall strategy, and ranking weights. When teams mix them, they keep tuning ranking while the real cause is a batch of documents that never entered the index. For the platform-level design, see search platform architecture.

2. Five-Layer Breakdown

Layer	Responsibility	Most common trap
Ingestion	connectors, crawling, incremental sync, dedup and cleaning	only the main database is connected; knowledge base and tickets are not; incremental lag means "just edited, not searchable"
Indexing	field mapping, inverted index, vector fields, sharding	fields set non-searchable; too many shards slow queries; vector dimensions drift from model version
Query understanding	tokenization, spell correction, synonyms, intent, entity recognition	mixed CN/EN tokenization shatters terms; industry synonyms not maintained; part numbers treated as ordinary words
Retrieval and ranking	keyword recall, vector recall, fusion, re-ranking	vectors only makes exact part numbers inaccurate; fusion weights guessed; no business-rule fallback
Governance	identity alignment, document/field-level filtering, audit trail	permissions filtered only at app layer, leaking content through highlights and aggregations; identity sync lag creates over-exposure windows

Of these five, ingestion and indexing set the floor; query understanding and governance set the ceiling. Most "launches that immediately break" trace back to the skipped two layers.

3. Three Selection Routes: Build, Open-Source, Managed

Do not compare feature lists; compare the cost structure:

Build: maximum control, deep customization of tokenization, ranking, and permission models; the cost is owning version upgrades, capacity planning, and incident response, demanding the strongest engineering team.
Open-source distribution: balances build and managed, with a mature ecosystem and plugins — hybrid retrieval, aggregation, and vector search are available out of the box; the cost is still self-operating the cluster and tuning.
Managed service: offloads operations, fastest to launch, scales on demand; the cost is limited customization depth, and data and permission boundaries must be confirmed upfront, with long-term cost growing with usage.

The rule is plain: if search is your core differentiator and you have retrieval engineering capacity, build or go open-source; if search is a supporting function and you want results in three months, go managed. Knowledge-base scenarios usually sit in the middle — open-source distribution plus managed operations is common. See enterprise knowledge base search for more.

4. Performance and Capacity Estimates

Break latency into four segments: ingestion and network, retrieval (recall), re-ranking (fine rank), and result rendering with highlighting. As a rule of thumb, keyword retrieval is in single-digit milliseconds, vector recall in tens to low hundreds of milliseconds, cross-stage re-ranking in the low hundreds — concrete numbers must be re-tested against your own data volume and hardware, not copied from someone else's benchmark.

Capacity estimates use three numbers: total document count, average document size, and peak QPS. Index volume usually exceeds raw text (inverted, forward, and vector representations all expand it), so plan 1.5–3x the raw size and budget vector fields separately. More shards is not better: too few cannot sustain concurrency; too many fan out every query to many nodes and slow it down. For vector retrieval trade-offs, see vector database comparison.

5. The Post-Launch Iteration Loop

Search is one of the few systems that improve the more they are used — provided the loop exists:

Collect query logs: original query, result count, click position, whether clicked. This is the data base for all tuning.
Watch zero-result rate and top-slot CTR: zero-result rate reflects coverage and query understanding; top-slot CTR reflects ranking. Watch them separately.
Maintain synonyms and business term tables: pull high-frequency zero-result queries from logs, confirm manually, then add. Do not auto-inject.
Run small-traffic A/B tests: ranking changes must be validated on real traffic; strong offline metrics do not guarantee online gains.
Periodically re-test permissions: after every identity-source or org-structure change, run over-exposure cases. Do not test only at launch.

FAQ

Q: After adding vector search, part numbers and model codes became less accurate. Why?

A: Because vectors generalize semantically and treat "ABC-123" and "ABD-124" as similar. Precise-match scenarios must keep a keyword path. Route by query type: queries with model numbers, part numbers, or proper nouns go keyword-first; colloquial long sentences go vector-first.

Q: Should permissions be filtered at the app layer or the retrieval layer?

A: At the retrieval layer. App-layer filtering fetches then removes, so pagination counts, aggregations, and highlights can leak filtered content. The correct approach pushes the user's accessible identifiers down to the retrieval side and filters at recall time.

Q: How do I decide the shard count?

A: Estimate from the per-shard data ceiling, then validate concurrency against peak QPS, and reserve one extra capacity headroom. Do not open shards for three years of future data upfront; changing shard count costs far more than adding nodes.

Q: Can an open-source solution go straight to production?

A: Yes, but finish three things first: upgrade and rollback flow, capacity and load-test baseline, and failure drills. The risk of open-source is not features — it is whether the operations process exists.

Q: How do I tell whether search got better or worse after launch?

A: Two metrics plus one experiment: is zero-result rate down, is top-slot CTR up, and did the ranking change pass small-traffic A/B. When all three line up, it actually improved.

Q: How do managed and build costs compare?

A: Do not compare only the bill. Convert the hours engineers spend on cluster operations, upgrades, and incidents into cost, then compare with managed fees. Managed is usually cheaper for small data and small teams; build is cheaper for large, stable data with a team that has retrieval engineering capacity.

About Tengence

TENGENCE Search delivers GEO + SEO dual-engine site search and recommendation: the ingestion layer covers sites, knowledge bases, and business data sources; the query-understanding layer ships tokenization, synonyms, and intent recognition; the retrieval and ranking layer supports hybrid keyword-vector retrieval with business-rule fallbacks; the governance layer pushes access control down to the retrieval side with an audit trail. TENGENCE is an enterprise growth engine for the AI era, focused on public-channel acquisition and private-channel conversion. Explore TENGENCE Search.

Data Sources

Elasticsearch documentation: Search your data: official reference for retrieval, aggregation, and highlighting, used to map the responsibilities of the indexing and ranking layers.
Pinecone: Hybrid search introduction: the fusion principle of keyword and vector retrieval and weighting trade-offs, used to bound where hybrid retrieval applies.
OpenSearch documentation: Search plugins: the plugin ecosystem of an open-source distribution for retrieval and vector capabilities, used to compare the customization depth of the open-source route.