Enterprise Search Platform Architecture: How to Build a System-Level Search

Summary: Enterprise search needs usually grow out of individual business systems—documents, tickets, products, and support records all need search. Each system building its own stack leads to fragmented experience, duplicated effort, and separate permission control. A search platform extracts retrieval capability into shared infrastructure: one indexing pipeline, one query service, one permission system, reused by multiple business sides. This article gives a six-layer architecture, offline and online dual pipelines, hybrid retrieval with reranking, multi-tenant isolation strategies, and the five most common architecture mistakes.

Key Takeaways

Enterprise search is not web search: there is no PageRank to lean on, freshness weight is higher, organizational context is critical, and permission is a hard core constraint.
Six-layer architecture: data ingestion → processing and indexing → query service → model service → permission and security (cross-cutting) → monitoring and evaluation.
Dual-pipeline decoupling: offline (ingestion → cleaning → chunking → indexing) and online (gateway → understanding → recall → rerank → generation → attribution) are separated, kept fresh by CDC incremental sync.
Hybrid retrieval plus reranking is the quality core: BM25 top 30 + vector top 30 → RRF fusion → rerank → top 10.
Permission runs through the whole flow: ACL is the hardest part (nested groups, inheritance, frequent change), and permission changes must propagate to the index immediately.
Three tenant-isolation options: independent index (strong isolation), shared index + tenant filter, independent database; choose by scale and security requirement.

Why You Need a Search Platform

A single business system wiring up its own search is fast in the short term but pays three costs in the long run:

Fragmented experience: what system A can find, system B cannot; the same term returns inconsistent results in two places.
Duplicated effort: tokenization, synonyms, vectorization, and permission filtering are each rebuilt from scratch.
Runaway permissions: each system manages its own permissions, raising both over-permission risk and audit difficulty.

The search platform's answer is to precipitate retrieval capability into shared infrastructure: one indexing pipeline + one query service + one permission system, with business sides connecting through APIs or embedded components. This lets retrieval quality iterate centrally—improve once, benefit everywhere.

Six-Layer Architecture Overview

A complete enterprise search system usually splits into six layers:

Data ingestion layer: connects to various data sources (document systems, file storage, code repositories, tickets, support, databases, object storage, knowledge bases), supporting scheduled sync, incremental update, permission sync, and change listening.
Data processing and indexing layer: document parsing, cleaning and deduplication, chunking, metadata extraction, Embedding vectorization, building three index types—vector index + inverted index + structured index.
Core query service layer: query intent recognition, multi-path recall, fusion reranking, context building, answer generation, and citation attribution.
Model service layer: Embedding, Rerank, LLM, OCR, document parsing, intent recognition, and other models—run locally as open source or connect to an existing model platform.
Permission and security layer: cross-cutting through the whole flow—access control and permission filtering from ingestion, index building, to result display, plus audit trails and output protection.
Monitoring and evaluation layer: recall and ranking metrics, eval-set regression, query-log analysis, cost and latency monitoring.

Offline Pipeline: Ingestion, Cleaning, Indexing

The offline pipeline decides what gets fed in, and also the quality ceiling:

Data source ingestion → cleaning / deduplication / permission parsing → document chunking → metadata extraction → Embedding → index building → publish and incremental update

The key engineering practice is CDC-driven incremental indexing:

App writes to Postgres ──CDC──▶ index queue ──▶ Worker enrichment (compute embedding, derive fields) ──▶ search engine

The benefit of this decoupling is concrete: if the search service goes down, it does not block business writes; rebuilding the index does not disturb the application. The wrong approach is full rebuild on every data change—the correct posture is CDC incremental sync, with full rebuild done weekly as routine.

For component selection, the common combination for the index-storage layer is a vector database (Milvus / Qdrant) + a full-text engine (Elasticsearch / OpenSearch) + a relational database (PostgreSQL) + a cache (Redis) + object storage (MinIO). Small or pilot deployments can start on a single machine with Docker Compose, then move to Kubernetes at scale (for selection details see C09 Vector Database Comparison).

Online Pipeline: From Query to Answer

The online pipeline is the part users directly perceive:

User query → API gateway (auth / rate limit) → query understanding → multi-path recall → fusion rerank → context building → LLM generation → citation attribution / safety filtering → return

Query understanding: intent classification (navigational / informational / transactional), entity recognition (person / date / project), query expansion (synonyms, e.g. PTO expanded to "PTO OR paid leave OR vacation"), spell correction, personal context (the same query means different things to different roles).
Multi-path recall: keyword retrieval + vector retrieval + structured retrieval + graph retrieval run concurrently, covering different matching paths.
Fusion rerank: merge and fine-rank the multi-path results (see next section).
Context building and generation: assemble snippets to feed the model for answer generation, with forced citation attribution.
Safety filtering: one more pass of permission and compliance before output.

Hybrid Retrieval and Reranking

The quality core of the online pipeline is combining keyword precision with vector semantics. Typical flow:

Query
├─ BM25 (keyword) ──▶ top 30
├─ Vector (vector) ──▶ top 30
└─ RRF fusion ──▶ top 30 ──▶ Reranker ──▶ top 10 ──▶ display

BM25 guarantees that exact matches like model numbers, proper nouns, and error codes are not lost.
Vector fills the semantic gap—"how to deploy" can hit the Deployment Guide even when the query contains no "deploy" word.
RRF (reciprocal rank fusion) merges the two result paths, covering both sides.
Reranker (cross-encoder fine rank) does one more fine rank, usually the single highest-yield step in the online pipeline.

Ranking is not a single score but a layered signal combination: lexical relevance (BM25, title often weighted 3x) + semantic similarity + business signals (freshness, popularity, click-through, conversion) + personalization, finally combined by a learning-to-rank model (e.g. gradient boosted trees). Train offline, serve online—do not put training into the request path (for ranking details see C05 Search Relevance Ranking).

Permission and Security: Through the Whole Flow

Permission is the hardest and most non-negotiable part of enterprise search.

ACL indexing: index an access-control list for each document—which users / groups / roles can read it. The difficulty lies in nested groups, inherited permissions, single-document overrides, and frequent change.
Immediate propagation: permission changes must be reflected in the index at once. After a user is removed from a channel, they should no longer find that channel's messages.
Pre-filter beats post-filter: shrink the candidate set with permission conditions at retrieval time; post-filter (retrieve then filter) can filter results down to zero when permissions are strict.
The title counts as content: displaying the title of a document a user cannot access is itself a permission violation.
Audit and protection: sensitive-information handling, query-log desensitization, output compliance filtering.

Multi-Tenant Isolation

SaaS scenarios serve multiple customers at once; there are three isolation strategies:

Strategy	Approach	Best for
Independent index	One index per tenant	Strong isolation, predictable query budget; first choice at SaaS scale
Shared index + tenant filter	Single index with a tenant_id field, every query must carry the filter	Low cost, but a missed filter means over-permission
Independent database + full-text search	One database per tenant (e.g. Postgres FTS)	Natural isolation, operations grow with tenant count

Which to choose depends on tenant count, data scale, compliance requirements, and operations capability. For the shared-index option, make the tenant filter a framework-level mandatory requirement, not something each caller remembers—one missed write is an over-permission.

Five Common Architecture Mistakes

Full rebuild on every change → switch to CDC + incremental, with full rebuild done weekly as routine.
No relevance tuning after launch → build an eval set, run evals, treat recall and ranking as continuous engineering, not one-time delivery.
Ignoring the tokenizer (analyzer) → non-English content like Chinese should not use the default tokenizer; mind tokenization granularity, stop words, synonyms, and language configuration.
Filtering after fetch → permission, deletion marks, tenant, and other conditions should be filtered at the engine layer; filtering after fetch is wasteful and may leak permission.
No backup → indexes are theoretically rebuildable, but replaying from CDC is time-consuming and degrades service meanwhile; take periodic snapshots.

Getting-Started Checklist (8 items)

Define boundaries first: which data sources to ingest, which business sides reuse, what the permission model is.
Drive the offline pipeline with CDC incremental; forbid full rebuild on every change; full rebuild is weekly routine.
Build three index types: inverted (exact) + vector (semantic) + structured (filter / aggregate).
The online pipeline must do hybrid retrieval (BM25 + vector + RRF fusion) and Reranker fine rank.
Query understanding must at least cover: correction, synonym expansion, entity recognition, intent classification.
Permission pre-filter + immediate propagation; titles are also bound by permission; desensitize logs.
Pick the isolation strategy by scale for multi-tenant; make the tenant filter a framework-level mandatory requirement for shared-index options.
Build an eval set and a monitoring dashboard (recall@k, zero-result rate, p95 latency, per-request cost), and iterate continuously.

The search platform ultimately delivers not "a search engine" but retrieval capability that multiple businesses can reuse. Query understanding, semantic recall, hybrid retrieval, permission awareness, and zero-result governance are exactly the core of enterprise search—see C10 Search Recall Improvement and C07 What Is RAG for details.

FAQ

Q: What is the difference between a search platform and each system building its own search?

A: The difference is reuse and consistency. Building separately leads to fragmented experience (the same term returns different results in different systems), duplicated effort (tokenization, synonyms, vectorization, and permission each done once), and runaway permissions. The platform unifies the indexing pipeline, query service, and permission system; business sides connect through APIs, quality iterates centrally, and one improvement benefits the whole line.

Q: Why must we use CDC incremental indexing?

A: Full rebuild is slow and disrupts service once data volume grows. CDC decouples "business write" from "index update"—after the app writes to the database, a change event drives the Worker to incrementally enrich and write to the index. The benefit is that a search-service failure does not block business writes, and rebuilding the index does not disturb the application. Full rebuild is done only weekly as routine.

Q: How are the two result paths of hybrid retrieval merged?

A: Usually with RRF (reciprocal rank fusion). The flow is keyword BM25 takes top 30, vector takes top 30, RRF fuses into top 30, then the Reranker cross-encoder fine-ranks to top 10 for display. Keywords guarantee exact terms are not lost, vectors fill the semantic gap, and reranking gives the final quality lift—this is the highest-yield step in the online pipeline.

Q: Should permission filtering go before or after retrieval?

A: It must go before retrieval. Use permission as a filter condition at the engine layer to shrink the candidate set; filtering after retrieval, when permissions are strict, can easily filter results down to zero and the user sees a blank directly. Also, permission changes must propagate to the index immediately, and the title of an inaccessible document must not be displayed—displaying the title itself is a violation.

Q: Is a shared index safe for multi-tenant?

A: It can be used, but the tenant filter must be a framework-level mandatory requirement (every query must carry tenant_id), not left to each caller's discipline—one missed write is an over-permission. For high-isolation or large-tenant-scale scenarios, an independent index is safer, at the cost of cost and operations. An independent-database option also works, with natural isolation but operations that grow with tenant count.

Q: Should we choose Elasticsearch or Postgres full-text search?

A: It depends on scale and needs. In 2026 most SaaS can be covered by Postgres FTS + pgvector (within tens of millions of documents, simple needs), saving a set of operations. Move to Elasticsearch / OpenSearch when you need log-scale full text, complex aggregation, or an existing ES stack. Mid-scale can also consider Typesense / Meilisearch. Decide by data volume, concurrency, and the team's operations capability.

About Tengence

The search platform is the key step for an enterprise to precipitate retrieval capability into infrastructure. Tengence Search provides query understanding, semantic recall, hybrid retrieval, reranking, and permission-aware retrieval, letting one platform serve multiple business scenarios. Tengence is the enterprise growth engine in the AI era, focused on public-traffic acquisition and private-domain conversion; its core products are Tengence GEO (dual-engine GEO + SEO) and Tengence Search.

Data Sources

System Design: Design Enterprise Search (Slack/Confluence)—Tech Interview: differences between enterprise and web search; connectors and incremental sync; ACL indexing difficulty and immediate propagation; five elements of query understanding; six classes of ranking signals (BM25 title 3x, freshness, organizational proximity, engagement, quality, semantic); hybrid retrieval RRF; pre/post permission filtering.
Enterprise Search—Baidu Baike: layered architecture (collection/ingestion, processing/indexing, search service, unified portal API + permission cross-cutting); Chinese tokenization and lexicon; document-level and collection-level security; knowledge-graph convergence trend.
2026 Enterprise AI Search Launch: From RAG Architecture to Permission, Evaluation, and Operations Loop—Ciyun Data: production-grade six major parts; nine-step online pipeline (gateway → understanding → multi-path recall → rerank → context → generation → attribution); seven-step offline pipeline; data-source type list.
Enterprise Intranet Smart Search: AI Search Private Deployment One-Click—Ciyun Data: private-deployment seven-layer architecture; component selection (Milvus/Qdrant/ES/OpenSearch/PostgreSQL/Redis/MinIO); model-service layer composition; Docker Compose vs K8s trade-offs.
Design a Search System—Elasticsearch, OpenSearch, and Postgres FTS in 2026—Manvendra Rajpoot: engine-selection comparison; CDC index pipeline and decoupling value; hybrid retrieval BM25 + vector + RRF + Rerank; layered ranking and learning-to-rank; three multi-tenant strategies; five common architecture mistakes.