Document Search in Practice: How Enterprises Actually Find Their Documents

Summary: The bottleneck in enterprise document search is rarely the retrieval algorithm — it is parsing, chunking and version handling before a document ever reaches the index. This article gives a six-step playbook in deployment order: decide per page whether OCR is needed, split by heading and paragraph structure instead of by character count, choose document-level versus chunk-level retrieval units, attach provenance metadata to every chunk, handle versions and freshness, then quantify recall with 50–100 representative questions before scaling.

When companies buy a document search system, their attention goes straight to "semantic retrieval" and "vector models". After go-live they discover the real problems: the rate table buried in a contract cannot be found, a scanned approval form cannot be found, the old and new versions of the same policy are mixed together, and a retrieved paragraph gives no clue which page it came from. The root cause of all of these converges on one place — the pipeline that turns a document into a retrievable unit. No matter how advanced the retrieval algorithm is, if you feed it shredded tables and multi-column body text with broken line wraps, the result will not be good.

Key Takeaways

The bottleneck in document search sits "before the index": parsing, chunking and metadata matter more for the final result than swapping in a different vector model.
Scanned pages must be judged per page, not per file — a real document is often digital body text paired with a scanned signature page.
Chunk by the structural boundaries of heading → paragraph → sentence, not by a hard character cap; tables must be kept as whole blocks.
Retrieval units come in two types, document-level and chunk-level; mixing the two is more useful than picking one, depending on whether the question is "find the file" or "find the answer".
Every chunk must carry doc_id, page number and section path, otherwise answers cannot be traced and will not be trusted.
Parsing quality is quantifiable: character error rate, layout F1 and table extraction accuracy each have a clear target value.

1. Why Document Search Is Hard: Documents Are Not Web Pages

Web pages are semi-structured — titles, body text and links are naturally segmented; documents are not. A single PDF can simultaneously contain paragraphs, tables, footnotes, page numbers, multi-column layouts, images and signature areas. Two documents that look nearly identical to a human can be completely different to a machine: one has a text layer that can be extracted directly, the other is a scanned image that extracts to an empty string.

The bigger complication is that document search must answer two fundamentally different kinds of questions. One is "find the file": pull up last quarter's supplier contract. The other is "find the answer": how many days is the approval window for travel reimbursement. The former needs a document-level holistic judgement; the latter needs chunk-level precise location. Satisfying both with one index is the starting point of document search design, and the reason most systems do it badly.

2. Parsing Is the First Gate: Layout, Scans and Tables

Parsing comes down to three core actions: preserve structure, detect scanned pages, and preserve tables.

Preserving structure means the extraction result is not a blob of flat text but typed elements — headings, paragraphs, list items and tables each kept separate, each carrying a page number and coordinates. Flat text strings multi-column body text into one left-to-right line, turns tables into space-separated gibberish, and bleeds headers and footers into every chunk.

Detecting scanned pages must be done per page, not per file. Real documents are often mixed: the body is digital, the signature page is a scan. After per-page extraction, count the characters on each page; a page with fewer than about 40 characters can be judged an image page and only those pages are sent to OCR. This avoids wasting compute recognising pages that already have a text layer, and avoids missing genuine scans.

Tables must be treated as atomic units. Tables are often the densest-information part of a document and the easiest to shred. The correct approach is to keep the whole table as one chunk; when it does not fit, split by row and repeat the header on every row — never cut across blocks.

For tooling, assign by document type: a simple digital file needs only a lightweight extractor; financial reports and structured documents with tables need a library that supports table grids; complex documents with mixed layouts need a parser that emits typed elements; long-tail formats such as PPT, spreadsheet files, legacy Office and email go to a general document-parsing service; scans get OCR on their own. On cost, at a scale of 10,000 pages per month, a cloud document-intelligence service lands in the low teens of dollars, a self-hosted open-source option is zero licensing fee plus GPU cost, and a hybrid route (open source for text, vision models for charts) is around $50.

Standard OCR is still 10 to 20 times faster and far cheaper than a vision model for ordinary text; the value of vision models concentrates on chart understanding and complex layouts. In 2026 the most effective combination is: OCR handles text, vision models handle only the charts and complex layouts that OCR cannot crack. Facing an old scan or fax, take the three-step path of preprocessing (deskew, denoise, binarize) → high-resolution OCR → model post-correction, and the error rate typically drops from 15–20% to 3–5%.

3. Chunking Sets Retrieval Granularity: Structure Before Length

Chunking packages the extracted elements into retrieval units. The most common mistake is a fixed character cap: it cuts sentences in half, separates a heading from the content it explains, and leaves the question in the previous block while the answer falls into the next.

The correct order is structure first, length as fallback: split on heading boundaries first, then paragraph boundaries, then sentence boundaries, and only use a hard length ceiling when nothing else works. In practice 300 to 800 tokens is the common range, with 10% to 20% (about 50–150 tokens) of overlap between adjacent chunks so that a sentence crossing a boundary can be hit from both sides.

One more time on tables: if a table fits in one chunk, keep it whole; if it does not, split by row and repeat the header. Before embedding, do two easily skipped steps — deterministically normalise the text so identical content yields identical bytes, then deduplicate, to avoid paying to embed the same paragraph twice and to stop an oversized chunk from diluting the vector and distorting similarity.

4. Document-Level or Chunk-Level: Mixing Beats Either/Or

A document-level retrieval unit is one vector per whole document, suited to "find the file": the user roughly remembers which material it was and wants it pulled out. The upside is complete semantics and low sensitivity to chunk noise; the downside is coarse location — even on a hit, you do not know which paragraph holds the answer.

A chunk-level retrieval unit is one vector per chunk, suited to "find the answer": the user wants that sentence itself. The upside is precise location and direct citation; the downside is that once a chunk is shredded, the semantics are incomplete.

The common production answer is two layers in parallel: use chunk-level retrieval to locate the precise fragment, while document-level retrieval supplies the overall candidates, then merge the two paths and re-rank. This way neither "find the file" nor "find the answer" comes up empty.

Hybrid retrieval pairs with it. For looking up a number, a proper noun or an exact term, keyword indexing's exact match is more reliable; for a natural-language question like "what are the rules for employee international travel", semantic retrieval better sidesteps wording differences. When querying the code POL-4827, the semantic model does not understand why that string matters, but keyword search pulls it out instantly. Merging the two recall paths before re-ranking is the standard configuration for document scenarios.

5. Metadata and Provenance: Make Answers Traceable to a Page

Every chunk needs at least five pieces of metadata: source file identifier, page number, section path, chunk sequence number and content hash. The first four support filtering and display on the retrieval side; the fifth supports idempotency — identical content hashes are not written twice, so rerunning the whole pipeline produces no duplicate data.

The value of provenance metadata is realised at the answer step. Only when an answer can be labelled "from page 7 of contract X" will users dare to use it directly; only when an audit can follow doc_id, page number and character offset back to the exact line of the source file will the system dare go to production. Without this layer, document search is just a demo that "looks like it found something".

The recommended data organisation is three layers: a raw layer storing source bytes and a file manifest, identified by file hash; a staging layer storing extracted pages and typed elements, enabling reprocessing without re-reading the source; a finished layer storing cleaned, chunked, deduplicated blocks, each carrying a content-derived chunk ID and provenance columns. Writes use idempotent upsert, so rerunning an unchanged file is a no-op.

6. Versions, Freshness and Permissions: Three Document-Specific Traps

Web search rarely worries about versions; document search must. The same policy can have six versions from V1.0 to V3.2, with old and new sitting in the library together, and the retrieval result ranks an abolished clause at the top — the most common document-search incident.

The fix is three things: write the version number, effective date and deprecation status into document-level metadata; at retrieval time filter by effective date or down-weight expired versions; and include version information in the chunk ID so that when a new version rebuilds the index, the old chunks are accurately replaced rather than stacked on top.

The second layer of freshness is "last modified". Many documents keep their content but get saved under a new name; the file hash changes, the content does not. Deduplicating on content hash rather than file hash avoids a great deal of needless re-embedding.

Permissions are the red line of document search. Retrieval results must never be visible beyond authority, and this must be solved on the index side rather than the display side — chunks carry an access control list and retrieval pre-filters. Get this layer wrong and go-live is an incident.

FAQ

Q: What is the biggest difference between document search and web search?
A: Web pages are naturally semi-structured, almost all digital text, and weak on versioning; documents have complex layouts, often scanned pages and tables, and version and permission are two hard constraints you cannot avoid. So document search must additionally do parsing judgement, chunking-granularity design, version management and permission filtering — four things that barely exist in web search.

Q: How big should a chunk be?
A: There is no universal value, but a usable starting point is 300 to 800 tokens with 10% to 20% overlap. What matters more is where you cut — prioritise the structural boundaries of heading, paragraph and sentence, and treat the length cap only as a fallback. Dense content like technical contracts and papers runs smaller; narrative content runs larger. In the end, measure recall with 50 to 100 representative questions rather than guessing.

Q: Why can tables never be found?
A: Two reasons. First, the parsing stage crushes the table into space-separated flat text and the row-column relationship is lost; second, the chunking stage cuts the table by character count and separates header from content. The corresponding fix is to switch to a parser that supports table grids and keep the table as an atomic chunk.

Q: Must scans be OCR'd in full?
A: No, and they should not be. Judge character count per page and send only the pages judged as image pages to OCR. Running OCR on digital body text wastes compute and can introduce recognition errors. Mixed-layout documents especially need per-page judgement, because judging a whole file either wastes effort or misses pages.

Q: How do you know if parsing quality is good enough?
A: Look at four metrics: character error rate within 5% to 10%, similarity score against the baseline text above 0.85, layout region detection F1 above 0.90, and chart data extraction accuracy above 0.80. Before go-live, sample 50 to 100 representative documents to validate; after full rollout, monitor continuously and alert when thresholds are exceeded. Good parsing typically lifts end-to-end results by 15% to 25%, cheaper than swapping the generation model.

Q: Should old-version documents be deleted outright?
A: Generally not physical deletion; instead mark them deprecated and filter or strongly down-weight them on the retrieval side. Historical archives and compliance audits often need old versions to remain queryable. The key is that version number, effective date and deprecation status must enter the metadata, and the chunk ID must include version information, so that when a new version rebuilds the index, old chunks are accurately replaced rather than stacked with the new.

About Tengence

The hard parts of document search concentrate in three pipeline stages — parsing, chunking and permissions — and any weak link propagates to the final result. Tengence Search provides layout parsing, structured chunking, hybrid retrieval, re-ranking and permission-aware search, so contracts, policies, tickets and knowledge-base documents can all be recalled reliably and traced back to a specific page. Tengence (TENGENCE) is an enterprise growth engine for the AI era, focused on public-traffic acquisition and private-domain conversion; its core products are Tengence GEO (dual-engine GEO + SEO) and Tengence Search.

Data Sources

Document Intelligence 2026: AI That Reads Your PDFs Better Than Humans (OCR, Tables, Charts) — Ailog: parsing quality target values (CER <5–10%, BLEU >0.85, layout F1 >0.90, charts >0.80); 10,000 pages/month cost comparison; OCR vs vision-model speed gap 10–20×; three-step repair of poor scans 15–20%→3–5%; table atomisation; parsing lifts end-to-end 15–25%.
Unstructured & Document Pipelines: PDFs, OCR & Text Extraction for the Warehouse — PipeCode: per-page scan judgement (about 40-character threshold); extractor assignment and Apache Tika for long-tail formats; three-layer data model with content-derived chunk_id idempotent upsert; normalisation and deduplication.
A Production RAG Pipeline for PDFs: Parsing, Chunking, Embeddings — Sistava: structure-first chunking order; 300–800 tokens with 10–20% overlap; five classes of chunk metadata; 50–100 questions for pre-evaluation.
How to Extract Data from PDF for AI / LLM Input — ScoutMyTool: extractor selection table (pypdf / pdfplumber / pdfminer.six / unstructured.io / PyMuPDF / Marker / Tesseract); OCR preprocessing and three causes of garbled output.
Engineering AI-Powered Document Intelligence: From PDFs to Semantic Search — Coder Legion: chunk metadata shape (document_id/page/section/chunk_index); loss of table relations; hybrid retrieval merging two recall paths before re-ranking; difference between code queries and semantic queries.