How to Run A/B Tests for Site Search: Metric Design and Practice
A/B test site search changes: split into recall, ranking, and result-page layers; search conversion rate primary, zero-result and first-click rates guardrails.
Sep 27, 2026
A/B test site search changes: split into recall, ranking, and result-page layers; search conversion rate primary, zero-result and first-click rates guardrails.
How to calculate search conversion rate: fix the numerator and denominator, read usage, zero-result rate and top-result CTR, then improve recall and ranking.
Build, open source Elasticsearch or SaaS? Score six dimensions: capability, staffing, license risk, scaling, compliance, three-year TCO, then pick by scale.
Cross-border search is not fixed by translating queries: tokenization, morphology and word order shift recall. Compare three architectures, analyzers, ranking.
Tengence Intelligent Search fixes four on-site search failures with query understanding, keyword and vector recall, fused ranking, rules and search analytics.
Build or buy search? In-house needs 6-8 people and 10-15 months to launch, plus 0.5-1.0 FTE. Compare Tengence Search SaaS on cost, timeline, and results.
Search relevance problems split into recall vs. ranking failures: offline replay, multi-path recall, two-stage ranking, Learning to Rank, and a 3-phase rollout.
Do not merge search and recommendation. Share the data and recall base, split ranking and strategy, then roll out in four phases with clear engineering metrics.
Semantic search matches meaning with vectors, keyword search matches wording. See how each works, where each fails, and why hybrid search is the default.
Cut site-search zero-results rate below 2% in eight weeks: rank queries by frequency, then close synonym, spelling, and index gaps with eight strategies.
WeCom