How to Evaluate Recommendation Systems: CTR, CVR, Diversity and Novelty
Recommendation systems need core metrics: CTR for appeal, CVR for value, diversity and novelty vs drift, coverage for long-tail; offline iterates, online A/B.
Sep 28, 2026
Recommendation systems need core metrics: CTR for appeal, CVR for value, diversity and novelty vs drift, coverage for long-tail; offline iterates, online A/B.
Filter bubbles come from a three-way loop, not algorithms alone. Five fixes: hybrid recall, diversity re-ranking, exploration, user control, and monitoring.
Collaborative filtering: UserCF for content platforms, ItemCF for stable e-commerce. A 5-dimension comparison, co-occurrence and content-vector blending.
Recommendation ROI comes from scenarios, not one big model. We rank eight high-ROI e-commerce scenarios by rollout difficulty, with data and metrics for each.
Recommendation placement design: pick the page, content, and attention share. Match each slot to page intent and measure cannibalization to confirm real gains.
WeCom