Hybrid Search and Retrieval Evaluation

you own this product
prerequisites
intermediate Python • basics of retrieval and embeddings • basic understanding of evaluation metrics (precision/recall) helpful but not required • Project 1 recommended but not required (standalone setup provided)
skills learned
BM25 keyword search • RRF hybrid ranking • retrieval metrics (precision, recall, MRR, NDCG) • comparing retrieval strategies (semantic vs. keyword vs. hybrid) • basic answer-quality scoring (relevance and groundedness)
Matteus Tanha
1 week · 6-8 hours per week · INTERMEDIATE
Included with a Manning Online subscription

pro $24.99 per month

  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose one free eBook per month to keep
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime

lite $19.99 per month

  • access to all Manning books, including MEAPs!

team

5, 10 or 20 seats+ for your team - learn more

Look inside

Semantic search understands meaning but fumbles exact terms. Keyword search nails specifics but misses nuance. What if you didn’t have to choose? In this liveProject, you’ll supercharge the retrieval system by fusing both into a smarter hybrid search engine. You’ll implement the battle-tested BM25 algorithm, and fuse their rankings with reciprocal rank fusion (RRF) into a single hybrid ranking. Then you'll prove it works by building a test set and measuring precision, recall, MRR, NDCG, and answer-quality metrics like relevance and groundedness across all three strategies.

This project is a part of the series Building an Agentic RAG Application.
This project is designed for learning purposes and is not a complete, production-ready application or solution.

project author

Matteus Tanha
Dr. Matteus Tanha is an AI engineer and architect with over a decade of experience building production machine learning and agentic AI systems. He is co-founder of Alpha Quants, a boutique AI consultancy serving finance and enterprise clients, and has led AI initiatives at organizations including the Financial Times and Zurich Insurance. At the Financial Times, he architected AskFT, a retrieval-augmented research assistant combining semantic search and LLM orchestration to serve over a million monthly users. His work spans hybrid retrieval systems, knowledge graphs, and multi-agent orchestration, with deep expertise in RAG architectures and vector and graph databases. Matteus holds a Ph.D. in Computational Chemistry from Carnegie Mellon University, where his research applied machine learning methods to quantum chemical computation.

prerequisites

This liveProject is for learners who want to add hybrid search and retrieval evaluation to a RAG-style pipeline.


TOOLS
  • Python (intermediate)
  • Jupyter Notebooks (basics)
  • Command line / Terminal (basics)
TECHNIQUES
  • Machine Learning Fundamentals
  • Database Concepts
  • Search and retrieval fundamentals
  • Evaluation metrics

features

Self-paced
You choose the schedule and decide how much time to invest as you build your project.
Project roadmap
Each project is divided into several achievable steps.
Get Help
While within the liveProject platform, get help from fellow participants and even more help with paid sessions with our expert mentors.
Compare with others
For each step, compare your deliverable to the solutions by the author and other participants.
book resources
Get full access to select books for 90 days. Permanent access to excerpts from Manning products are also included, as well as references to other resources.
choose your plan

pro

monthly
annual
$24.99
$249.99
only $20.83 per month
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Hybrid Search and Retrieval Evaluation project for free

team

monthly
annual
$49.99
$499.99
only $41.67 per month
  • five seats for your team
  • access to all Manning books, MEAPs, liveVideos, liveProjects, and audiobooks!
  • choose another free product every time you renew
  • choose twelve free products per year
  • exclusive 50% discount on all purchases
  • renews monthly, pause or cancel renewal anytime
  • renews annually, pause or cancel renewal anytime
  • Hybrid Search and Retrieval Evaluation project for free