From the course: Fundamentals of AI Engineering: Principles and Practical Applications
Join today to access over 24,300 courses taught by industry experts.
Building a hybrid retriever
From the course: Fundamentals of AI Engineering: Principles and Practical Applications
Building a hybrid retriever
- [Instructor] Welcome back. Today, we're going to build a hybrid retriever. To get started, open up Chapter_6, and open up the notebook titled 06_03.ipynb. In the upper right-hand corner of your notebook, make sure you've selected the .vn virtual environment. In our previous video, we compared BM25 and vector semantic search. Finding that each approach, has unique strengths and weaknesses. Namely, BM25 excels at finding exact match keywords, but misses semantic relationships, whereas vector search captures semantic meaning, but misses exact match terms. Today, we're going to combine these approaches to create a hybrid retriever that leverages the strengths of both methods. This is a common practice in production retrieval systems to enhance both precision and recall. Now, there are two main categories of hybrid retrieval techniques. The first are fusion approaches. These retrieve documents separately with each method and then combine the results. For example, a simple fusion…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.