From the course: Fundamentals of AI Engineering: Principles and Practical Applications

Introduction to embeddings

From the course: Fundamentals of AI Engineering: Principles and Practical Applications

Start my 1-month free trial Buy for my team

Introduction to embeddings

“

- [Instructor] Welcome back. Let's talk about one of the most important concepts in modern AI, embeddings. I'll explain what embeddings are, why they're so powerful, and how they're transforming the way we build AI systems. At their core, embeddings are just lists of numbers, but these aren't random numbers. They're carefully constructed numerical representations that capture the meaning of whatever they represent. Think about how challenging it is to get computers to understand language. Words like happy and joyful mean similar things, but to a computer, they're completely different strings of characters. Embedding solves this problem by converting words, sentences, or entire documents into points in a high-dimensional space where similar items are positioned together. This allows AI systems to reason about meaning rather than just exact matching words. Embeddings have revolutionized AI by allowing machines to understand concepts in a way that's eerily similar to how humans think. When we think of dog, we automatically associate it with concepts like pet, animal, loyal, and so on. Good embeddings capture these same associations mathematically. This capability powers many AI applications that you use daily, semantic search engines that understand your query intent, recommendation systems that find similar products or content, and AI assistants that grasp the context of your questions. Embeddings are the foundation that makes all of this possible. So how do embeddings actually work? Modern embeddings are created by neural networks trained on massive datasets to learn the relationship between items. For example, these models might analyze billions of sentences to learn that king is to queen as man is to woman. Each embedding is a vector in a high-dimensional space, typically hundreds of dimensions. We can measure how similar two items are by calculating the distance between their vectors or the angle between them. Smaller distances or angles mean greater similarity. While we can't visualize hundreds of dimensions, we can project embeddings to 2D or 3D to get a glimpse of how they organize information with similar concepts clustered together. Embeddings aren't just for text. We can create embeddings for virtually any type of data, images, audio, video, user behavior, and more. Each type captures the semantic essence of its domain. In this series, we'll focus primarily on text embeddings, which have seen remarkable advances in recent years. We'll explore models like sentence transformers that can convert entire sentences or documents into powerful embeddings. We'll also touch on domain-specific embeddings, which are fine-tuned for particular fields, like medicine or law, and even multimodal embeddings that can connect concepts across different types of data. Building with embeddings involves an entire ecosystem of tools and techniques. First, you need embeddings models that convert your raw data into vectors. Then, vector databases to store and search these embeddings efficiently at scale. Finally, you build applications on top of this foundation. Let's explore each part of this ecosystem, starting with embeddings models powered by sentence transformers, then moving on to vector databases like Chroma, and finally, showing how to combine them into powerful applications. Embeddings are transforming the way we build AI systems by enabling machines to understand meaning in ways that were impossible just a few years ago. As AI engineers, mastering embeddings will allow you to build more intelligent, more intuitive, and more user-friendly applications that truly understand your users and their needs.

Contents