glossary terms
Cosine Similarity
- Category
- RAG, Embeddings & Search
- Difficulty
- Intermediate
Definition
A mathematical metric used to measure the cosine of the angle between two non-zero vectors in a multi-dimensional space, determining how similar their orientations are regardless of their magnitude.
How It Works and Context
In the context of AI and machine learning, data such as text, images, or audio are converted into numerical vectors (embeddings). Cosine similarity evaluates the relationship between these vectors by calculating the cosine of the angle between them. A value of 1 indicates that the vectors point in the exact same direction (perfect similarity), 0 indicates they are orthogonal (no correlation), and -1 indicates they are diametrically opposed. Unlike Euclidean distance, which measures the straight-line distance between points, cosine similarity focuses exclusively on the orientation of the vectors. This makes it particularly effective for high-dimensional data where the absolute magnitude of the vector might be less important than the underlying semantic pattern or 'direction' of the information.
Why It Matters
It allows AI models to retrieve relevant documents or context by comparing the vector of a user's query against a database of pre-computed document embeddings.
Real-world Example
Imagine a RAG system for a legal database. When a user asks, 'What are the penalties for contract breach?', the system converts this query into a vector. It then calculates the cosine similarity between this query vector and thousands of legal document vectors. The system retrieves the documents with the highest cosine similarity scores, ensuring the AI receives the most semantically relevant legal text to generate an accurate, evidence-based answer.
Common Mistakes
- Confusing cosine similarity with Euclidean distance; the former ignores magnitude, while the latter is sensitive to it.
- Assuming that a high cosine similarity score always implies factual correctness, rather than just semantic proximity.
- Neglecting to normalize vectors when using other distance metrics, which can lead to incorrect similarity calculations.
- Applying cosine similarity to sparse data without considering the impact of dimensionality on the resulting angle.
Frequently Asked Questions
When should I use cosine similarity instead of Euclidean distance?
Use cosine similarity when the semantic orientation of the data is more important than the absolute magnitude, such as in text classification or document retrieval. Use Euclidean distance when the actual distance or 'size' of the data points is a critical feature.
Does cosine similarity work with negative values?
Yes, cosine similarity can range from -1 to 1. A negative value indicates that the vectors are pointing in opposite directions, which can be useful in specific recommendation scenarios where you want to identify items that are the 'opposite' of a user's preference.
Is cosine similarity computationally expensive?
It is generally efficient, especially when vectors are normalized to unit length. In that case, the cosine similarity is equivalent to the dot product, which is a highly optimized operation in modern linear algebra libraries and GPU-accelerated frameworks.