Announcing EAP for Vector Support in Azure SQL Database

We are super excited to announce the Early Adopter Preview of Native Vector Support in Azure SQL Database and are currently accepting requests from customers who wish to participate.

SQL beyond RDBMS

Vector databases have gained a lot of prominence lately. These specialized repositories handle vector data, crucial for applications like semantic search, chatbots, and recommendation systems.

However, a paradigm shift is underway: Why maintain a separate Vector database when Azure SQL Database can seamlessly accommodate vector embeddings?

By integrating vector search into Azure SQL, you simplify application development, coexisting with operational data for efficient similarity searches, joins, and aggregations—all while leveraging Azure SQL’s sophisticated query optimizer and robust enterprise features. The integration of vector search within RDBMS allows customers to perform vector similarity searches alongside traditional SQL queries, enhancing data analysis and decision-making.

Embeddings

Embedding is the mathematical representation of objects (such as words, paragraphs, documents, images, or audio) as dense vectors. These vectors are typically generated by deep learning models and are instrumental in various machine learning and AI applications. By translating data into a vector space, embeddings can effectively capture and quantify the semantic similarities between related concepts, facilitating tasks such as semantic search and natural language processing.

Embeddings capture semantic similarity between similar concepts. For example, consider word embeddings: Clusters of related words emerge (e.g., “computer,” “software,” “machine” in one cluster; “lion,” “cow,” “cat,” “dog” in another). The gap between clusters highlights dissimilarity in meaning or context

The real complex part is calculating the embeddings, but thanks to Azure OpenAI, everyone has an easily accessible REST service that can be used to get the embeddings using pre-trained ML models. Once the embeddings are generated, they can be stored into a SQL Server database. This allows you to store the embeddings alongside the data they represent, and to perform vector search queries to find similar data points.

A great article to understand how embeddings work, is the following: Introducing text and code embeddings.

Vector Search Use Cases

Vector search refers to the process of finding all vectors in a dataset that are similar to a specific query vector. Therefore, a query vector for the word human searches the entire dataset for similar vectors, and thus similar words: in this example it should find the word person as a close match. This closeness, or distance, is measured using a distance metric such as cosine distance