Vector Search with Azure SQL Database

With the public preview of integrated vectorization, a groundbreaking capability of vector search in Azure AI Search (previously Azure Cognitive Search), you can easily perform vector search with data stored in Azure SQL Database. This feature aims to streamline the process of chunking, generating, storing, and querying vectors for vector search in Azure AI Search.

Key Concepts in Integrated Vectorization

Vector Search:

In Azure AI Search, vector search refers to the capability of indexing, storing, and retrieving vector embeddings from a search index.

Example with Data Stored in Azure SQL Database

Let's create an example of an AI application that responds to user queries based on the Azure SQL DB table of Amazon product reviews. The expected behavior would be something like:

[User search]: Canned dog food [AI Response]: After searching through our product database, I recommend <product ID> because...

Behind the scenes, the following steps are taken:

1. Set up a sample table in SQL DB and upload data to it.
2. Set up an indexer in Azure AI Search to pull data into the index.
3. Automatically chunk and vectorize the data using an Azure OpenAI Embedding service.
4. Use Azure AI Search to process the user's query and search for the most relevant data.

You can find detailed documentation here.

Chunk, Vectorize, Configure, and Store Data

To create an index and configure integrated vectorization, follow these steps:

1. Take the combined text (summary + review text) from each product review.
2. Create an indexer that uses a skillset to pull data from the SQL Database, separate the text into chunks, vectorize each chunk, and store all the necessary data in the index.

Example search result:

Search Score: 0.88524085
Parent Id: 1 | Chunk Id: f59640a3248d_1_pages_0
Product Id: B001E4KFG0
Text Chunk: 
Summary: Good Quality Dog Food 
Review: I have bought several of the Vitality canned dog food products and have found them all to be of good quality. 

Review Summary: Good Quality Dog Food 
Review Text: I have bought several of the Vitality canned dog food products and have found them all to be of good quality. I came home and to my surprise realized that I could save $20 each time I bought dog food if I just buy it off Amazon.<br /><br />All in all, I definitely recommend and give my stamp of approval to natural balance dog food. I mix her dry food with a little wet food and my golden loves it. I came home and to my surprise realized that I could save $20 each time I bought dog food if I just buy it off Amazon.<br /><br />All in all, I definitely recommend and give my stamp of approval to natural balance dog food. 

Review Score: 5

Search Score: 0.8640232
Parent Id: 98 | Chunk Id: 0bc498329489_98_pages_0
Product Id: B0019CW0HE
Text Chunk: 
Summary: Great allergy sensitive dog food, dogs love it 
Review: Our pup has experienced allergies in forms of hotspots and itching from other dog foods. 

Review Summary: Great allergy sensitive dog food, dogs love it 
Review Text: Our pup has experienced allergies in forms of hotspots and itching from other dog foods.