GenAI Fundamentals: Implementing RAG with Azure Search & OpenAI
A technical journey through Retrieval Augmented Generation

by Ankit Shivam

Generative AI Fundamentals
Large Language Models
Trained on vast amount of to predict next tokens
Content Creation
Generate human-like text, images, code on demand
Limitations
Hallucinations, outdated knowledge, no source attribution , does not know your data
Retrieval Augmented Generation (RAG)
Retrieve
Find relevant documents from knowledge base
Augment
Add context to user's query
Generate
LLM creates response using retrieved information
Basic RAG Pattern
Searching for Answers: Why Traditional Search Falls Short?
Traditional search engines struggle with meaning beyond exact words:
User Query
"How do I Reset my Password?"
Available Document
"Steps to recover your password access"
The Problem
  • Traditional keyword search looks for exact words
  • It might miss relevant content if it doesn't match keywords exactly
We Need More Than Just Key Words
Need something that can capture
  • The intent behind the question
  • The similarity in meaning - even if the words are different
Vector embeddings turn meaning into math
They convert words and concepts into numerical representations that capture semantic relationships
Understanding Vector Embeddings
Understanding Meaning From Numbers
1
Words to Numbers
  • System doesn't understand words - it understands numbers.
  • It turns text into list of numbers(vector).
2
Mapping Meanings
  • Similar words and ideas get mapped to similar numbers in the vector space.
  • "Reset password" and "recover account" become close together.
3
Helps RAG find the right document
  • It doesn't search by keyword -it serches by meaning
Core Components of RAG System
Storage
Where documents live (e.g., PDFs in Azure Blob Storage)
Embedding Model
Converts text into vectors (e.g., OpenAI’s Ada-002)
Search & Indexing Service
Finds relevant content using keyword and vector search (e.g., Azure AI Search)
Large Language Model (LLM)
Generates response using retrieved content (e.g , GPT-4)
Orchestrator
Puts all pieces together to serve end user request (e.g., Azure Functions, AI Foundary)
Next Steps & Demo Preparation
Environment Setup
  • Storage Account
  • Azure Open AI Resource (ADA-002, GPT)
  • Azure AI Search
  • Azure AI Foundary
Data Preparation
  • Various meeting transcripts for last 1 year
Demo Scenarios
  • A summarizer assistant that reviews past meetings, keeps track of action items, and offers additional features
Summary

Data Sources

Azure AI Search

Azure Open AI (ADA-002)

Azure Open AI (GPT)

What are going to build or help build next ?