Beginner1h55m

Building and Evaluating Advanced RAG

Instructors: Jerry Liu, Anupam Datta

Earn an accomplishment with PRO · Updated Oct 7, 2025

Enroll for Free

Beginner
1h55m
6 Video Lessons
4 Code Examples
Earn an accomplishment with PRO
Updated Oct 7, 2025
Instructors: Jerry Liu, Anupam Datta
TruEra and LlamaIndex
Learn more aboutMembership PRO Plan

Enroll for Free

What you'll learn

Learn methods like sentence-window retrieval and auto-merging retrieval, improving your RAG pipeline’s performance beyond the baseline.
Learn evaluation best practices to streamline your process, and iteratively build a robust system.
Dive into the RAG triad for evaluating the relevance and truthfulness of an LLM’s response:Context Relevance, Groundedness, and Answer Relevance.

Course recap

PRO

This course, taught by Jerry Liu (LlamaIndex) and Anupam Datta (TruEra), covers two advanced retrieval methods -- sentence window retrieval and auto-merging retrieval -- that significantly improve context quality over basic RAG pipelines. It pairs these with the RAG triad evaluation framework (context relevance, groundedness, answer relevance) to enable systematic iteration and improvement.

Concept map

Key concepts

Retrieval-Augmented GenerationCore pattern of retrieving context from external data to ground LLM responses
EmbeddingsVector representations used for similarity-based retrieval; BGE models used for both embedding and re-ranking
RAG triadThree evaluation metrics: context relevance (is retrieved context relevant to query), groundedness (is response supported by context), answer relevance (is response relevant to query)
Sentence window retrievalEmbed individual sentences but expand to surrounding window for LLM context
Auto-merging retrievalHierarchical node structure where child nodes merge into parent when majority are retrieved

Lesson highlights

1.Introduction — Overview of advanced RAG and evaluation framework
2.Advanced RAG Pipeline — Setting up basic and advanced pipelines with LlamaIndex and TruLens benchmarks
3.RAG Triad of Metrics — Deep dive into context relevance, groundedness, and answer relevance feedback functions
4.Sentence-window Retrieval — Detailed implementation with node parser, metadata replacement, and re-ranking
5.Auto-merging Retrieval — Hierarchical node parsing, merging heuristic, and parameter iteration
6.Conclusion — Key takeaways and next steps

About this course

Retrieval Augmented Generation (RAG) stands out as one of the most popular use cases of large language models (LLMs). This method facilitates the integration of an LLM with an organization’s proprietary data.

To successfully implement RAG, it is essential to enhance retrieval techniques for obtaining coherent contexts and employ effective evaluation metrics.

In this course, we’ll explore:

Two advanced retrieval methods: Sentence-window retrieval and auto-merging retrieval that perform better compared to the baseline RAG pipeline.
Evaluation and experiment tracking: A way evaluate and iteratively improve your RAG pipeline’s performance.
The RAG triad: Context Relevance, Groundedness, and Answer Relevance, which are methods to evaluate the relevance and truthfulness of your LLM’s response.