You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: units/en/bonus-unit2/monitoring-and-evaluating-agents-notebook.mdx
+2-3Lines changed: 2 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,9 +6,8 @@
6
6
7
7
# Bonus Unit 2: Observability and Evaluation of Agents
8
8
9
-
<Tip>
10
-
You can follow the code in <ahref="https://colab.research.google.com/#fileId=https%3A//huggingface.co/agents-course/notebooks/blob/main/bonus-unit2/monitoring-and-evaluating-agents.ipynb"target="_blank">this notebook</a> that you can run using Google Colab.
11
-
</Tip>
9
+
> [!TIP]
10
+
> You can follow the code in <ahref="https://colab.research.google.com/#fileId=https%3A//huggingface.co/agents-course/notebooks/blob/main/bonus-unit2/monitoring-and-evaluating-agents.ipynb"target="_blank">this notebook</a> that you can run using Google Colab.
12
11
13
12
In this notebook, we will learn how to **monitor the internal steps (traces) of our AI agent** and **evaluate its performance** using open-source observability tools.
Copy file name to clipboardExpand all lines: units/en/unit1/actions.mdx
+4-5Lines changed: 4 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,9 @@
1
1
# Actions: Enabling the Agent to Engage with Its Environment
2
2
3
-
<Tip>
4
-
In this section, we explore the concrete steps an AI agent takes to interact with its environment.
5
-
6
-
We’ll cover how actions are represented (using JSON or code), the importance of the stop and parse approach, and introduce different types of agents.
7
-
</Tip>
3
+
> [!TIP]
4
+
> In this section, we explore the concrete steps an AI agent takes to interact with its environment.
5
+
>
6
+
> We’ll cover how actions are represented (using JSON or code), the importance of the stop and parse approach, and introduce different types of agents.
8
7
9
8
Actions are the concrete steps an **AI agent takes to interact with its environment**.
Copy file name to clipboardExpand all lines: units/en/unit1/thoughts.mdx
+8-14Lines changed: 8 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,13 +1,10 @@
1
1
2
2
# Thought: Internal Reasoning and the ReAct Approach
3
3
4
-
<Tip>
5
-
6
-
In this section, we dive into the inner workings of an AI agent—its ability to reason and plan. We’ll explore how the agent leverages its internal dialogue to analyze information, break down complex problems into manageable steps, and decide what action to take next.
7
-
8
-
Additionally, we introduce the ReAct approach, a prompting technique that encourages the model to think “step by step” before acting.
9
-
10
-
</Tip>
4
+
> [!TIP]
5
+
> In this section, we dive into the inner workings of an AI agent—its ability to reason and plan. We’ll explore how the agent leverages its internal dialogue to analyze information, break down complex problems into manageable steps, and decide what action to take next.
6
+
>
7
+
> Additionally, we introduce the ReAct approach, a prompting technique that encourages the model to think “step by step” before acting.
11
8
12
9
Thoughts represent the **Agent's internal reasoning and planning processes** to solve the task.
13
10
@@ -85,10 +82,7 @@ Action: Finish["It's 18°C and cloudy in Paris."]
| Best suited for | Logic, math, internal tasks | Info-seeking, dynamic multi-step tasks |
87
84
88
-
<Tip>
89
-
90
-
Recent models like **Deepseek R1** or **OpenAI’s o1** were fine-tuned to *think before answering*. They use structured tokens like `<think>` and `</think>` to explicitly separate the reasoning phase from the final answer.
91
-
92
-
Unlike ReAct or CoT — which are prompting strategies — this is a **training-level technique**, where the model learns to think via examples.
93
-
94
-
</Tip>
85
+
> [!TIP]
86
+
> Recent models like **Deepseek R1** or **OpenAI’s o1** were fine-tuned to *think before answering*. They use structured tokens like `<think>` and `</think>` to explicitly separate the reasoning phase from the final answer.
87
+
>
88
+
> Unlike ReAct or CoT — which are prompting strategies — this is a **training-level technique**, where the model learns to think via examples.
Copy file name to clipboardExpand all lines: units/en/unit1/what-are-agents.mdx
+2-3Lines changed: 2 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -83,9 +83,8 @@ The most common AI model found in Agents is an LLM (Large Language Model), which
83
83
84
84
Well known examples are **GPT4** from **OpenAI**, **LLama** from **Meta**, **Gemini** from **Google**, etc. These models have been trained on a vast amount of text and are able to generalize well. We will learn more about LLMs in the [next section](what-are-llms).
85
85
86
-
<Tip>
87
-
It's also possible to use models that accept other inputs as the Agent's core model. For example, a Vision Language Model (VLM), which is like an LLM but also understands images as input. We'll focus on LLMs for now and will discuss other options later.
88
-
</Tip>
86
+
> [!TIP]
87
+
> It's also possible to use models that accept other inputs as the Agent's core model. For example, a Vision Language Model (VLM), which is like an LLM but also understands images as input. We'll focus on LLMs for now and will discuss other options later.
Copy file name to clipboardExpand all lines: units/en/unit1/what-are-llms.mdx
+2-5Lines changed: 2 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -118,11 +118,8 @@ The table below illustrates the diversity of special tokens.
118
118
</tbody>
119
119
</table>
120
120
121
-
<Tip>
122
-
123
-
We do not expect you to memorize these special tokens, but it is important to appreciate their diversity and the role they play in the text generation of LLMs. If you want to know more about special tokens, you can check out the configuration of the model in its Hub repository. For example, you can find the special tokens of the SmolLM2 model in its <ahref="https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct/blob/main/tokenizer_config.json">tokenizer_config.json</a>.
124
-
125
-
</Tip>
121
+
> [!TIP]
122
+
> We do not expect you to memorize these special tokens, but it is important to appreciate their diversity and the role they play in the text generation of LLMs. If you want to know more about special tokens, you can check out the configuration of the model in its Hub repository. For example, you can find the special tokens of the SmolLM2 model in its <ahref="https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct/blob/main/tokenizer_config.json">tokenizer_config.json</a>.
You can follow the code in <ahref="https://huggingface.co/agents-course/notebooks/blob/main/unit2/langgraph/agent.ipynb"target="_blank">this notebook</a> that you can run using Google Colab.
23
-
</Tip>
21
+
> [!TIP]
22
+
> You can follow the code in <ahref="https://huggingface.co/agents-course/notebooks/blob/main/unit2/langgraph/agent.ipynb"target="_blank">this notebook</a> that you can run using Google Colab.
Copy file name to clipboardExpand all lines: units/en/unit2/langgraph/first_graph.mdx
+2-3Lines changed: 2 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,9 +9,8 @@ Now that we understand the building blocks, let's put them into practice by buil
9
9
10
10
This example demonstrates how to structure a workflow with LangGraph that involves LLM-based decision-making. While this can't be considered an Agent as no tool is involved, this section focuses more on learning the LangGraph framework than Agents.
11
11
12
-
<Tip>
13
-
You can follow the code in <ahref="https://huggingface.co/agents-course/notebooks/blob/main/unit2/langgraph/mail_sorting.ipynb"target="_blank">this notebook</a> that you can run using Google Colab.
14
-
</Tip>
12
+
> [!TIP]
13
+
> You can follow the code in <ahref="https://huggingface.co/agents-course/notebooks/blob/main/unit2/langgraph/mail_sorting.ipynb"target="_blank">this notebook</a> that you can run using Google Colab.
Copy file name to clipboardExpand all lines: units/en/unit2/langgraph/introduction.mdx
+2-3Lines changed: 2 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,9 +16,8 @@ In this unit, you'll discover:
16
16
### 4️⃣ [Alfred, the document Analyst agent](./document_analysis_agent)
17
17
### 5️⃣ [Quiz](./quizz1)
18
18
19
-
<Tipwarning={true}>
20
-
The examples in this section require access to a powerful LLM/VLM model. We ran them using the GPT-4o API because it has the best compatibility with langGraph.
21
-
</Tip>
19
+
> [!WARNING]
20
+
> The examples in this section require access to a powerful LLM/VLM model. We ran them using the GPT-4o API because it has the best compatibility with langGraph.
22
21
23
22
By the end of this unit, you'll be equipped to build robust, organized and production ready applications !
Copy file name to clipboardExpand all lines: units/en/unit2/llama-index/agents.mdx
+6-5Lines changed: 6 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,13 +16,13 @@ LlamaIndex supports **three main types of reasoning agents:**
16
16
2.`ReAct Agents` - These can work with any AI that does chat or text endpoint and deal with complex reasoning tasks.
17
17
3.`Advanced Custom Agents` - These use more complex methods to deal with more complex tasks and workflows.
18
18
19
-
<Tip>Find more information on advanced agents on <ahref="https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/agent/workflow/base_agent.py">BaseWorkflowAgent</a></Tip>
19
+
> [!TIP]
20
+
> Find more information on advanced agents on <ahref="https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/agent/workflow/base_agent.py">BaseWorkflowAgent</a>
20
21
21
22
## Initialising Agents
22
23
23
-
<Tip>
24
-
You can follow the code in <ahref="https://huggingface.co/agents-course/notebooks/blob/main/unit2/llama-index/agents.ipynb"target="_blank">this notebook</a> that you can run using Google Colab.
25
-
</Tip>
24
+
> [!TIP]
25
+
> You can follow the code in <ahref="https://huggingface.co/agents-course/notebooks/blob/main/unit2/llama-index/agents.ipynb"target="_blank">this notebook</a> that you can run using Google Colab.
26
26
27
27
To create an agent, we start by providing it with a **set of functions/tools that define its capabilities**.
28
28
Let's look at how to create an agent with some basic tools. As of this writing, the agent will automatically use the function calling API (if available), or a standard ReAct agent loop.
@@ -155,6 +155,7 @@ agent = AgentWorkflow(
155
155
response =await agent.run(user_msg="Can you add 5 and 3?")
156
156
```
157
157
158
-
<Tip>Haven't learned enough yet? There is a lot more to discover about agents and tools in LlamaIndex within the <ahref="https://docs.llamaindex.ai/en/stable/examples/agent/agent_workflow_basic/">AgentWorkflow Basic Introduction</a> or the <ahref="https://docs.llamaindex.ai/en/stable/understanding/agent/">Agent Learning Guide</a>, where you can read more about streaming, context serialization, and human-in-the-loop!</Tip>
158
+
> [!TIP]
159
+
> Haven't learned enough yet? There is a lot more to discover about agents and tools in LlamaIndex within the <ahref="https://docs.llamaindex.ai/en/stable/examples/agent/agent_workflow_basic/">AgentWorkflow Basic Introduction</a> or the <ahref="https://docs.llamaindex.ai/en/stable/understanding/agent/">Agent Learning Guide</a>, where you can read more about streaming, context serialization, and human-in-the-loop!
159
160
160
161
Now that we understand the basics of agents and tools in LlamaIndex, let's see how we can use LlamaIndex to **create configurable and manageable workflows!**
Copy file name to clipboardExpand all lines: units/en/unit2/llama-index/components.mdx
+12-8Lines changed: 12 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -27,9 +27,8 @@ Now, let's dive a bit deeper into the components and see how you can **combine c
27
27
28
28
## Creating a RAG pipeline using components
29
29
30
-
<Tip>
31
-
You can follow the code in <ahref="https://huggingface.co/agents-course/notebooks/blob/main/unit2/llama-index/components.ipynb"target="_blank">this notebook</a> that you can run using Google Colab.
32
-
</Tip>
30
+
> [!TIP]
31
+
> You can follow the code in <ahref="https://huggingface.co/agents-course/notebooks/blob/main/unit2/llama-index/components.ipynb"target="_blank">this notebook</a> that you can run using Google Colab.
33
32
34
33
There are five key stages within RAG, which in turn will be a part of most larger applications you build. These are:
35
34
@@ -50,7 +49,8 @@ There are three main ways to load data into LlamaIndex:
50
49
2.`LlamaParse`: LlamaParse, LlamaIndex's official tool for PDF parsing, available as a managed API.
51
50
3.`LlamaHub`: A registry of hundreds of data-loading libraries to ingest data from any source.
52
51
53
-
<Tip>Get familiar with <ahref="https://docs.llamaindex.ai/en/stable/module_guides/loading/connector/">LlamaHub</a> loaders and <ahref="https://github.com/run-llama/llama_cloud_services/blob/main/parse.md">LlamaParse</a> parser for more complex data sources.</Tip>
52
+
> [!TIP]
53
+
> Get familiar with <ahref="https://docs.llamaindex.ai/en/stable/module_guides/loading/connector/">LlamaHub</a> loaders and <ahref="https://github.com/run-llama/llama_cloud_services/blob/main/parse.md">LlamaParse</a> parser for more complex data sources.
54
54
55
55
**The simplest way to load data is with `SimpleDirectoryReader`.**
56
56
This versatile component can load various file types from a folder and convert them into `Document` objects that LlamaIndex can work with.
@@ -124,7 +124,8 @@ pipeline = IngestionPipeline(
124
124
)
125
125
```
126
126
127
-
<Tip>An overview of the different vector stores can be found in the <ahref="https://docs.llamaindex.ai/en/stable/module_guides/storing/vector_stores/">LlamaIndex documentation</a>.</Tip>
127
+
> [!TIP]
128
+
> An overview of the different vector stores can be found in the <ahref="https://docs.llamaindex.ai/en/stable/module_guides/storing/vector_stores/">LlamaIndex documentation</a>.
128
129
129
130
130
131
This is where vector embeddings come in - by embedding both the query and nodes in the same vector space, we can find relevant matches.
@@ -176,7 +177,8 @@ Once again, this is fully customisable but there are three main strategies that
176
177
-`compact` (default): similar to refining but concatenating the chunks beforehand, resulting in fewer LLM calls.
177
178
-`tree_summarize`: create a detailed answer by going through each retrieved text chunk and creating a tree structure of the answer.
178
179
179
-
<Tip>Take fine-grained control of your query workflows with the <ahref="https://docs.llamaindex.ai/en/stable/module_guides/deploying/query_engine/usage_pattern/#low-level-composition-api">low-level composition API</a>. This API lets you customize and fine-tune every step of the query process to match your exact needs, which also pairs great with <ahref="https://docs.llamaindex.ai/en/stable/module_guides/workflow/">Workflows</a> </Tip>
180
+
> [!TIP]
181
+
> Take fine-grained control of your query workflows with the <ahref="https://docs.llamaindex.ai/en/stable/module_guides/deploying/query_engine/usage_pattern/#low-level-composition-api">low-level composition API</a>. This API lets you customize and fine-tune every step of the query process to match your exact needs, which also pairs great with <ahref="https://docs.llamaindex.ai/en/stable/module_guides/workflow/">Workflows</a>
180
182
181
183
The language model won't always perform in predictable ways, so we can't be sure that the answer we get is always correct. We can deal with this by **evaluating the quality of the answer**.
182
184
@@ -190,7 +192,8 @@ Let's look at the three main evaluators available:
190
192
-`AnswerRelevancyEvaluator`: Evaluate the relevance of the answer by checking if the answer is relevant to the question.
191
193
-`CorrectnessEvaluator`: Evaluate the correctness of the answer by checking if the answer is correct.
192
194
193
-
<Tip>Want to learn more about agent observability and evaluation? Continue your journey with the <ahref="https://huggingface.co/learn/agents-course/bonus-unit2/introduction">Bonus Unit 2</a>.</Tip>
195
+
> [!TIP]
196
+
> Want to learn more about agent observability and evaluation? Continue your journey with the <ahref="https://huggingface.co/learn/agents-course/bonus-unit2/introduction">Bonus Unit 2</a>.
194
197
195
198
```python
196
199
from llama_index.core.evaluation import FaithfulnessEvaluator
<Tip>Want to learn more about components and how to use them? Continue your journey with the <ahref="https://docs.llamaindex.ai/en/stable/module_guides/">Components Guides</a> or the <ahref="https://docs.llamaindex.ai/en/stable/understanding/rag/">Guide on RAG</a>.</Tip>
244
+
> [!TIP]
245
+
> Want to learn more about components and how to use them? Continue your journey with the <ahref="https://docs.llamaindex.ai/en/stable/module_guides/">Components Guides</a> or the <ahref="https://docs.llamaindex.ai/en/stable/understanding/rag/">Guide on RAG</a>.
242
246
243
247
We have seen how to use components to create a `QueryEngine`. Now, let's see how we can **use the `QueryEngine` as a tool for an agent!**
0 commit comments