diff --git a/17-multi-turn/README.md b/17-multi-turn/README.md new file mode 100644 index 0000000..d77532b --- /dev/null +++ b/17-multi-turn/README.md @@ -0,0 +1,149 @@ +# Multi-Turn vs Single-Turn Agent Testing + +This example demonstrates the differences between multi-turn and single-turn agents in Contextual AI through comprehensive testing scenarios. + +## Overview + +The notebook creates two agents with identical configurations except for their multi-turn settings: +- **MultiTurnAgent**: Maintains conversation context across multiple messages +- **SingleTurnAgent**: Treats each message independently without memory + +## What You'll Learn + +- How to enable/disable multi-turn functionality in Contextual AI agents +- The difference in behavior between multi-turn and single-turn agents +- How to use `conversation_id` to continue previous conversations +- Best practices for testing agent memory capabilities + +## Key Concepts + +### Multi-Turn Agents +- Remember conversation history using `conversation_id` +- Can reference previous messages and maintain context +- Ideal for conversational interfaces and complex interactions + +### Single-Turn Agents +- Process each message independently +- No memory of previous interactions +- Suitable for stateless operations and simple Q&A + +### Conversation Management +- The first message in a conversation generates a `conversation_id` +- Subsequent messages use this ID to maintain context +- Conversations can be resumed later using the same ID + +## Test Methodology + +The notebook follows a systematic approach: + +1. **Setup**: Creates two identical agents with different multi-turn settings +2. **Initial Facts**: Shares the same information with both agents +3. **Memory Testing**: Asks follow-up questions to test retention +4. **Comparison**: Validates expected vs actual behavior +5. **Recurring Conversations**: Tests conversation continuity + +### Test Scenarios + +| Scenario | MultiTurnAgent | SingleTurnAgent | +|----------|----------------|-----------------| +| Initial message with facts | ✓ Acknowledges | ✓ Acknowledges | +| Follow-up question about facts | ✓ Remembers | ✗ Forgets | +| Complex recall request | ✓ Full context | ✗ No context | +| Conversation resumption | ✓ Continues | N/A | + +## Running the Notebook + +### Prerequisites +- Contextual AI API key +- Python environment with required packages + +### Setup Steps +1. Update the `API_KEY` variable with your Contextual AI API key +2. Run all cells sequentially +3. Review the test results summary at the end + +### Expected Outputs +- Multi-turn agent should pass memory tests (remember color and cats) +- Single-turn agent should fail memory tests (forget previous information) +- All 5 tests should pass if multi-turn functionality works correctly + +## Configuration Details + +### Agent Configuration +```python +agent_configs = { + "global_config": { + "enable_multi_turn": True, # Key difference between agents + "enable_filter": False, + "enable_rerank": False, + "should_check_retrieval_need": True, + }, + "reformulation_config": { + "enable_query_expansion": False, + "enable_query_decomposition": False, + }, + "generate_response_config": { + "model": 'vertex_ai/gemini-2.0-flash-lite' + } +} +``` + +### Query with Conversation ID +```python +response = client.agents.query.create( + agent_id=agent.id, + messages=[{"content": "Your message", "role": "user"}], + conversation_id=conversation_id # Enables context retention +) +``` + +## Test Data + +The example uses simple, verifiable facts: +- **Favorite color**: Blue +- **Pet count**: 3 cats +- **Pet names**: Whiskers, Mittens, Shadow + +These concrete values make it easy to verify whether agents remember information correctly. + +## Cleanup + +The notebook includes an optional cleanup section to delete the test agents and datastore after experimentation. Uncomment the cleanup code if you want to remove the created resources. + +## Use Cases + +### Multi-Turn Agents Are Ideal For: +- Customer service chatbots +- Interactive tutorials +- Complex problem-solving sessions +- Personalized conversations + +### Single-Turn Agents Are Ideal For: +- FAQ systems +- Simple classification tasks +- Stateless API endpoints +- One-off queries + +## Troubleshooting + +### Common Issues +- **Agent doesn't remember**: Ensure you're using the correct `conversation_id` +- **Tests fail**: Check API key validity and agent configuration +- **Unexpected behavior**: Verify the `enable_multi_turn` setting + +### Debugging Tips +- Monitor conversation IDs in the output +- Check agent responses for context clues +- Validate test data matches expected values + +## Next Steps + +After understanding multi-turn behavior, consider: +- Implementing conversation persistence +- Adding conversation branching +- Building stateful applications +- Exploring advanced context management + +--- + +**Note**: Remember to keep your API key secure and never commit it to version control. \ No newline at end of file diff --git a/17-multi-turn/multi_turn.ipynb b/17-multi-turn/multi_turn.ipynb new file mode 100644 index 0000000..326feea --- /dev/null +++ b/17-multi-turn/multi_turn.ipynb @@ -0,0 +1,929 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "5235406f", + "metadata": {}, + "source": [ + "# Contextual AI Multi-Turn vs Single-Turn Chat Testing\n", + "\n", + "This notebook demonstrates the difference between multi-turn and single-turn agents in Contextual AI by creating two fresh agents.\n", + "\n", + "**What we're testing:**\n", + "- **MultiTurnAgent**: Will be created with multi-turn enabled - should remember conversation context\n", + "- **SingleTurnAgent**: Will be created with multi-turn disabled - should NOT remember previous messages\n", + "- **Recurring conversations**: Can continue from a previous conversation using conversation_id\n", + "\n", + "**Test approach:**\n", + "1. Create two new agents with different multi-turn settings\n", + "2. Tell both agents some simple facts (favorite color and pet names) \n", + "3. Ask if they remember those facts\n", + "4. Test recurring conversation capability\n", + "\n", + "**Expected results:**\n", + "- MultiTurnAgent should remember facts across messages\n", + "- SingleTurnAgent should forget facts between messages" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "aee6dba7", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Collecting contextual-client\n", + " Using cached contextual_client-0.8.0-py3-none-any.whl.metadata (16 kB)\n", + "Collecting matplotlib\n", + " Downloading matplotlib-3.10.7-cp312-cp312-macosx_11_0_arm64.whl.metadata (11 kB)\n", + "Collecting anyio<5,>=3.5.0 (from contextual-client)\n", + " Using cached anyio-4.11.0-py3-none-any.whl.metadata (4.1 kB)\n", + "Collecting distro<2,>=1.7.0 (from contextual-client)\n", + " Using cached distro-1.9.0-py3-none-any.whl.metadata (6.8 kB)\n", + "Collecting httpx<1,>=0.23.0 (from contextual-client)\n", + " Using cached httpx-0.28.1-py3-none-any.whl.metadata (7.1 kB)\n", + "Collecting pydantic<3,>=1.9.0 (from contextual-client)\n", + " Downloading pydantic-2.12.2-py3-none-any.whl.metadata (85 kB)\n", + "Collecting sniffio (from contextual-client)\n", + " Using cached sniffio-1.3.1-py3-none-any.whl.metadata (3.9 kB)\n", + "Collecting typing-extensions<5,>=4.10 (from contextual-client)\n", + " Using cached typing_extensions-4.15.0-py3-none-any.whl.metadata (3.3 kB)\n", + "Collecting idna>=2.8 (from anyio<5,>=3.5.0->contextual-client)\n", + " Using cached idna-3.11-py3-none-any.whl.metadata (8.4 kB)\n", + "Collecting certifi (from httpx<1,>=0.23.0->contextual-client)\n", + " Using cached certifi-2025.10.5-py3-none-any.whl.metadata (2.5 kB)\n", + "Collecting httpcore==1.* (from httpx<1,>=0.23.0->contextual-client)\n", + " Using cached httpcore-1.0.9-py3-none-any.whl.metadata (21 kB)\n", + "Collecting h11>=0.16 (from httpcore==1.*->httpx<1,>=0.23.0->contextual-client)\n", + " Using cached h11-0.16.0-py3-none-any.whl.metadata (8.3 kB)\n", + "Collecting annotated-types>=0.6.0 (from pydantic<3,>=1.9.0->contextual-client)\n", + " Using cached annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)\n", + "Collecting pydantic-core==2.41.4 (from pydantic<3,>=1.9.0->contextual-client)\n", + " Downloading pydantic_core-2.41.4-cp312-cp312-macosx_11_0_arm64.whl.metadata (7.3 kB)\n", + "Collecting typing-inspection>=0.4.2 (from pydantic<3,>=1.9.0->contextual-client)\n", + " Using cached typing_inspection-0.4.2-py3-none-any.whl.metadata (2.6 kB)\n", + "Collecting contourpy>=1.0.1 (from matplotlib)\n", + " Using cached contourpy-1.3.3-cp312-cp312-macosx_11_0_arm64.whl.metadata (5.5 kB)\n", + "Collecting cycler>=0.10 (from matplotlib)\n", + " Using cached cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)\n", + "Collecting fonttools>=4.22.0 (from matplotlib)\n", + " Downloading fonttools-4.60.1-cp312-cp312-macosx_10_13_universal2.whl.metadata (112 kB)\n", + "Collecting kiwisolver>=1.3.1 (from matplotlib)\n", + " Using cached kiwisolver-1.4.9-cp312-cp312-macosx_11_0_arm64.whl.metadata (6.3 kB)\n", + "Collecting numpy>=1.23 (from matplotlib)\n", + " Downloading numpy-2.3.4-cp312-cp312-macosx_14_0_arm64.whl.metadata (62 kB)\n", + "Requirement already satisfied: packaging>=20.0 in /Users/pedropacheco/Projects/examples/.venv/lib/python3.12/site-packages (from matplotlib) (25.0)\n", + "Collecting pillow>=8 (from matplotlib)\n", + " Downloading pillow-12.0.0-cp312-cp312-macosx_11_0_arm64.whl.metadata (8.8 kB)\n", + "Collecting pyparsing>=3 (from matplotlib)\n", + " Using cached pyparsing-3.2.5-py3-none-any.whl.metadata (5.0 kB)\n", + "Requirement already satisfied: python-dateutil>=2.7 in /Users/pedropacheco/Projects/examples/.venv/lib/python3.12/site-packages (from matplotlib) (2.9.0.post0)\n", + "Requirement already satisfied: six>=1.5 in /Users/pedropacheco/Projects/examples/.venv/lib/python3.12/site-packages (from python-dateutil>=2.7->matplotlib) (1.17.0)\n", + "Using cached contextual_client-0.8.0-py3-none-any.whl (154 kB)\n", + "Using cached anyio-4.11.0-py3-none-any.whl (109 kB)\n", + "Using cached distro-1.9.0-py3-none-any.whl (20 kB)\n", + "Using cached httpx-0.28.1-py3-none-any.whl (73 kB)\n", + "Using cached httpcore-1.0.9-py3-none-any.whl (78 kB)\n", + "Downloading pydantic-2.12.2-py3-none-any.whl (460 kB)\n", + "Downloading pydantic_core-2.41.4-cp312-cp312-macosx_11_0_arm64.whl (1.9 MB)\n", + "\u001b[2K \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m29.8 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m\n", + "\u001b[?25hUsing cached typing_extensions-4.15.0-py3-none-any.whl (44 kB)\n", + "Downloading matplotlib-3.10.7-cp312-cp312-macosx_11_0_arm64.whl (8.1 MB)\n", + "\u001b[2K \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.1/8.1 MB\u001b[0m \u001b[31m33.0 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m \u001b[31m34.9 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\n", + "\u001b[?25hUsing cached annotated_types-0.7.0-py3-none-any.whl (13 kB)\n", + "Using cached contourpy-1.3.3-cp312-cp312-macosx_11_0_arm64.whl (273 kB)\n", + "Using cached cycler-0.12.1-py3-none-any.whl (8.3 kB)\n", + "Downloading fonttools-4.60.1-cp312-cp312-macosx_10_13_universal2.whl (2.8 MB)\n", + "\u001b[2K \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.8/2.8 MB\u001b[0m \u001b[31m31.3 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m\n", + "\u001b[?25hUsing cached h11-0.16.0-py3-none-any.whl (37 kB)\n", + "Using cached idna-3.11-py3-none-any.whl (71 kB)\n", + "Using cached kiwisolver-1.4.9-cp312-cp312-macosx_11_0_arm64.whl (64 kB)\n", + "Downloading numpy-2.3.4-cp312-cp312-macosx_14_0_arm64.whl (5.1 MB)\n", + "\u001b[2K \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.1/5.1 MB\u001b[0m \u001b[31m31.9 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m\n", + "\u001b[?25hDownloading pillow-12.0.0-cp312-cp312-macosx_11_0_arm64.whl (4.7 MB)\n", + "\u001b[2K \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.7/4.7 MB\u001b[0m \u001b[31m31.9 MB/s\u001b[0m \u001b[33m0:00:00\u001b[0m\n", + "\u001b[?25hUsing cached pyparsing-3.2.5-py3-none-any.whl (113 kB)\n", + "Using cached sniffio-1.3.1-py3-none-any.whl (10 kB)\n", + "Using cached typing_inspection-0.4.2-py3-none-any.whl (14 kB)\n", + "Using cached certifi-2025.10.5-py3-none-any.whl (163 kB)\n", + "Installing collected packages: typing-extensions, sniffio, pyparsing, pillow, numpy, kiwisolver, idna, h11, fonttools, distro, cycler, certifi, annotated-types, typing-inspection, pydantic-core, httpcore, contourpy, anyio, pydantic, matplotlib, httpx, contextual-client\n", + "\u001b[2K \u001b[38;2;114;156;31m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m22/22\u001b[0m [contextual-client]0m \u001b[32m19/22\u001b[0m [matplotlib]\n", + "\u001b[1A\u001b[2KSuccessfully installed annotated-types-0.7.0 anyio-4.11.0 certifi-2025.10.5 contextual-client-0.8.0 contourpy-1.3.3 cycler-0.12.1 distro-1.9.0 fonttools-4.60.1 h11-0.16.0 httpcore-1.0.9 httpx-0.28.1 idna-3.11 kiwisolver-1.4.9 matplotlib-3.10.7 numpy-2.3.4 pillow-12.0.0 pydantic-2.12.2 pydantic-core-2.41.4 pyparsing-3.2.5 sniffio-1.3.1 typing-extensions-4.15.0 typing-inspection-0.4.2\n", + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install contextual-client " + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "2da6c1c5", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Libraries imported successfully!\n" + ] + } + ], + "source": [ + "# Import required libraries\n", + "import time\n", + "from datetime import datetime\n", + "from contextual import ContextualAI\n", + "\n", + "print(\"Libraries imported successfully!\")" + ] + }, + { + "cell_type": "markdown", + "id": "d386cdc7", + "metadata": {}, + "source": [ + "## Configuration and Agent Creation\n", + "\n", + "Set up the API credentials and create two new agents - one with multi-turn enabled and one without." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2c93ab7d", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Client initialized successfully!\n", + "Ready to create agents...\n" + ] + } + ], + "source": [ + "# Configuration - API Key only (we'll create agents dynamically)\n", + "API_KEY = \"= 3: # If at least multi-turn tests pass\n", + " print(\"⚠️ PARTIAL SUCCESS: Multi-turn agent is working, but some edge cases failed.\")\n", + "else:\n", + " print(\"❌ FAILURE: Multi-turn functionality is not working as expected.\")\n", + "\n", + "print(\"\\nWhat each test validates:\")\n", + "print(\"- Tests 1-2: Multi-turn agent should remember conversation context\")\n", + "print(\"- Tests 3-4: Single-turn agent should NOT remember previous messages\")\n", + "print(\"- Test 5: Conversation can be continued using conversation_id\")\n", + "print(\"=\" * 60)" + ] + }, + { + "cell_type": "markdown", + "id": "0497cf96", + "metadata": {}, + "source": [ + "## Cleanup (Optional)\n", + "\n", + "If you want to delete the agents we created for this test, run the cell below." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "44c90728", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Uncomment lines above to delete agents\n" + ] + } + ], + "source": [ + "# #Cleanup: Delete the test agents (optional)\n", + "# #Uncomment the lines below if you want to delete the agents we created\n", + "\n", + "# print(\"Deleting test agents...\")\n", + "\n", + "# # Delete MultiTurnAgent\n", + "# try:\n", + "# client.agents.delete(multi_turn_agent.id)\n", + "# print(f\"✓ MultiTurnAgent ({multi_turn_agent.id}) deleted\")\n", + "# except Exception as e:\n", + "# print(f\"✗ Error deleting MultiTurnAgent: {e}\")\n", + "\n", + "# # Delete SingleTurnAgent \n", + "# try:\n", + "# client.agents.delete(single_turn_agent.id)\n", + "# print(f\"✓ SingleTurnAgent ({single_turn_agent.id}) deleted\")\n", + "# except Exception as e:\n", + "# print(f\"✗ Error deleting SingleTurnAgent: {e}\")\n", + "\n", + "# # Delete datastore\n", + "# try:\n", + "# client.datastores.delete(datastore_id)\n", + "# print(f\"✓ Datastore ({datastore_id}) deleted\")\n", + "# except Exception as e:\n", + "# print(f\"✗ Error deleting Datastore: {e}\")\n", + "\n", + "# print(\"Cleanup completed!\")\n", + "\n", + "print(\"Uncomment lines above to delete agents\")" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": ".venv", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.11" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/README.md b/README.md index b9ef84e..1156f36 100644 --- a/README.md +++ b/README.md @@ -26,6 +26,7 @@ This repository contains practical examples and demonstrations of how to interac - [Monitoring RAG](14-monitoring) - Using Metrics API to monitor your RAG agent - [Metadata Introduction](15-metadata-intro/) - Working with metadata in your RAG Agent - [Build your own Matthew McConaughey](16-matthew-mcconaughey) - Building your own custom RAG agent. +- [Multi-turn Conversation](17-multi-turn) - A chatbot that uses previous messages in the conversation as context for its responses. ### Integrations - [CrewAI Multi-Agent Workflow](13-crewai-multiagent/) - Using CrewAI in a MultiAgent workflow