Building a Reliable Multi-Agent customer support chatbot- Router and Assistant RAG Agent Part-2

This is in continuation of our article series for building customer support Chatbot. In this Article we will explore Router and Assistant crew design in more detail.

Crews

We build our all LangGraph workflow nodes with CrewAI Crew. Each Crew has its own state and graph flow defined by using one or more agents and tasks. Each agent / task can have access to one or more tools. We use CrewAI for better code management and modularity and separation among different modules of our Chatbot system.

Router Crew

Router crew analyses user question using LLM to route it to one appropriate sub crew / sub graph flow to handle the overall user experience.

We create Crew with its own agent and task. System prompt for Router agent enforces LLM to return only the name of the agent(team member). Based on returned name of agent LangGraph work flow and state is updated to run respective Crew. Router agent code is shown in below gist.

Line 35- 42 does this actual routing to delegate the user message handling to designated Agent.

Line 44–71 — Router CrewAI agent definition with its own role name , goal and back story which internally creates the system prompt for LLM.

Line 73–86 — Router Crew AI task definition, this defines the actual task LLM is asked to perform along with system prompt from CrewAI agent.

AI Assistant Crew

This agent handles all user questions/messages related to bank, its services, products etc. System prompt for Assistant agent enforces LLM to answer user from retrieved RAG vector store context and chat history.

This crew consist of Assistant Agent and Grader Agent with their specific tasks.

Assistant Agent

This agent is responsible to generate the answer for users message/question with or without relevant context, fetched tool. It has access rag_tool to fetch the relevant context from bank knowledge base/pdfs/documents. We use CrewAI built-in tool “PDFSearchTool” with llama3 and huggingface sentence transformer for embeddings.

Grader Agent

Answer generated by Assistant agent is graded if its relevant to users question. Grader filter out for any errors in answer, and tries to refine the answer in concise manner.

Code for Assistant crew is shared below-

Line 1–33 — we create a custom RAG tool “search_vectorstore” which can access PDF search. RAg tool is available to assistant agent so that LLM can call /use it based on user querstion.

Line 35–68 — We create Assistant Crew to be used as LangGraph node.

Line 70–101 — We create assistant agent with role, goal and backstory defining agents System prompt.

Line 103- End — We define assistant task to tell the LLM to answer user question using vector store context in best possible way.

Stay Tuned for Part 3 of this series, we will be covering Authenticator Agent design and related code.