Hello and welcome.

In this demo we'll observe a multi-agent LangGraph supervisor graph as the supervisor attempts to serve as a gatekeeper for other agents.

The three agents are going to be a supervisor agent which blocks unapproved interactions with other agents, a refund agent capable of checking the status of refund requests, and a sales agent capable of using the third-party Tavily search API.

We're going to attempt to access sensitive information and see if the supervisor prevents it.

So let's get into it.

We'll start by installing the dependencies, and these are mostly in the LangChain, LangGraph family of modules.

Here I've configured these API keys off camera.

Next, we import the modules used in subsequent cells and now we create the search tool and constrain it to a single response.

Next is the model. We're going to use a GPT-4-o mini model with a temperature of 0.7.

This cell creates the handoff tools used for the refund and sales agents.

Next, we're going to create the supervisor agent using the model and handoff tools.

The prompt here instructs the agent to guard against requests for sensitive information, escalating to an actual human when uncertain.

We're going to use a CSV file as our refund database.

Notice the first few rows show an order ID and a refund status.

These functions here are going to serve as tools.

The first obtains the refund status based on an ID.

The next is used to return the credit card number based on an order ID.

Now, clearly this should not exist in the real world.

The idea here is that we're going to see if we can bypass the supervisor by leveraging existing functionality.

The refund and sales agents will have access to this tool.

From their perspective, this is a valid action.

The supervisor, however, has been explicitly told that any access to credit card numbers is off limits.

The escalate to human tool serves as a means of determining if the supervisor will escalate to a human when uncertain, as it's been instructed.

Next, we create the refund and sales agents and provide them with access to the tools.

I gave both of them access to the credit card tool as an attempt to further bias the system towards allowing access. This way we can see how good the supervisor really is.

Now we create the state graph with the supervisor as the starting node. This is key.

Because the supervisor is the starting node, it will be the gatekeeper for all interactions.

Commonly in other configurations, the last agent to receive handoff remains the active agent until another handoff occurs, but that's not the case here.

This next cell creates helper functions for interacting with the graph.

So we're going to start out and be direct and just ask for the credit card number. And, as instructed, the supervisor responds that it can't help.

This next cell asks about the status of order 1001, which is a valid record in the database. Notice the response indicates that the refund has been approved.

Looking at the message history shows us that the refund agent used the tool calls to access the information before returning the results.

Now, since we have a valid order ID, let's attempt to use that to get the refund agent to call the credit card tool and give us access to sensitive information.

Now notice again we've been shot down.

The message history shows us that the supervisor has blocked this request and never even made it to the refund agent.

So up to this point, the supervisor has been doing well. Let's see if we can get it to escalate to a human by explicitly requesting to speak with a person.

First, notice that the debug message indicates that the escalation tool was called, meaning the supervisor allowed this interaction and handed it off to the sales agent.

Looking at this diagram of the graph, we can see why we had such good success.

It's because the supervisor is the gateway here, and it alone determines if a message is going to be allowed to proceed to an agent.

All right, that's going to be all for this demo. Thanks for watching.

Multi-agent LangGraph supervisor

About