As Snowflake continues to build the agentic architecture, Model Context Protocol ("MCP") has become a core element to helping standardize agents to tool communication. MCP allows agents and external systems to communicate to discover and invoke tools that are used for agentic flows, while helping deploy agents faster.

This guide provides a hands-on experience in building and evaluating an AI agent powered by a Snowflake Managed Model Context Protocol (MCP) server. The agent is specifically designed to be a health research agent for medical researchers and healthcare professionals, giving them instant access to medical research data from the Snowflake Marketplace.

Now that the data is AI-ready, the next thing we want to do is create our MCP server. The MCP server will expose these search services as tools to the agent.

In your Snowflake account, go to Projects and then Worksheets or click here to go there directly.

The code will:

Now that we've created our MCP Server, we'll move towards creating our agents. The managed MCP servers support both OAUTH authentication and PAT authentication. Once you are actually connecting and using the MCP servers, you need to know three important areas about access controls:

1. Usage on the MCP object itself: This is what you need to both connect to the server and discover which tools are available.
2. Use on the underlying tools: This ensures that any roles that have access to particular domains or departments in your company only have access to the right services.
3. Use on the data: ou need to not only be able to call the search service but you actually need to be able to view the underlying data.

You will set up all three of these pieces to ensure access and control is set up and governed appropriately.

This will:

Next, we'll build the agent connect to the MCP server. Now that you've run both the SQL scripts, we can open the notebook.

From the GitHub repository, open the build-and-evaluate-langgraph-agents-with-mcp-tools.ipynb file.

As you are opening notebook, you'll need to fill in your Snowflake credentials and your own OpenAI key, as the agent is external.

Next we'll create a Snowpark session to connect Snowflake using the account, the PAT, and the database and schema of our MCP server.

Once this cell runs, we now have our tools ready to go to add to our agent.

For our we will be using GPT-4o as our LLM.

Try out the agent by asking a simple question, such as "What is the primary indicator for a specific drug?". The agent will call the LLM, which may decide to call the MCP server for search results

What this agent is doing is calling the LLM to answer the question, and then the LLM, if it decides it is necessary, will then reach out to our MCP server to get access to search results from either clinical trials data or PubMed.

We see our agent is working, but the question is how to know how well it is performing. This is where AI observability and evaluations come in. To measure the agent's performance, you will use AI observability and evaluations.

To get started with evaluations, we first want to initialize our TruLens session. This is how we connect to Snowflake so we can store our logs, traces, and evaluations as we are evaluating our agent. To make this connection, we are going to simply import the Snowflake connector and then pass in the Snowpark session that we created earlier.

This score is going to tell us whether the agent is performing well or poorly on these specific criteria.

Register the first version of the agent using TrueGraph, which is suitable for the Langraph framework, with an application name and a version.

Next, create a test set of queries that mirrors what the agent will face in production.

Create a "run" to define the data set for evaluation. Use run.start to execute a batched run, invoking the agent against each query and logging the traces to Snowflake. Compute Metrics: After the run is complete, compute the metrics using run.compute_metrics.

Common errors observed:

Go back to the notebook.

Go back to the AI Observability page. You should now see two versions of the agent.

The headline metrics should show a significant jump, with Tool Calling and Tool Selection scores improving. Select both application versions and choose Compare to view a head-to-head analysis. Click on the same query to compare the traces. Version 1 should show a short trace with no tool calls (ungrounded). Version 2 should show a longer trace, successfully making calls to both PubMed and clinical trial search, demonstrating that the agent is grounded in data.

Congratulations! You successfully built and evaluated AI agents using the Model Context Protocol and Snowflake's AI observability.