[0:05]Hi everyone. Uh my name is Sean Henry. Uh I work on Microsoft Foundry on our developer tools and services like Microsoft Agent Framework, Semantic Kernel, AutoGen, and the Foundry Agent Service. And today I'd like to talk about some of the new features we've added to Microsoft Foundry and the Agent Framework to help you build AI powered automation and multi-agent orchestrations. But before I dive into some of the new features, I'd like to look back on the last year. You know, we've been talking about AI agents for for for about a year now. And um, you know, with Microsoft Foundry and and Microsoft Agent Framework, we've enabled you to build, deploy, and operate your agents at scale. And so some of the things I'm going to talk about today is how we've unified to a single uh Microsoft Agent Framework, how we're building multi-agent workflows uh that allow you to visualize your workflows and and and have your agents work together. How you can deploy and manage and evaluate everything in one place with Foundry. How you can customize your experience for your users with channels and UI, and how you can apply compliance and government governance to to your AI applications. And we've been doing this and we've been learning from our customers over the last year, um, and over 1 million agents and over 25,000 organizations are now using Foundry and AutoGen and Semantic Kernel. To build out scenarios like like customer support and sales assistance, internal productivity, knowledge assistance, document research, you know, the list goes on and on of the things you can build when you start building your agents and then stringing them together into these multi-agent uh workflows and orchestration patterns. But as we were kind of talking to folks over the year and building with them and building with our partners, we had a number of of developer challenges that came up. You know, things that we heard a lot was, where do I start? How do I start building these these agents? What is the best way to get started? What frameworks should I use? I'm a developer. What you know, how do I build these agents? How do I connect them to my other services? Uh, how do I, how do I make them work together if I build these agents that are talking to tools and and talking to NCV servers? Like how do I make these agents talk together? How do I know what my agent's doing once I deploy it? How do I know how it's performing, um, is it, is it meeting expectations? Is it, is it falling off over time? You know, with LLMs, things, things are not always perfect all the time. There are, there are a little bit, they can be random and stochastic. And so you want to be able to continuously measure your systems. Finally, once I've built my system and I've got it working, how do I deploy it and manage it at scale, um, and how do I help my users, uh, interface and talk to my agents and agent systems? So we'll go through all of these problems and and we'll, we'll talk about how, uh, how we've approached those with Microsoft Agent Framework and, um, and, and, uh, Microsoft Foundry. So first of all, if you're not familiar with Microsoft Agent Framework, but you may be familiar with with some of our libraries like AutoGen and Semantic Kernel, the Foundry SDK, the Microsoft 365 Agent SDK. Um, we've taken, uh, Microsoft Agent Foundry and made it our new framework. You can think of it as a successor to Semantic Kernel and AutoGen, you know, the same, the team, same team that brought you both of those libraries has built Microsoft Agent Framework as our kind of one unified Microsoft pro code experience for building, uh, for building agents and agent systems at Microsoft. And it takes the best from AutoGen and Semantic Kernel, it builds on top of Foundry SDK and integrates really well with Microsoft Foundry, as well as with the M365 ecosystem through the M365 Agent SDK. And what does it do? Well, it's your connector to all your AI services and that's one of the important things about Microsoft Agent Framework is it's not just for connecting to Foundry services. Obviously, it connects to thousands of of AI models in in Foundry and the the Foundry IQ and knowledge services that are available, um, and all the services from Azure. But also connects to the broader ecosystem of AI services. You can talk to models directly from Open AI, um, you can talk to models like uh from Google Gemini or Anthropic or AWS Bedrock, DeepSeek, all we have connectors for all of these, as well as local models. So you can use things like Foundry Local, Ola, to connect to models that are hosted and running, um, on your machine. We also connect to the other AI services that you may need like knowledge, knowledge and memory services, um, for for your vector stores that you need for, um, uh, knowledge bases for your agents, and for agent memory services that are becoming more and more important to give your agents, um, long-term contacts like MemZero. You can also connect directly to other agents, uh, either directly to the Foundry Agent Service, agents hosted in Foundry, um, but also to other, other, uh, hosting services like CoPilot Studio or Bedrock Agents or or LangGraph. Um, and all this can be done either through direct connections in Microsoft Agent Framework or through our long list of open standards we support from Open API and MCP for tools connections, uh, and logic apps for connections into that ecosystem for Microsoft, as well as A2A for agent to agent communications. I'll show you some examples of some new UI frameworks that we've enabled as well, uh, including integration with ChatKit and AGUI, so you can quickly add UI on top of your agents. And finally, because, uh, we, we want to be enterprise ready, um, uh, at Microsoft Agent Framework supports the full set of Open Telemetry, uh, Gen AI, um, um, uh, Gen AI, um, specification so that you can, um, you can take all of your output from your your agent systems and send them either directly to Foundry or to any other service that supports OTEL, so you can evaluate them and, um, and monitor your agents in production. So all this is available today. We announced Microsoft Agent Framework in preview and it's available today in .NET and Python. And this allows you to build a full spectrum of agent solutions, all the way from the most basic agents where, you know, you're really just uh querying an LLM to uh advanced single agents that do rag to large multi-agent systems that are interacting together. Now, one of the first questions you may have is like, Sean, you know, I've been using Semantic Kernel and AutoGen like, how do I migrate to Agent Framework? And you know, don't worry, we've got your back. We have a great set of tools, uh, built directly into Visual Studio Code that can take your existing Semantic Kernel and AutoGen projects and upgrade them. And I've been using myself, a lot of our partners have used these and just been, uh, I've just been overwhelmed with how well this works, uh, uh, in conjunction with Co, uh, GitHub CoPilot. Um, just really a fantastic way to upgrade and migrate your code, uh, from from Semantic Kernel and AutoGen into the latest versions of Microsoft Agent Framework. But what I'd like to do now is I'd like to show you a quick demo and a tour of how you can build a sophisticated multi-agent system with, uh, with Agent Framework. So I'm going to pop over to Visual Studio Code here. Um, and what I will show you here is, um, I have a, uh, I have a project that I've built for event planning. Um, and, uh, it has a bunch of agents in it and you can see how easy it is to create an agent with, um, uh, with Microsoft Agent Framework. You can see here I've, uh, this is a function that's for creating the agent for me, but created an agent itself, uh, it's just five or six lines of code. I can name the agent, give it a description, give it a prompt, um, and then connect it to all my agent tools. And I'll talk about a few things here. So one is the prompt. Um, you know, our our prompt is defined over here. Um, we can define it in text, we can have templates and slot filling and all those things, um, that you may want to do with a prompt. Um, we can, we can give it tools. Here's an MCP tool that I've here's here's an MCP tool that I've given it, a nice tool for sequential thinking, um, that it that it can use in order to think through a problem. And these are defined simply in my MCP file so I can connect those easily into, uh, Agent Framework. I can give it a set of built-in tools. For example, here's a code interpreter tool, um, that I can use, so I can execute code, um, while it's thinking through a process. Um, and if I was going too fast, we have hundreds of samples on the Agent Framework repo that you can go and take a look and use. But what I want to do is I want to actually connect all these agents together. I want to build a bunch of agents and connect them together. So in order to do that, I've built a workflow within Agent Framework. And so here's my workflow, um, I use a workflow builder, uh, you know, I give it a name and a description, and then I take all the agents that I've created here, my coordinator agent, my venue agent, my budget agent, and I connect them all together into a graph. And one thing I can do here and in Agent Framework is I can connect not just agents, but I've wrapped up these agents here in this case an executor, which means I can add my own code in there as well. I don't just need to have an agent. I can run any arbitrary code. I can even not run an agent if I just have, um, you know, a node on my workflow that I want to execute. But all this is a little hard to see in code, so let's pop over to our new, um, our new Dev UI here, which shows us what, uh, what this multi-agent system actually looks like. So, in Agent Framework, if if you just type Dev UI, it'll bring up, uh, and execute your agents in our in our Dev environment where you can inspect your agents and see how they're running. And you can see here I have all my agents that I've created. I've got my, my coordinator, I've got my venue specialist, my budget analyst, my logistics manager, and you can see I've all connected them together to my event coordinator. That's what I was building in code earlier. And from here I can run it. I can give it an input. Um, you know, I can say, plan a corporate holiday party for 50 people, give it a budget. And you can see over here, it's working through, um, it's working through, oh, it looks like my model wasn't set up, but normally you would see, uh, you know, your events flow through here. I can see my traces and I can see my tools as they were being executed through all these flows and it would, it would, it would zoom in and and and take a look at all of, uh, all these agents as they run. And so that's a great way to, um, execute your agents, to see what they're doing, um, and inspect them and inspect them as they're being built. Um, a couple other features that we've added, I'll move back over to PowerPoint here, um, uh, beyond just building agents in, um, in Agent Framework. Um, we've also added PerView integration. So very easy for you to take your inputs and your outputs, we have like a middleware system within Agent Framework that you can connect up to that's connected up to the PerView APIs. So if you're using Microsoft PerView for data loss prevention and analysis of, uh, of your organization of data in your organization, you can now connect that directly up to your agents and you can have your agents, um, uh, use the same data loss, uh, prevention policies that you use within your organization. Um, that's a very cool piece and very easy to do, um, within the PerView APIs. I'll pop back over to code real quick. I have another visual demo to talk about some of the, um, the UI pieces that we've added. Um, we've added integration into both ChatKit and AGUI. So ChatKit is the UI library that Open AI has, has developed for, um, integrating with agents and, and agent systems. Um, and it's a really nice set of UI packages that enable you to have a nice, clean, simple interface and customizable for for your users. So you can see here, I have a demo here of ChatKit plus Agent Framework and, you know, it kind of looks a little bit like, like Chat GPT, um, and that's kind of intentional. It it gives you a familiar interface for your users. They have all the usual things. I can see my history. I can chat. I have a couple suggestions here. So I've built this app that has a uses a tool, an MCP tool that goes and and gets the weather. Um, I can get the weather in New York. I can select a city here. One of the nice things with ChatKit is they have this concept of widgets, so they can return a UI that I that the LLM can specify, um, itself, and then dynamically populate it with things like, uh, here I can see what the weather in Seattle is. Uh, spoiler alert, the weather's not great in Seattle, um, today. Uh, but as as we get the results, okay, 29 degrees. Actually, that's not the right weather. It's not. I think that's supposed to be Fahrenheit. I think that messed up. Um, it it's cold and foggy here today. Um, but we can also do, uh, we can also do that with, uh, we can also do other things like, uh, you know, analyze an image. You can see here we can, uh, we can upload attachments, so I can take an image here. I'll take a nice image of, I have this nice soda can, this agent soda can I I AI generated, um, earlier. So we'll see if it can figure out what this is, but as all those controls makes a lot easier for building in capabilities, um, like, um, like file attachments. So you can see here it's analyzed that image and it uses the model directly. Um, as well as ChatKit, we've built in support for AGUI, um, which is a protocol, uh, used a lot with, um, used a lot with, uh, UI components like CoPilot Kit, um, in order to build, uh, UI on top of of agent and AI systems. And very similar ChatKit, um, you can build things. You can build a demo here. Here's a very, um, you know, simple, uh, clean interface for using CoPilot Kit to build your agents. Um, you know, we can ask it to generate a sonnet for it and it'll, um, uh, it'll work and we can do thumbs up and thumbs down and all those events can be returned to your application and copy paste. Um, you can also do very complex things with Agent Framework. I think with Agent Framework and AGUI, um, there's a generating UI, you know, just like, um, just like with CoPilot, you can generate dynamic UI, um, and then what's really nice is you can share state between your agent and UI as well. So you can build these complex forms, um, that talk to your agent and that state is maintained, um, by your agent. Okay, so that's some of the new tools in Agent Framework and some of the things that you can do, um, when you're building, uh, agent systems with, uh, with Agent Framework. But now that you've built your agent, um, let's talk about what how you can host it within Foundry. Um, so if you build your agent in code, now you want to host it somewhere. And of course, you can host it on things I mentioned durable functions. You can host it in Azure Container Apps. You can host it anywhere you want. You can host it, um, in any, any kind of cloud service that, um, uh, you know, supports web services. It's just, it's just code in a framework. But we want to make it really easy for you to build your agents and your agent systems and then get them automatically and easily connected up to Foundry, so that you can do things like like evaluations and and and channels and, um, and monitoring of your agents without having to hook that up independently. So, um, we've enabled, we've enabled, uh, Foundry to host agents, uh, within the Foundry Agent Service. This allows you to take your custom code built in either Microsoft Agent Framework or LangGraph. And with our developer tooling, we'll package that up into a container, um, and then deploy it and host it into Foundry.
[16:49]This gives you, uh, managed runtime that's serverless and, uh, auto scales the compute and a dedicated execution environment. Um, this is all based on Azure Container Apps. Um, you can integrate with the built-in tools in Foundry, so all the tools that you have deployed into Foundry, your agent tools, you can access those from your hosted agents. Of course, you can observe and monitor your agent execution, you can look at your performance metrics, your debug logs, you can do a cost analysis within Foundry. You can do continuous evaluation of your agent systems. You can see how they're performing over time. You can use the guard rails that are built into Foundry and the compliance checks, and you can see all your agents in the Foundry control plane. You can also interoperate between your agents and Foundry. Once they're deployed into Foundry as code agents, um, between Foundry and CoPilot Studio or the Microsoft 365 or Agent 365 ecosystem. And of course, you get all the foundation of Foundry like bring your own virtual networks, um, and resources for state storage. You can use all those within, within hosted agents. So I'll flip over, I'll do a really, a really quick demo of this. One of the things I I'd like to show is that, um, you know, obviously I have, um, I have a favorite around, um, using, um, using Agent Framework, but we've enabled you to use, um, the frameworks that that you like. So, for example, you can use, um, uh, this example here, which is, sorry, wrong page. Um, this one here, uh, which is an example using LangGraph. So you can see I have my LangGraph application here. Um, this is a very simple, uh, calculator example from from LangGraph, uh, if you ever used LangGraph, you probably started with this, um, with this sample. Um, it has a bunch of tools to enable you to, you know, do some basic math. Um, and we can take this sample and, um, and, and once we've connected it up to, uh, Foundry using this agent.yaml to configure it. So we have this agent.yaml file which tells us about the agent, um, tells us the models it's using, tells us the protocol it's using like responses. Um, we then use that and we use the tools you're already familiar with like AZD. Uh, we can call AZD AI Agent, um, create, and we create this agent. We connect it into our tools, um, and then we can do something like, um, like AZD up, um, and deploy this up into Foundry. And this takes a few minutes to do, so I'll pop over here to one I've deployed earlier, um, over in Foundry. Um, we can see this is the new Foundry portal. I have a few agents that I've created here, but you can see here that calculator agent has been deployed. And one of the things you can see is I have two different types of agents available now in Foundry. I have my declarative agents, my prompt agents that I'm familiar with, where I can specify, um, a prompt and tools and connections, um, within Foundry, um, and, and, and test these and evaluate them and trace them and monitor them in Foundry. But I also have my code agents, so my hosted agents. These are my agents that have been deployed into Foundry, uh, from my toolings. This is the calculator agent I built earlier. I've deployed into Foundry. I can't edit the prompt now because it's all based in code, um, but I can still, I can still talk to it. Um, I I, uh, I can, um, I can look at the traces and monitors and evaluations of this agent, um, and I can see how it's performing over time. So there's a very simple agent, not that much to see, but I can, I can look at the traces for it. I can, I can look how often it's been run. I can monitor it, I can do evaluations, I can do all those things within, uh, the agent, the agent playground. I can also connect my agents together. So you saw I had two agents here. I had my calculator agent. I also had a problem interpreter agent, which is kind of takes natural language and and, um, uh, breaks the problem down so it can be used by the calculator agent. And I can connect these together into a workflow. So just like I showed you workflows built in code and agent framework, we can also build these workflows visually in, uh, in Foundry. Um, so I can take agents, I can connect them together here. I've created this agent. I've connected it to the agent I've already created, which is my problem interpreter agent and my code agent. So I can connect my LangGraph agents together with my Agent Framework agents together with my Foundry agents and how these all work together. So that the, the sample I showed earlier with all those agents coordinating together, I could have easily built this visually right here in Foundry. And then we can evaluate it here by running it in preview. Um, and we can say, uh, you know, in natural language here, what is seven, uh, times six? Uh, and our first agent will interpret that natural language and then generate, uh, generate the math problem that, um, that our calculator agent will now go and solve. So you can see the output of one agent is now becoming the input of the other agent. We can connect those together and we can build very complex workflows this way. Um, we can also, you know, take these workflows that we've built in Foundry and we can migrate them back into, um, into Agent Framework as well. Um, so we can take the declarative version of these workflows. We can take the YAML, we can execute that directly into, uh, agent framework declaratively or we can open it in VS Code and we can do a conversion there with that migration assistant from, uh, from declarative Foundry workflows into, uh, Agent Framework Code in C# or Python. Okay, so to recap, back to slides. Um, you know, we talked about, uh, we talked about a bunch of, uh, problems and challenges that developers have faced when they're building their multi-agent systems and I talked about some solutions to these. So, you know, where do I start to build an agent? Uh, well, in my opinion, Microsoft Agent Framework is the way to go. Um, we like all frameworks though at at Foundry and we've offered abilities and connectors for you to use, um, uh, all the major frameworks, uh, for building agent systems with Microsoft framework. But we have a ton of new features in Microsoft Agent Framework and I'm really proud of the work that we've done there. Um, you can connect to, you know, how do I connect my agents to other services? Well, we have full support for MCP and A2A for connecting both tools and agents together. Um, how do I make my agents work together? We have dozens of patterns and samples and examples and workflows in the Microsoft Agent Framework repo, um, that you can use to build these very sophisticated and reliable workflows for orchestrating not just your agents together, but even your, you know, code-based workflows, any sort of code you can orchestrate together, um, and deploy it at scale in a distributed system using Microsoft Agent Framework and things like the Durable Agent Framework. Um, you can connect these into your telemetry systems using our, um, OTEL Telemetry as well as our new integration with Microsoft PerView, so you can have your own, uh, DLP and compliance policies associated to your agent systems. And of course, once you've built all your systems locally and and and have them running the way you want in your development environment, you can deploy those directly to Foundry, whether you're using Microsoft Agent Framework or LangGraph. And then once you've built your agent systems, of course, you want customers to use them. So we built integration with these, uh, really great user interface libraries from Open AI's ChatKit to AGUI, um, with a lot of extensibility and capabilities already built into them for building really nice UIs on top of your agent systems. So, just to wrap everything up, Foundry Agent Service, Microsoft Agent Framework working together to allow you to build your agent systems, connect them to your models, your Foundry IQ and your tools, uh, building multi-agent systems, hosted agents, and workflows together that you can now deploy, um, into your own applications, into M365 and into Agent 365, all built on top of, uh, the foundational layers of security, compliance, and governance from, uh, Microsoft Foundry. If you want to learn more, uh, you can start today by typing AI.azure.com and trying out the the new Foundry portal. Um, the Agent Framework is open source and on GitHub. Uh, please go please go try it out there. Um, you search for for Microsoft Agent Framework on GitHub, uh, the team is all there responding to discussions and issues and and you can join the community there. You can learn about building agents, um, on on Microsoft Learn, um, the QR code here you can try, as well as engage directly with us through Discord or through our discord, uh, our GitHub discussions on Foundry. Um, thank you, and like please enjoy the next year. I I I'm really excited to what you see what everyone can build with, um, uh, Microsoft Foundry and Microsoft Agent Framework.
