Introducing Azure AI Agent Service: Your AI Assistant

 The Azure AI Agent Service

Azure's managed features let developers build safe, stateful, self-governing AI bots that automate all business processes.

To maximize autonomous AI agent potential, organizations need adaptive, safe platforms for development, deployment, and monitoring.

Azure AI Agent Service enables autonomous agents

At Ignite 2024, Azure unveiled the public preview of Azure AI Agent Service, a package of feature-rich, managed capabilities that combines all the models, data, tools, and services enterprises need to automate any business function. This announcement is driven by client needs and autonomous AI agent possibilities.

Flexible Azure AI Agent Service is use case-independent. Personal productivity agents that send emails and set up meetings, research agents that track market trends and generate reports, sales agents that investigate leads and automatically qualify them, customer service agents that follow up with personalized messages, and developer agents that update your code base or evolve a code repository interactively represent countless opportunities to automate repetitive tasks and open up new opportunities.

What distinguishes Azure AI Agent Service?

After talking to hundreds of companies, it found four necessary components to quickly manufacture safe, reliable agents:

• Develop and automate processes quickly: Agents must smoothly interact with tools, systems, and APIs to perform deterministic or non-deterministic activities.
•  Use knowledge connectors and a big memory: Agents must access internal and external information sources and monitor conversations to complete tasks.
• Flexible model selection: Agents built with the correct model can improve data integration, performance in unique situations, and cost effectiveness in scaled agent deployments.
• Built-in enterprise readiness: Agents must scale with an organization's needs, meet data protection and compliance regulations, and complete tasks reliably.

Azure AI Agent Service uses the user-friendly interface and broad toolbox in the Azure AI Foundry SDK and site to provide end-to-end agent development through a single product surface.

Let's explore Azure AI Agent Service's capabilities.

Automation and fast agent creation with rich integrations

Azure AI Agent Service, based on OpenAI's strong yet flexible Assistants API, enables rapid agent development with built-in memory management and a complex interface to smoothly link with various compute platforms and bridge LLM capabilities with general purpose, programmed operations

• Let your agent use 1400+ Azure Logic Apps connectors: Logic Apps' wide connector ecosystem lets agents perform user actions. Logic apps simplify Azure Portal workflow business logic to connect agents to external systems, tools, and APIs. Connectors include Azure App Service, Dynamics365 Customer Voice, Teams, M365 Excel, MongoDB, Dropbox, Jira, Gmail, Twilio, SAP, Stripe, ServiceNow, and more.
• Azure Functions allows stateless or stateful code-based actions beyond chat mode: Let your agent call APIs and wait for events. Azure Functions and Durable tasks provide serverless execution of synchronous, asynchronous, and event-driven tasks, such as invoice approval and supply chain monitoring.
• Code Interpreter helps your agent safely run Python code, handle several data types, and generate data and visual files. The Assistants API doesn't allow storage data use.
• Create OpenAPI-standard tool library: Connect your AI agent to an external API using OpenAPI 3.0 for scalable application compatibility. Infrastructure and web services integration benefit from custom tools that authenticate access and connections with managed identities (Microsoft Entra ID) for security.
• Llama Stack agents can use cloud-hosted tools: Llama Stack SDK developers can use Azure AI Agent Service. Cloud-hosted, enterprise-grade scalable solutions will be wireline compatible with Llama Stack.

Large knowledge environment anchor agent outputs

Easy to create an ecosystem of enterprise knowledge sources that help agents access and analyze data from several sources, improving user query responses. These connectors match your network and function well with data. Built-in data sources:

• Live web data Using Bing data, your agent can provide current information. This addresses LLMs' incapacity to answer “top news headlines” factually.
• Private data in Microsoft SharePoint can aid agents in providing appropriate responses. OBO authentication restricts agents to SharePoint data the end user has authorization for.
• Talk to Microsoft Fabric structured data: Power your company's data-driven decision making without SQL or data context. Fabric AI Skills let your agent create generative AI-based conversational Q&A systems on Fabric data. Fabric connects data securely with OBO authentication.
• Agent outputs should include Azure AI Search, Azure Blob, and local file private data: Azure reinvented the File Search tool in Assistants API to enable you import an Azure AI Search index or create a new one using Blob Storage or local storage with a data ingestion pipeline. This new file search gives you full control over your private data with Azure storage account file storage and Azure Search Resource search indexes.
• Competitive advantage using licensed data: To give agents the newest, finest data for your use case, license data from private data vendors like Tripadvisor. More licensed data from additional industries and professions will be added.

In addition to corporate information, AI agents need thread or conversation state management to maintain context, customize interactions, and optimize performance. Azure AI Agent Service streamlines thread management and improves interactions by managing and retrieving end-user conversation history. This also unlocks AI agent model context window constraints.

GPT-4o, Llama 3, or another suitable model

Developers love building AI assistants with Azure OpenAI Service Assistants API's latest OpenAI GPT models. You can develop task-specific agents, optimize TCO, and more with Azure's cutting-edge models from leading model suppliers.

• Azure AI Agent Service will enable Azure AI Foundry models and leverage cross-model compatible, cloud-hosted tools for code execution, retrieval-augmented generation, and more. In addition to Azure OpenAI models, the Azure Models-as-a-Service API lets developers create Meta Llama 3.1, Mistral Large, and Cohere Command R+ agents.
• Multi-modal capability allows AI agents handle data formats other than text, expanding application cases. GPT-4o will enable picture and audio modalities so you may analyze and blend data to obtain insights, make decisions, and provide user-specific outputs.

For creating safe, enterprise-ready agents from scratch

Enterprises can protect sensitive data and meet regulations with Azure AI Agent Service.

• Bring your own storage: Unlike Assistants API, you can now safely attach enterprise data sources for your agent.
• BYO virtual network: Protect network interactions and data privacy with agent programs that never send data publicly.
• Keyless setup, OBO authentication: On-behalf-of authentication simplifies agent configuration and authentication, simplifying resource management and deployment.
• Performance and scaling are unlimited with Azure AI Agent Service on supplied deployments. Flexible, predictable latency, and high throughput are now possible with agent-powered apps.
• Track agent performance with OpenTelemetry: Know your AI agent's reliability and performance. The Azure AI Foundry SDK adds OpenTelemetry-compatible metrics to your monitoring dashboard for offline and online agent output analysis.
•  Content filtering and XPIA mitigation promote responsible building: Azure AI Agent Service uses prebuilt and custom content filters to detect harmful content at varying severity levels.

Prompt shields prevent cross-prompt injection assaults on agents. Like Azure OpenAI Service, Azure AI Agent Service prompts and completions are not used to train, retrain, or improve Microsoft or 3rd party products or services without your consent. Delete customer data anytime.

Create powerful multi-agent systems using Azure AI Agent Service

Azure AI Agent Service comes pre-configured with Assistants API-compatible multi-agent orchestration frameworks, such as Semantic Kernel and AutoGen, developed by Microsoft Research.

Create singleton agents with Azure AI Agent Service to construct a new multi-agent solution with the most reliable, scalable, and secure agents. AutoGen, which is always growing, coordinates these agents and humans to find the best cooperation patterns. You can migrate AutoGen features that show production value into Semantic Kernel for non-breaking updates and production support.