Demystifying Azure AI Foundry: A Beginner's Guide

Microsoft

Mar 06, 2025

Are you ready to dive into the transformative world of Artificial Intelligence but unsure where to start? You landed at the right page. Welcome! Whether you’re just getting started or have a bit of experience under your belt, this read will provide you with the foundational knowledge needed to unlock the potential of Azure AI Foundry.

Azure AI Foundry offers a powerful suite of tools and Machine Learning Models. Let’s start harnessing its capabilities as an initial step to create innovative AI solutions that can revolutionize your projects or business.

As a Microsoft Technical Trainer, delivering classes on AI and Data for the past several years, I have witnessed the significant impact AI has had across various industries. My objective here is to equip you with the foundational knowledge and confidence necessary to embark on your AI journey.

Azure AI Foundry (formerly Azure AI Studio) is a suite of tools that makes artificial intelligence accessible to everyone. It allows users to build, deploy, and manage AI solutions easily. Azure AI Foundry can be leveraged to address real-world business challenges and foster innovation.

For instance, Azure AI Foundry can enhance predictive maintenance in manufacturing by analyzing sensor data to foresee equipment failures to minimize downtime and costs. In retail, it can perform customer sentiment analysis across social media, reviews, and surveys, offering insights into customer satisfaction and areas for improvement. For more ways that Azure AI Foundry can help your organization, see customer stories here: Azure AI Foundry - Generative AI Development Hub and here: Need inspirations? Real AI Apps stories by Azure customers to help you get started

With Azure AI Foundry, you can explore and develop various AI models and services tailored to your goals. The platform supports scalability, enabling proof of concepts to become full production applications effortlessly. It also has support for continuous monitoring and refinement ensuring long-term success.

Here's a brief overview of AI Foundry's main architectural components and their integration.

Azure AI Foundry architecture - Azure AI Foundry

At the top level, AI Foundry provides access to the following resources:

Management Center: Used to manage AI Foundry resources like hubs, projects, connected resources, and deployments. It is a part of the Azure AI Foundry portal that streamlines governance and management activities. In the management center, you can view and manage:

Projects and resources
Quotas and usage metrics
Govern access and permissions

For more information see: Management center overview - Azure AI Foundry

AI Foundry Hub: The main top-level resource in the AI Foundry portal, offering a centralized method for managing security, connectivity, and computing resources across playgrounds and projects. Once a hub is established, developers can create projects from it and access shared resources like storage accounts, key vaults, databases and others without requiring continuous assistance from an IT administrator. The hub uses the Azure Machine Learning service and this Azure Provider, Microsoft.MachineLearningServices/workspaces and the type is hub. It offers:

Security features, including a managed network for projects and model endpoints.
Computer resources for development, fine-tuning, open-source, and serverless model deployments.
Connections to other Azure services, such as Azure OpenAI and Azure AI Search.
An Azure storage account for data and artifacts.

AI Foundry Project: A project is part of the hub. Projects help organize work, save state across tools like prompt flow and enable collaboration. You can share files and data source connections within a project.

Hubs support multiple projects and users. Projects manage billing, access, and provide data isolation, using dedicated storage containers to securely share files among project members.

Once you have a project, you can connect to it from your code. You can explore models and capabilities before creating a project, but once you're ready to build, customize, test, and operationalize, a project is where you'll want to be.

The Azure provider for a project is Microsoft.MachineLearningServices/workspaces, and the type is Project. The project offers:

Reusable assets like datasets, models, and indexes.
A container for uploading data within the hub's storage.
Private data access for project members.
Model deployments from the catalog and adjusted model endpoints.

Connections: AI Foundry hubs and projects use connections to access resources from other services, such as an Azure Storage Account, Azure OpenAI, or other Azure AI services.

Once you have setup the Azure Foundry Hub, Project and Connection, its time to start exploring and deploying the models.

Model Catalog: Available in Azure AI Foundry portal to discover and use a wide range of models for building generative AI applications. The model catalog features hundreds of models across model providers such as Azure OpenAI Service, Meta, NVIDIA, Hugging Face, DeepSeek, and of course the models trained by Microsoft like Phi.

Azure AI Model Catalog – Foundation Models

You can search and discover models that meet your needs. The model catalog also offers the model performance benchmark metrics which can be accessed by using Compare Models feature or from the model card Benchmark tab.

Models need to be deployed to make them available for receiving inference requests. Azure AI Foundry offers a comprehensive suite of deployment options for those models depending on your needs and model requirements.

Prompt Flow: This feature can be used to generate, customize, or run a flow. A flow is an executable instruction set that can implement the AI logic. Flows can be created or run via multiple tools, like a prebuilt canvas, LangChain, etc. Iterations of a flow can be saved as assets; once deployed a flow becomes an API.

Prompt flow in Azure AI Foundry portal - Azure AI Foundry

A prompt is sent to a model, consisting of the user input, system message, and any examples. User input is text submitted in the chat window. System message is a set of instructions to the model scoping its behaviors and functionality.

Evaluators: Helpful tools to assess the frequency and severity of content risks or undesirable behavior in AI responses. Performing iterative, systematic evaluations with the right evaluators can help teams measure and address potential response quality, safety, or security concerns throughout the AI development lifecycle. You can explore Azure AI Foundry benchmarks to evaluate and compare models on publicly available datasets.

Deploy: Remember to deploy your model setup. Deployments are hosted within an endpoint and can receive data from clients and send responses back in real-time. You can invoke the endpoint for real-time inference for chat, copilot or another generative AI application.

AI Foundry has all the needed capabilities including content filters, Responsible AI factors and Security. More about these aspects in my next article.

Playgrounds: Be sure to test your model using the Playgrounds on Azure AI Foundry. The portal offers a chat playground for deploying and interacting with AI chat models, allowing you to refine them before production deployment.