The Future of AI: Customizing AI agents with the Semantic Kernel agent framework
The blog post Customizing AI agents with the Semantic Kernel agent framework discusses the capabilities of the Semantic Kernel SDK, an open-source tool developed by Microsoft for creating AI agents and multi-agent systems. It highlights the benefits of using single-purpose agents within a multi-agent system to achieve more complex workflows with improved efficiency. The Semantic Kernel SDK offers features like telemetry, hooks, and filters to ensure secure and responsible AI solutions, making it a versatile tool for both simple and complex AI projects.Announcing DeepSeek-V3 on Azure AI Foundry and GitHub
We are pleased to announce the availability of DeepSeek-V3 on Azure AI Foundry model catalog with token-based billing. This latest iteration is part of our commitment to enable powerful, efficient, and accessible AI solutions through the breadth and diversity of choice in the model catalog.The Future of AI: Reduce AI Provisioning Effort - Jumpstart your solutions with AI App Templates
In the previous post, we introduced Contoso Chat – an open-source RAG-based retail chat sample for Azure AI Foundry, that serves as both an AI App template (for builders) and the basis for a hands-on workshop (for learners). And we briefly talked about five stages in the developer workflow (provision, setup, ideate, evaluate, deploy) that take them from the initial prompt to a deployed product. But how can that sample help you build your app? The answer lies in developer tools and AI App templates that jumpstart productivity by giving you a fast start and a solid foundation to build on. In this post, we answer that question with a closer look at Azure AI App templates - what they are, and how we can jumpstart our productivity with a reuse-and-extend approach that builds on open-source samples for core application architectures.The Future of AI: Harnessing AI for E-commerce - personalized shopping agents
Explore the development of personalized shopping agents that enhance user experience by providing tailored product recommendations based on uploaded images. Leveraging Azure AI Foundry, these agents analyze images for apparel recognition and generate intelligent product recommendations, creating a seamless and intuitive shopping experience for retail customers.Prepare and get ready for AI-900 Certification
The Azure AI Fundamentals Training program is designed to provide a foundational understanding of artificial intelligence (AI) concepts and how AI services can be utilized on Azure. This training is ideal for individuals new to AI, aiming to build a solid understanding of AI concepts, practical applications, and the Responsible AI considerations involved. Throughout the program, participants will explore various AI services available on Azure. By the end of the program, attendees will be equipped with the knowledge to implement and manage AI solutions using Azure's tools and services The training program will run from 7th January, and we will have live sessions on YouTube and Discord from 8 - 9pm GMT+3 Earn a Certification Voucher Upon successful completion of the program, Kenyan participants will receive a certification voucher. By earning this certification, you'll be able to showcase your knowledge and skills to potential employers and colleagues, giving you a competitive edge in the job market. Key Program Takeaways Gain a foundational understanding of AI concepts and their applications. Explore Azure's AI services and tools. Learn how to implement practical AI solutions using Azure. Understand the responsible AI considerations and best practices for developing and deploying AI projects. Learner Checklist As a learner/participant, this is how you can participate in the program: Catch up and rewatch our previous AI sessions on Microsoft Reactor. Link: https://aka.ms/aifundamentalstraining-reactors Sign up and participate in the Microsoft Learn Challenge: https://aka.ms/aifundamentalstraining-csc - closing soon! Certification vouchers are up for grabs once you complete the challenge for all Kenyan participants. Once you sign up or complete the challenge, fill the form https://aka.ms/aifundamentalstraining-voucher to be eligible for a certification voucher. Join the Discord Community to interact with other learners: https://aka.ms/aifundamentalstraining-discord Sign up to the AINSI Skills Navigator to customize your learning journey at https://aka.ms/aifundamentalstraining-navigator Continue learning and exploring: https://aka.ms/aifundamentalstraining-collection Online Sessions Calendar The training program will run from 7th January, and we will have live sessions on YouTube and Discord from 8 - 9pm GMT+3 Week Topic Live Sessions Link Description 7 Jan Introduction to Artificial Intelligence and Azure AI Services YouTube Embark on a journey to explore the fundamentals of artificial intelligence (AI) with Azure. 9 Jan Microsoft Azure AI Fundamentals: Computer Vision YouTube Dive into the world of computer vision with Azure and discover how to harness the power of AI to analyze and interpret visual data. 14 Jan Microsoft Azure AI Fundamentals: Natural Language Processing YouTube Dive into the fascinating world of natural language processing (NLP) with Azure and learn how to build intelligent applications that can understand and interpret human language 16 Jan Generative AI pt 1 - Fundamentals of Generative AI YouTube Step into the world of generative AI with Azure and discover how to create new content such as text, images, music, and code using advanced AI models. 23 Jan Responsible generative AI YouTube Explore the principles and practices of responsible AI with Azure. 28 Jan Document Intelligence and Knowledge Mining YouTube Discover the power of Azure AI Search and learn how to build intelligent search solutions that can transform your data into actionable insights 29 Jan Generative AI pt 2 - Introduction to Azure AI Foundry YouTube Step into the world of generative AI with Azure and discover how to create new content such as text, images, music, and code using advanced AI models in Azure AI Foundry. 30 Jan Certification Readiness Session Discord Prepare to ace your Microsoft certification exams with this comprehensive walkthrough session. What are you waiting for? Rewatch and engage with live sessions https://aka.ms/aifundamentalstraining-reactors Join in and learn together with us! Remember, certification vouchers are up for grabs! Preparing for the AI-900 Certification can be a rewarding experience that opens up new opportunities in the field of AI and ML. By following the tips and utilizing the resources provided, you'll be well on your way to achieving your certification. Stay motivated, keep learning, and good luck on your journey to becoming AI-900 certified! All learning resources can be found at: MS Learn Collection: https://aka.ms/aifundamentalstraining-collection Enjoyed the session? Send us your feedback, the good, the bad and the ugly at: https://aka.ms/aifundamentalstraining-feedbackNeed inspirations? Real AI Apps stories by Azure customers to help you get started
In this blog, we present a tapestry of authentic stories from real Azure customers. You will read about how AI-empowered applications are revolutionizing enterprises and the myriad ways organizations choose to modernize their software, craft innovative experiences, and unveil new revenue streams. We hope that these stories inspire you to embark upon your own Azure AI journey. Before we begin, be sure to bookmark the newly unveiled Plan on Microsoft Learn—meticulously designed for developers and technical managers—to enhance your expertise on this subject. Inspiration 1: Transform customer service Intelligent apps today can offer a self-service natural language chat interface for customers to resolve service issues faster. They can route and divert calls, allowing agents to focus on the most complex cases. These solutions also enable customer service agents to quickly access contextual summaries of prior interactions offer real-time recommendations and generally enhance customer service productivity by automating repetitive tasks, such as logging interaction summaries. Prominent use cases across industries are self-service chatbots, the provision of real-time counsel to agents during customer engagements, the meticulous analysis and coaching of agents following each interaction, and the automation of summarizing customer dialogues. Below is a sample architecture for airline customer service and support. Azure Database for PostgresSQL. Azure Kubernetes Services hosts web UI and integrates with other components. In addition, this app uses RAG, with Azure AI Search as the retrieval system, and Azure OpenAI Service provides LLM capabilities, allowing customer service agents and customers to ask questions using natural language. Air India, the nation’s flagship carrier, updated its existing virtual assistant’s core natural language processing engine to the latest GPT models, using Azure OpenAI services. The new AI-based virtual assistant handles 97% of queries with full automation and saves millions of dollars on customer support costs. "We are on this mission of building a world-class airline with an Indian heart. To accomplish that goal, we are becoming an AI-infused company, and our collaboration with Microsoft is making that happen.” — Dr. Satya Ramaswamy, Chief Digital and Technology Officer, Air India In this customer case, the Azure-powered AI platform also supports Air India customers in other innovative ways. Travelers can save time by scanning visas and passports during web check-in, and then scan baggage tags to track their bags throughout their journeys. The platform’s voice recognition also enables analysis of live contact center conversations for quality assurance, training, and improvement. Inspiration #2: Personalize customer experience Organizations now can use AI models to present personalized content, products, or services to users based on multimodal user inputs from text, images, and speech, grounded on a deep understanding of their customer profiles. Common solutions we have seen include conversational shopping interfaces, image searches for products, product recommenders, and customized content delivery for each customer. In these cases, product discovery is improved through searching for data semantically, and as a result, personalized search and discovery improve engagement, customer satisfaction, and retention. Three areas are critical to consider when implementing such solutions. First, your development team should examine the ability to integrate multiple data types (e.g., user profiles, real-time inventory data, store sales data, and social data.) Second, during testing, ensure that pre-trained AI models can handle multi-modal inputs and can learn from user data to deliver personalized results. Lastly, your cloud administrator should implement scalability measures to meet variable user demands. ASOS, a global online fashion retailer, leveraged Azure AI Foundry to revolutionize its customer experience by creating an AI-powered virtual stylist that could engage with customers and help them discover new trends. "Having a conversational interface option gets us closer to our goals of fully engaging the customer and personalizing their experience by showing them the most relevant products at the most relevant time.” — Cliff Cohen, Chief Technology Officer, ASOS In this customer case, Azure AI Foundry enabled ASOS to rapidly develop and deploy their intelligent apps, integrating natural language processing and computer vision capabilities. Enabled ASOS to rapidly develop and deploy their intelligent app, integrating natural language processing and computer vision capabilities. This solution takes advantage of Azure’s ability to support cutting-edge AI applications in the retail sector, driving business growth and customer satisfaction. Inspiration #3: Accelerate product innovation Building customer-facing custom copilots has the promise to provide enhanced services to your customers. This is typically achieved through using AI to provide data-driven insights that facilitate personalized or unique customer interactions, to enable customer access to a wider range of information, while improving search queries and making data more accessible. You can check out a sample architecture for building your copilot below. DocuSign, a leader in e-signature solutions with 1.6 million global customers, pioneered an entirely new category of agreement management designed to streamline workflows and created Docusign Intelligent Agreement Management (IAM). The IAM platform uses sophisticated multi-database architecture to efficiently manage various aspects of agreement processing and management. At the heart of the IAM platform is Azure AI, which automates manual tasks and processes agreements using machine learning models. "We needed to transform how businesses worked with a new platform. With Docusign Intelligent Agreement Management, built with Microsoft Azure, we help our customers create, commit to, manage, and act on agreements in real-time.” — Kunal Mukerjee, VP, Technology Strategy and Architecture, Docusign The workflow begins with agreement data stored in an Azure SQL Database and is then transferred through an ingestion pipeline to Navigator, an intelligent agreements repository. In addition, the Azure SQL Database Hyperscale service tier serves as the primary transactional engine, providing virtually unlimited storage capacity and the ability to scale compute and storage resources independently. Inspiration #4: Optimize employee workflows With AI-powered apps, businesses can organize unstructured data to streamline document management and information, leverage natural language processing to create a conversational search experience for employees, provide more contextual information to increase workplace productivity and summarize data for further analysis. Increasingly we have seen solutions such as employee chatbots for HR, professional services assistants (legal/tax/audit), analytics and reporting agents, contact center agent assistants, and employee self-service and knowledge management (IT) centers. It’s essential to note that adequate prompt engineering training can improve employee queries, and your team should examine the capability of integrating copilot with other internal workloads; lastly, make sure your organization implements continuous innovation and delivery mechanisms to support new internal resources and optimize chatbot dialogs. Improving the lives of clinicians and patients Medigold Health, one of the United Kingdom’s leading occupational health service providers, migrated applications to Azure OpenAI Service, with Azure Cosmos DB for logging and Azure SQL Database for data storage, achieving the automation of clinician processes, including report generation, leading to a 58% rise in clinician retention and greater job satisfaction. With Azure App Service, Medigold Health was also able to quickly and efficiently deploy and manage web applications, enhancing the company’s ability to respond to client and clinician needs. "We knew with Microsoft and moving our AI workloads to Azure, we’d get the expert support, plus scalability, security, performance, and resource optimization we needed.” — Alex Goldsmith, CEO, Medigold Health Inspiration #5: Prevent fraud and detect anomalies Increasingly, organizations leverage AI to identify suspicious financial transactions, false account chargebacks, fraudulent insurance claims, digital theft, unauthorized account access or account takeover, network intrusions or malware attacks, and false product or content reviews. If your company can use similar designs, take a glance at a sample architecture for building an interactive fraud analysis app below. Azure Cosmos DB. Transactional data is available for analytics in real-time (HTAP) using Synapse Link. All the other financial transactions such as stock trading data, claims, and other documents are integrated with Microsoft Fabric using Azure Data Factory. This setup allows analysts to see real-time fraud alerts on a custom dashboard. Generative AI denoted here uses RAG, with Azure OpenAI Service of the LLM, and Azure AI Search as the retrieval system. Fighting financial crimes in the gaming world Kinectify, an anti-money laundering (AML) risk management technology company, built its scalable, robust, Microsoft Azure-powered AML platform with a seamless combination of Azure Cosmos DB, Azure AI Services, Azure Kubernetes Service, and the broader capabilities of Azure cloud services. "We needed to choose a platform that provided best-in-class security and compliance due to the sensitive data we require and one that also offered best-in-class services as we didn’t want to be an infrastructure hosting company. We chose Azure because of its scalability, security, and the immense support it offers in terms of infrastructure management.” — Michael Calvin, CTO, Kinectify With the new solutions in place, Kinectify detects 43% more suspicious activities achieves 96% faster decisions, and continues to champion handling a high volume of transactions reliably and identifying patterns, anomalies, and suspicious activity. Inspiration #6: Unlock organizational knowledge We have seen companies building intelligent apps to surface insights from vast amounts of data and make it accessible through natural language interactions. Teams will be able to analyze conversations for keywords to spot trends and better understand your customers. Common use cases can include knowledge extraction and organization, trend and sentiment analysis, curation of content summarization, automated reports, and research generation. Below is a sample architecture for enterprise search and knowledge mining. H&R Block, the trusted tax preparation company, envisioned using generative AI to create an easy, seamless process that answers filers’ tax questions, maintains safeguards to ensure accuracy, and minimizes the time to file. Valuing Microsoft’s leadership in security and AI and the longstanding collaboration between the two companies, H&R Block selected Azure AI Foundry and Azure OpenAI Service to build a new solution on the H&R Block platform to provide real-time, reliable tax filing assistance. By building an intelligent app that automates the extraction of key data from tax documents, H&R Block reduced the time and manual effort involved in document handling. The AI-driven solution significantly increased accuracy while speeding up the overall tax preparation process. "We conduct about 25 percent of our annual business in a matter of days.” — Aditya Thadani, Vice President, H&R Block Through Azure’s intelligent services, H&R Block modernized its operations, improving both productivity and client service and classifying more than 30 million tax documents a year. The solution has allowed the company to handle more clients with greater efficiency, providing a faster, more accurate tax filing experience. Inspiration #7: Automate document processing Document intelligence through AI applications helps human counterparts classify, extract, summarize, and gain deeper insights with natural language prompts. When adopting this approach, organizations are recommended to also consider prioritizing the identification of tasks to be automated, and streamline employee access to historical data, as well as refine downstream workload to leverage summarized data. Here is a sample architecture for large document summarization. Volve Group, one of the world’s leading manufacturers of trucks, buses, construction equipment, and marine and industrial engines, streamlined invoice and claims processing, saving over 10,000 manual hours with the help of Microsoft Azure AI services and Azure AI Document Intelligence. "We chose Microsoft Azure AI primarily because of the advanced capabilities offered, especially with AI Document Intelligence.” — Malladi Kumara Datta, RPA Product Owner, Volvo Group Since launch, the company has saved 10,000 manual hours—about 850-plus manual hours per month. Inspiration #8: Accelerate content delivery Using generative AI, your new applications can automate the creation of web or mobile content, such as product descriptions for online catalogs or visual campaign assets based on marketing narratives, accelerating time to market. It also helps you enable faster iteration and A/B testing to identify the best descriptions that resonate with customers. This pattern generates text or image content based on conversational user input. It combines the capabilities of Image Generation and Text Generation, and the content generated may be personalized to the user, data may be read from a variety of data sources, including Storage Account, Azure Cosmos DB, Azure Database for PostgreSQL, orAzure SQL. JATO Dynamics, a global supplier of automotive business intelligence operating in more than 50 countries, developed Sales Link with Azure OpenAI Service, which now helps dealerships quickly produce tailored content by combining market data and vehicle information, saving customers 32 hours per month. "Data processed through Azure OpenAI Service remains within Azure. This is critical for maintaining the privacy and security of dealer data and the trust of their customers.” — Derek Varner, Head of Software Engineering, JATO Dynamics In addition to Azure OpenAI, JATO Dynamics used Azure Cosmos DB to manage data from millions of transactions across 55 car brands. The database service also empowers scalability and quick access to vehicle and dealer transaction data, providing a reliable foundation for Sales Link. Closing thoughts From innovative solutions to heartwarming successes, it’s clear that a community of AI pioneers is transforming business and customer experiences. Let’s continue to push boundaries, embrace creativity, and celebrate every achievement along the way. Here’s to many more stories of success and innovation! Want to be certified as an Azure AI Engineer? Start preparing with this Microsoft Curated Learning Plan.
Transform Insurance Industry Workflows Using Generative AI Models and Azure Services
This article highlights an innovative automated solution designed to transform the processing of insurance claim forms for the insurance industry. Previously, underwriters were limited to handling just two to three claims per day, significantly hampering operational efficiency. With the implementation of this solution, companies have achieved a remarkable 60% increase in daily claim processing capacity. Built on Azure services, this architecture revolutionizes the management of claim forms submitted via email by automating critical tasks such as data extraction, classification, summarization, evaluation, and storage. Leveraging the power of AI and machine learning, this solution ensures faster, more accurate claim evaluations, enabling insurance companies to make informed decisions efficiently. The result is enhanced operational scalability, improved customer satisfaction, and a streamlined claims process. Scenario In the insurance industry, claim forms often arrive as email attachments, requiring manual processing to classify, extract, and validate information before it can be stored for analysis and reporting. This solution automates the process by leveraging Azure services to classify, extract, and summarize information from Insurance claim forms. Using Responsible AI evaluation, it ensures the performance of Large Language Models (LLMs) meets high standards. The data is then stored for further analysis and visualization in Power BI, where underwriters can access consumable reports. Architecture Diagram Components Azure Logic Apps: Automates workflows and integrates apps, data, and services. Used here to process emails, extract PDF attachments, and initiate workflows with an Outlook connector for attachment, metadata, and email content extraction. Azure Blob Storage: Stores unstructured data at scale. Used to save insurance claim forms in PDF and metadata/email content in TXT formats. Azure Functions: Serverless compute for event-driven code. Orchestrates workflows across services. Azure Document Intelligence: AI-powered data extraction from documents. Classifies and extracts structured content from ACCORD forms. Azure OpenAI: Provides advanced language models. Summarizes email content for high-level insights. LLM Evaluation Module (Azure AI SDK): Enhances Azure OpenAI summaries by evaluating and refining output quality. Azure AI Foundry: Manages Azure OpenAI deployments and evaluates LLM performance using Responsible AI metrics. Azure Cosmos DB: Globally distributed NoSQL database. Stores JSON outputs from Azure OpenAI and Document Intelligence. Microsoft Power BI: Visualizes Cosmos DB data with interactive reports for underwriters. Workflow Description The workflow for processing claims efficiently leverages a series of Azure services to automate, structure, and analyze data, ensuring a fast, accurate, and scalable claims management system. 1. Email Processing with Azure Logic Apps The process begins with a pre-designed Azure Logic Apps workflow, which automates the intake of PDF claim forms received as email attachments from policyholders. By using prebuilt Outlook connectors, it extracts key details like sender information, email content, metadata, and attachments, organizing the data for smooth claims processing. This automation reduces manual effort, accelerates claim intake, and minimizes data capture errors. 2. Secure Data Storage in Azure Blob Storage Once emails are processed, the necessary PDF attachments, email content, and email metadata are stored securely in Azure Blob Storage. This centralized, scalable repository ensures easy access to raw claim data for subsequent processing. Azure Blob’s structured storage supports efficient file retrieval during later stages, while its scalability can handle growing claim volumes, ensuring data integrity and accessibility throughout the entire claims processing lifecycle. 3. Workflow Orchestration with Azure Functions The entire processing workflow is managed by Azure Functions, which orchestrates serverless tasks such as document classification, data extraction, summarization, and LLM evaluation. This modular architecture enables independent updates and optimizations, ensuring scalability and easier maintenance. Azure Functions streamlines operations, improving the overall efficiency of the claims processing system. a. Document Classification: The next step uses Azure Document Intelligence to classify documents with a custom pretrained model, identifying insurance claim forms. This step ensures the correct extraction methods are applied, reducing misclassification and errors, and eliminating much of the need for manual review. The ability to customize the model also adapts to changes in document formats, ensuring accuracy and efficiency in later processes. b. Content Extraction: Once the insurance form is properly classified, Azure Document Intelligence extracts specific data points from the PDF claim forms, such as claim numbers and policyholder details. The automated extraction process saves time, reduces manual data entry, and improves accuracy, ensuring essential data is available for downstream processing. This capability also helps in organizing the information for efficient claim tracking and report generation. c. Document Intelligence Output Processing: The results are extracted in JSON format and then parsed and organized for storage in Azure Cosmos DB, ensuring that all relevant data is systematically stored for future use. d. Summarizing Content with Azure OpenAI: Once data is extracted, Azure OpenAI generates summaries of email content, highlighting key claim submission details. These summaries make it easier for underwriters and decision-makers to quickly understand the essential points without sifting through extensive raw data. e. Quality Evaluation with LLM Evaluation SDK: After summarization, the quality of the generated content is evaluated using the LLM Evaluation Module in the Azure AI SDK. This evaluation ensures that the content meets accuracy and relevance standards, maintaining high-quality benchmarks and upholding responsible AI practices. Insights from this evaluation guide the refinement and improvement of models used in the workflow. f. LLM Performance Dashboard with Azure AI Foundry: Continuous monitoring of the workflow’s quality metrics is done via the evaluation dashboard in Azure AI Foundry. Key performance indicators like Groundedness, fluency, coherence, and relevance are tracked, ensuring high standards are maintained. This regular monitoring helps quickly identify performance issues and informs model optimizations, supporting the efficiency of the claims processing system. g. Summarization Output Processing: After evaluation, the results from the OpenAI summarization output are parsed and stored in Cosmos DB, ensuring that all relevant data is saved in a structured format for easy access and retrieval. 4. Storing Data in Azure Cosmos DB The structured data, including parsed JSON outputs and summaries, is stored in Azure Cosmos DB, a fully managed, globally distributed NoSQL database. This solution ensures processed claim data is easily accessible for further analysis and reporting. Cosmos DB’s scalability can accommodate increasing claim volumes, while its low-latency access makes it ideal for high-demand environments. Its flexible data model also allows seamless integration with other services and applications, improving the overall efficiency of the claims processing system. 5. Data Visualization with Microsoft Power BI The final step in the workflow involves visualizing the stored data using Microsoft Power BI. This powerful business analytics tool enables underwriters and other stakeholders to create interactive reports and dashboards, providing actionable insights from processed claim data. Power BI’s intuitive interface allows users to explore data in depth, facilitating quick, data-driven decisions. By incorporating Power BI, the insurance company can effectively leverage stored data to drive business outcomes and continuously improve the claims management process. Related Use cases: Healthcare - Patient Intake and Medical Claims Processing: Automating the extraction and processing of patient intake forms and medical claims for faster reimbursement and improved patient care analysis. See the following article for more information on how to implement a solution like this. Financial Services - Loan and Mortgage Application Processing: Streamlining loan application reviews by automatically extracting and summarizing financial data for quicker decision-making. Retail - Supplier Invoice and Purchase Order Processing: Automating invoice and purchase order processing for faster supplier payment approvals and improved financial tracking. Legal contract and Document Review: Automating the classification and extraction of key clauses from legal contracts to enhance compliance and reduce manual review time. See the following article for more information on how to implement a solution like this. Government - Tax Filing and Documentation Processing: Automating the classification and extraction of tax filing data to ensure compliance and improve audit efficiency. To find solution ideas and reference architectures for Azure based solutions curated by Microsoft, go to the Azure Architecture Center and search with keywords like “retail”, “legal”, “healthcare”, etc. You’ll find hundreds of industry-related solutions that can help jumpstart your design process. Contributors: This article is maintained by Microsoft. It was originally written by the following contributors. Principal authors: Manasa Ramalinga| Principal Cloud Solution Architect – US Customer Success Oscar Shimabukuro Kiyan| Senior Cloud Solution Architect – US Customer SuccessAzure AI Foundry, GitHub Copilot, Fabric and more to Analyze usage stats from Utility Invoices
Overview With the introduction of Azure AI Foundry, integrating various AI services to streamline AI solution development and deployment of Agentic AI Workflow solutions like multi-modal, multi-model, dynamic & interactive Agents etc. has become more efficient. The platform offers a range of AI services, including Document Intelligence for extracting data from documents, natural language processing and robust machine learning capabilities, and more. Microsoft Fabric further enhances this ecosystem by providing robust data storage, analytics, and data science tools, enabling seamless data management and analysis. Additionally, Copilot and GitHub Copilot assist developers by offering AI-powered code suggestions and automating repetitive coding tasks, significantly boosting productivity and efficiency. Objectives In this use case, we will use monthly electricity bills from the utilities' website for a year and analyze them using Azure AI services within Azure AI Foundry. The electricity bills is simply an easy start but we could apply it to any other format really. Like say, W-2, I-9, 1099, ISO, EHR etc. By leveraging the Foundry's workflow capabilities, we will streamline the development stages step by step. Initially, we will use Document Intelligence to extract key data such as usage in kilowatts (KW), billed consumption, and other necessary information from each PDF file. This data will then be stored in Microsoft Fabric, where we will utilize its analytics and data science capabilities to process and analyze the information. We will also include a bit of processing steps to include Azure Functions to utilize GitHub Copilot in VS Code. Finally, we will create a Power BI dashboard in Fabric to visually display the analysis, providing insights into electricity usage trends and billing patterns over the year. Utility Invoice sample Building the solution Depicted in the picture are the key Azure and Copilot Services we will use to build the solution. Set up Azure AI Foundry Create a new project in Azure AI Foundry. Add Document Intelligence to your project. You can do this directly within the Foundry portal. Extract documents through Doc Intel Download the PDF files of the power bills and upload them to Azure Blob storage. I used Document Intelligence Studio to create a new project and Train custom models using the files from the Blob storage. Next, in your Azure AI Foundry project, add the Document Intelligence resource by providing the Endpoint URL and Keys. Data Extraction Use Azure Document Intelligence to extract required information from the PDF files. From the resource page in the Doc Intel service in the portal, copy the Endpoint URL and Keys. We will need these to connect the application to the Document Intelligence API. Next, let’s integrate doc intel with the project. In the Azure AI Foundry project, add the Document Intelligence resource by providing the Endpoint URL and Keys. Configure the settings as needed to start using doc intel for extracting data from the PDF documents. We can stay within the Azure AI Foundry portal for most of these steps, but for more advanced configurations, we might need to use the Document Intelligence Studio. GitHub Copilot in VS Code for Azure Functions For processing portions of the output from Doc Intel, what better way to create the Azure Function than in VS Code, especially with the help of GitHub Copilot. Let’s start by installing the Azure Functions extension in VS Code, then create a new function project. GitHub Copilot can assist in writing the code to process the JSON received. Additionally, we can get Copilot to help generate unit tests to ensure the function works correctly. We could use Copilot to explain the code and the tests it generates. Finally, we seamlessly integrate the generated code and unit tests into the Functions app code file, all within VS Code. Notice how we can prompt GitHub Copilot from step 1 of Creating the Workspace to inserting the generated code into the Python file for the Azure Function to testing it and all the way to deploying the Function. Store and Analyze information in Fabric There are many options for storing and analyzing JSON data in Fabric. Lakehouse, Data Warehouse, SQL Database, Power BI Datamart. As our dataset is small, let’s choose either SQL DB or PBI Datamart. PBI Datamart is great for smaller datasets and direct integration with PBI for dashboarding while SQL DB is good for moderate data volumes and supports transactional & analytical workloads. To insert the JSON values derived in the Azure Functions App either called from Logic Apps or directly from the AI Foundry through the API calls into Fabric, let’s explore two approaches. Using REST API and the other Using Functions with Azure SQL DB. Using REST API – Fabric provides APIs that we can call directly from our Function to insert records using HTTP client in the Function’s Python code to send POST requests to the Fabric API endpoints with our JSON data. Using Functions with Azure SQL DB – we can connect it directly from our Function using the SQL client in the Function to execute SQL INSERT statements to add records to the database. While we are at it, we could even get GitHub Copilot to write up the Unit Tests. Here’s a sample: Visualization in Fabric Power BI Let's start with creating visualizations in Fabric using the web version of Power BI for our report, UtilitiesBillAnalysisDashboard. You could use the PBI Desktop version too. Open the PBI Service and navigate to the workspace where you want to create your report. Click on "New" and select "Dataset" to add a new data source. Choose "SQL Server" from the list of data sources and enter "UtilityBillsServer" as the server name and "UtilityBillsDB" as the DB name to establish the connection. Once connected, navigate to the Navigator pane where we can select the table "tblElectricity" and the columns. I’ve shown these in the pictures below. For a clustered column (or bar) chart, let us choose the columns that contain our categorical data (e.g., month, year) and numerical data (e.g., kWh usage, billed amounts). After loading the data into PBI, drag the desired fields into the Values and Axis areas of the clustered column chart visualization. Customize the chart by adjusting the formatting options to enhance readability and insights. We now visualize our data in PBI within Fabric. We may need to do custom sort of the Month column. Let’s do this in the Data view. Select the table and create a new column with the following formula. This will create a custom sort column that we will use as ‘Sum of MonthNumber’ in ascending order. Other visualizations possibilities: Other Possibilities Agents with Custom Copilot Studio Next, you could leverage a custom Copilot to provide personalized energy usage recommendations based on historical data. Start by integrating the Copilot with your existing data pipeline in Azure AI Foundry. The Copilot can analyze electricity consumption patterns stored in your Fabric SQL DB and use ML models to identify optimization opportunities. For instance, it could suggest energy-efficient appliances, optimal usage times, or tips to reduce consumption. These recommendations can be visualized in PBI where users can track progress over time. To implement this, you would need to set up an API endpoint for the Copilot to access the data, train the ML models using Python in VS Code (let GitHub Copilot help you here… you will love it), and deploy the models to Azure using CLI / PowerShell / Bicep / Terraform / ARM or the Azure portal. Finally, connect the Copilot to PBI to visualize the personalized recommendations. Additionally, you could explore using Azure AI Agents for automated anomaly detection and alerts. This agent could monitor electricity bill data for unusual patterns and send notifications when anomalies are detected. Yet another idea would be to implement predictive maintenance for electrical systems, where an AI agent uses predictive analytics to forecast maintenance needs based on the data collected, helping to reduce downtime and improve system reliability. Summary We have built a solution that leveraged the seamless integration of pioneering AI technologies with Microsoft’s end-to-end platform. By leveraging Azure AI Foundry, we have developed a solution that uses Document Intelligence to scan electricity bills, stores the data in Fabric SQL DB, and processes it with Python in Azure Functions in VS Code, assisted by GitHub Copilot. The resulting insights are visualized in Power BI within Fabric. Additionally, we explored potential enhancements using Azure AI Agents and Custom Copilots, showcasing the ease of implementation and the transformative possibilities. Finally, speaking of possibilities – With Gen AI, the only limit is our imagination! Additional resources Explore Azure AI Foundry Start using the Azure AI Foundry SDK Review the Azure AI Foundry documentation and Call Azure Logic Apps as functions using Azure OpenAI Assistants Take the Azure AI Learn courses Learn more about Azure AI Services Document Intelligence: Azure AI Doc Intel GitHub Copilot examples: What can GitHub Copilot do – Examples Explore Microsoft Fabric: Microsoft Fabric Documentation See what you can connect with Azure Logic Apps: Azure Logic Apps Connectors About the Author Pradyumna (Prad) Harish is a Technology leader in the GSI Partner Organization at Microsoft. He has 26 years of experience in Product Engineering, Partner Development, Presales, and Delivery. Responsible for revenue growth through Cloud, AI, Cognitive Services, ML, Data & Analytics, Integration, DevOps, Open Source Software, Enterprise Architecture, IoT, Digital strategies and other innovative areas for business generation and transformation; achieving revenue targets via extensive experience in managing global functions, global accounts, products, and solution architects across over 26 countries.The Future of AI: Power Your Agents with Azure Logic Apps
Building intelligent applications no longer requires complex coding. With advancements in technology, you can now create agents using cloud-based tools to automate workflows, connect to various services, and integrate business processes across hybrid environments without writing any code.
Distillation: Turning Smaller Models into High-Performance, Cost-Effective Solutions
by Vishal Yadav, Nikhil Pandey Introduction Large Language Models (LLMs) have transformed the landscape of natural language processing (NLP) with their ability to understand and generate human-like text. However, their size and complexity often pose challenges in terms of deployment, speed, and cost. Usually for specialized niche tasks, we end up deploying the best available model even though we don’t utilize all its capabilities. This is where distillation comes in, offering a method to create (fine-tune) smaller, customized, more efficient models, while retaining much of the performance of a significantly larger state-of-the-art model. What is distillation? Distillation is a technique designed to transfer knowledge of a large pre-trained model (the "teacher") into a smaller model (the "student"), enabling the student model to achieve comparable performance to the teacher model. This technique allows users to leverage the high quality of larger LLMs, while reducing inference costs in a production environment, thanks to the smaller student model. How distillation works? In distillation, knowledge can be transferred from teacher to student model in several ways. Here, we specifically discuss response-based, offline distillation, where the student model learns to mimic the output (only predictions) of the teacher model, and the teacher model is not trained during distillation. Teacher Model: A large, high-capacity teacher model that is already pre-trained on massive datasets. This model has learnt rich representations and complex patterns from the data which allows it to generalize well even on unseen tasks. Knowledge Extraction: The teacher model generates outputs based on given inputs, which are then used as training data for the student model. This involves not just mimicking outputs but also understanding the underlying reasoning processes. Student Model Training: A smaller student model is trained using the extracted knowledge as a guide. The student model learns to mimic the teacher model's behavior and predictions on specific tasks. Advantages Reduced Size: The resulting student model is significantly smaller, making it easier to deploy in resource-constrained environments. Lower Cost: Running smaller models incurs lower operational costs while maintaining competitive performance levels. Task-Specific Optimization: Distillation can be tailored for specific applications, enhancing efficiency and accuracy. Performance: Smaller models exhibit significantly lower latency compared to larger models, which in turn boosts the throughput of the deployment. Customization: Distillation allows users to select desirable traits from multiple larger models and transfer them to smaller models. Personalization: Personality traits can be incorporated into the model, enabling it to respond with relevant answers when queried about its personality. Synthetic Data Generation: At scale data generation can be done either only for labels or from scratch using just seed/meta data. Generalization: Distillation can help student models generalize better by learning from the teacher model's knowledge and avoiding overfitting. Improved Multilingual Capabilities: The multilingual performance of smaller models can be significantly enhanced with the help of teacher models making them suitable for global applications. Distillation in Azure AI Foundry Distillation as a Service is now supported on Azure allowing a variety of task types and more to be added soon. Following tasks are supported. Summarization: Given a document (article) to summarize, generate an entity-dense summary of the document. Conversational Assistant: Generate AI assistant responses on single-turn and multi-turn conversational datasets. To generate each response, the available chat history and the current user prompt are utilized. Natural Language Understanding (NLU) o MATH: Generate numeric answers to math problems. o Natural Language Inference (NLI): Given premise and hypothesis, determine if premise entails the hypothesis, or contradicts the hypothesis, or is neutral i.e. neither entails not contradicts the hypothesis. o Multiple-Choice Question Answering: Given question and answer choices, determine the correct answer choice. Distillation Process Overview of the two-step distillation process: (1) Generate synthetic data using a task-specific, elaborate prompt (2) Train (and infer from) the student model using a shorter prompt (Figure source: https://arxiv.org/pdf/2410.18588) The distillation process involves two main steps: generate high quality synthetic data (labels) using the teacher model, followed by instruction-based finetuning of the student model. Data Generation High-quality data generation is crucial for the student model's performance. Azure provides a proprietary library of advanced prompts, to generate high-quality synthetic data for all supported tasks, utilizing techniques such as Chain of Thought (CoT) or Chain of Density (CoD), and other best practices. This option can be enabled by passing the `enable_chain_of_thought` parameter while invoking the distillation pipeline, ensuring reasoning-based answers and consequently high-quality data for distillation. Instruction Fine-Tuning The next step is to fine-tune the smaller model using the task-specific generated data. This involves using a concise, task-specific prompt and training with the input and generated output (excluding reasoning steps). These innovations ensure significant performance gains for a given task while minimizing the cost (number of tokens) for the user. When using user-provided prompts, the same prompt is applied in both data generation and fine-tuning. Distillation Code Snippet Distillation is supported by the Azure SDK and CLI. Support for this was added in version 1.22.0 of azure-ai-ml. Ensure that the azure-ai-ml package is >= 1.22.0 before using the code snippet below. Model Offerings Teacher Models Currently Meta Llama 3.1 405B Instruct is supported as the teacher model for distillation. Student Models Currently Meta Llama 3.1 8B Instruct is supported as the student model for distillation. Soon all Microsoft’s Phi 3 and 3.5 Instruct series models will also be available for distillation. The following table demonstrates our current and upcoming student model offerings. Student Model Region Availability Meta Llama 3.1 8B Instruct West US 3 Available Phi 3/3.5 Instruct East US 2 Coming Soon At the time of this writing, fine-tuning of Meta Llama 3.1 Instruct series of models, and deployment of such fine-tuned models, is only available in West US 3 region. Whereas fine-tuning of Microsoft’s Phi 3 Instruct series of models, and deployment of such fine-tuned models, is only available in East US 2 region. Ensure your AI Foundry project is setup in the appropriate region for your selected student model. Notebooks Distilling Large Language Models for NLI Tasks: A Practical Guide Notebook - Distillation with Large Language Models This notebook provides a comprehensive guide on how to distil a large teacher model into a smaller student model, specifically for Natural Language Inference (NLI) tasks. It uses the Meta Llama 3.1 405B Instruct as the teacher and the Meta Llama 3.1 8B Instruct as the student model. Key Highlights Teacher and Student Models: The process uses Meta Llama 3.1 405B Instruct as the teacher model and Meta Llama 3.1 8B Instruct as the student model. Prerequisites: Ensure you have subscribed to the required models and set up an AI Foundry project in the West US 3 region for distillation of a Meta Llama 3.1 8B Instruct student model. SDK Installation: Install necessary SDKs such as azure-ai-ml, azure-identity, and mlflow. Dataset Preparation: Use the ConjNLI dataset from Hugging Face for training and validation. Distillation Job: Configure and run the distillation job to transfer knowledge from the teacher to the student model. Deployment: Optionally, deploy the distilled model to a serverless endpoint and perform sample inferences. This notebook simplifies the complex task of model distillation, making it accessible even to those new to NLP and model training. Results Using the ConjNLI dataset and Chain-Of-Thought (CoT) distillation, we obtain the following accuracy (%) metrics. Dataset Student Model Teacher (405B) with CoT Prompting Student with CoT Prompting Student Distilled on CoT-prompted Teacher Output ConjNLI (dev) Meta Llama 3.1 8B Instruct 69.98 52.81 63.88 ConjNLI (dev) Phi 3 Mini 128k Instruct 69.98 43.98 57.78 Distillation with the Meta Llama 3.1 8B Instruct and Phi 3 Mini 128k Instruct student models provides approximately 21% and 31% improvement respectively over directly prompting the student model using CoT prompt. For detailed results on other datasets and tasks, we refer the user to check the published results in our knowledge distillation paper. Conclusion Distillation represents a significant step forward in development and deployment of LLM/SLM at scale. By transferring the knowledge from a large pre-trained model (teacher) to a smaller, more efficient model (student), distillation offers a practical solution to the challenges of deploying large models, such as high costs and complexity. This technique not only reduces model size and operational costs but also enhances the performance of student models for specific tasks. The support for distillation on Azure AI Foundry further simplifies the process, making it accessible for various applications, such as summarization, conversational assistance, and natural language understanding tasks. Furthermore, the detailed, hands-on example notebooks provided in Azure Github can help facilitate easier adoption. In summary, distillation not only bridges the gap between generalist understanding and specialized application but also makes the way for a more sustainable and practical approach to leveraging LLMs in real-world scenarios.
