Azure OpenAI recipes for Azure API Management Service
I've just published a new post on my blog: Azure OpenAI recipes for Azure API Management Service. In this post, I cover various recipes for integrating Azure OpenAI with Azure API Management, including setting up backend resources, implementing throttling, round-robin calls, and generating reports. Check it out and let me know your thoughts! https://nicolgit.github.io/aoai-recipes-for-apim/Logic Apps Aviators Newsletter - March 2025
In this issue: Ace Aviator of the Month News from our product group News from our community Ace Aviator of the Month March’s Ace Aviator: Dieter Gobeyn What’s your role and title? What are your responsibilities? I work as an Azure Solution Architect; however, I remain very hands-on and regularly develop solutions to stay close to the technology. I design and deliver end-to-end solutions, ranging from architectural analysis to full implementation. My responsibilities include solution design, integration analysis, contributing to development, reviewing colleagues’ work, and proposing improvements to our platform. I also provide Production support when necessary. Can you give us some insights into your day-to-day activities and what a typical day in your role looks like? My days can vary greatly, but collaboration with my globally distributed team is always a priority. I begin my day promptly at 8 AM to align with different time zones. After our daily stand-up, I often reach out to colleagues to see if they need assistance or follow-up on mails/team messages. A significant portion of my day involves solution design—gathering requirements, outlining integration strategies, and collaborating with stakeholders. I also identify potential enhancements, perform preliminary analysis, and translate them into user stories. I also spend time on technical development, building features, testing them thoroughly, and updating documentation for both internal and client use. On occasions where deeper investigation is needed, I support advanced troubleshooting, collaborating with our support team if issues demand additional expertise. If a release is scheduled, I sometimes manage deployment activities in the evening. What motivates and inspires you to be an active member of the Aviators/Microsoft community? I’ve always valued the sense of community that comes from sharing knowledge. Early in my career, attending events and meeting fellow professionals helped me bridge the gap between theory and real-world practice. This informal environment encourages deeper, hands-on knowledge exchange, which often goes beyond what official documentation can provide. Now that I’m in a more senior role, I believe it’s my responsibility—and pleasure—to give back. Contributing to the community enables me to keep learning, connect with fantastic people, and grow both technically and personally. Looking back, what advice do you wish you had been given earlier that you’d now share with those looking to get into STEM/technology? Master the fundamentals, not just the tools. It’s easy to get caught up in the newest frameworks, cloud platforms, and programming languages. However, what remains constant are the core concepts such as networking, data structures, security, and system design. By understanding the ‘why’ behind each technology, you’ll be better equipped to design future-proof solutions and adapt fast as tools and trends evolve. What has helped you grow professionally? Curiosity and a commitment to continuous learning have been key. I’m always keen to understand the ‘why’ behind how things work. Outside my normal job, I pursue Microsoft Reactor sessions, community events, and personal projects to expand my skills. Just as important is receiving open, honest feedback from peers and being honest with oneself. Having mentors or colleagues who offer both challenges and support is crucial for growth, as they provide fresh perspectives and help you refine your skills. In many cases, I’ve found it takes effort outside standard working hours to truly develop my skills, but it has always been worth it. If you had a magic wand that could create a feature in Logic Apps, what would it be and why? I’d love to see more uniformity & predictability across adapters, for example in terms of their availability for both stateless and stateful workflows. Currently, certain adapters—like the timer trigger—are either unavailable in stateless workflows or behave differently. Unifying adapter support would not only simplify solution design decisions, but also reduce proof-of-concept overhead and streamline transitions between stateless and stateful workflows as requirements evolve. News from our product group Logic Apps Live Feb 2025 Missed Logic Apps Live in February? You can watch it here. You will find a live demo for the Exporting Logic Apps Standard to VS Code, some updates on the new Data Mapper User Experience and lots of examples on how to leverage Logic Apps to create your Gen AI solutions. Exporting Logic App Standard to VS Code Bringing existing Logic Apps Standard deployed in Azure to VS Code are now simpler with the new Create Logic Apps Workspaces from package. New & Improved Data Mapper UX in Azure Logic Apps – Now in Public Preview! We’re excited to announce that a UX update for Data Mapper in Azure Logic Apps is now in Public Preview! We have continuously improved Data Mapper, which is already generally available (GA), based on customer feedback. Parse or chunk content for workflows in Azure Logic Apps (Preview) When working with Azure AI Search or Azure OpenAI actions, it's often necessary to convert content into tokens or divide large documents into smaller pieces. The Data Operations actions, "Parse a document" and "Chunk text," can help by transforming content like PDFs, CSVs, and Excel files into tokenized strings and splitting them based on the number of tokens. These outputs can then be used in subsequent actions within your workflow. Connect to Azure AI services from workflows in Azure Logic Apps Integrate enterprise services, systems, and data with AI technologies by connecting your logic app workflows to Azure OpenAI and Azure AI Search resources. This guide offers an overview and practical examples on how to use these connector operations effectively in your workflow. Power Automate migration to Azure Logic Apps (Standard) Development teams often need to build scalable, secure, and efficient automation solutions. If your team is considering migrating flows from Microsoft Power Automate to Standard workflows in Azure Logic Apps, this guide outlines the key advantages of making the transition. Azure Logic Apps (Standard) is particularly beneficial for enterprises running complex, high-volume, and security-sensitive workloads. AI playbook, examples, and other resources for workflows in Azure Logic Apps AI capabilities are increasingly essential in applications and software, offering time-saving and innovative tasks like chat interactions. They also facilitate the creation of integration workloads across various services, systems, apps, and data within enterprises. This guide provides building blocks, examples, samples, and resources to demonstrate how to use AI services, such as Azure OpenAI and Azure AI Search, in conjunction with other services and systems to build automated workflows in Azure Logic Apps. Collect ETW trace in Logic App Standard An Inline C# script to collect Event Tracing for Windows (ETW) and store it in a text file, from within your Logic Apps. Typical Storage access issues troubleshooting With this blog post we intend to provide you more tools and visibility on how to troubleshoot your Logic App and accelerate your service availability restore. Download Logic App content for Consumption and Standard Logic App in the Portal It's common to see customers needing to download the JSON contents for their Logic Apps, either to keep a copy of the code or to initiate CI/CD. The methods to download this are very simple, accessible on a single button. Running Powershell inline with Az commands- Logic App Standard With the availability of the Inline "Execute Powershell code" action, a few questions have been brought to us like for example how to execute Az commands with this action. Deploy Logic App Standard with Application Routing Feature Based on Terraform and Azure Pipeline This article shared a mature plan to deploy logic app standard then set the application routing features automatically. It's based on Terraform template and Azure DevOps Pipeline. News from our community Azure Logic Apps: create Standard Logic App projects in Visual Studio Code from Azure portal export Post by Stefano Demiliani How many times you had the need to create a new Azure Logic App workflow starting from an existing one? Personally, this happens a lot of time… Starting with version 5.18.7 (published some days ago), the Azure Logic Apps (Standard) extension for Visual Studio Code provides the capability to create Standard Azure Logic App projects from an existing Logic App exported from the Azure portal. Bridging the Gap: Azure Logic Apps Meets On-Prem Fileshares Post by Tim D'haeyer The end of BizTalk Server is fast approaching, signaling a significant shift in the Microsoft integration landscape. With this transition, the era of on-premises integration is drawing to a close, prompting many organizations to migrate their integration workloads to Azure. One key challenge in this process is: “How can I read and write from an on-premises file share using Logic Apps?” Thankfully, this functionality has been available for some time with Azure Logic Apps Standard. Azure Logic Apps vs. Power Apps vs. Power Automate: What to Use When? Post by Prashant Singh The Architect’s Dilemma: Logic Apps vs. Power Apps vs. Power Automate! In my latest blog, I compare Logic Apps, Power Automate, and Power Apps—helping you pick the right one! Securing Azure Logic Apps: Prevent SQL Injection in Complex SQL Server Queries Post by Cameron McKay Executing COMPLEX queries as raw SQL is tempting in Logic App workflows. It's clear how to protect SQL CRUD actions in Logic Apps. BUT how do we protect our complex queries? In the Logic App Standard tier, built-in connectors run locally within the same process as the logic app Post by Sandro Pereira In the Logic App Standard tier, built-in connectors run locally within the same process as the logic app, reducing latency and improving performance. This contrasts with the Consumption model, where many connectors rely on external dependencies, leading to potential delays due to network round-trips. This makes Logic App Standard an ideal choice for scenarios where performance and low-latency integration are critical, such as real-time data processing and enterprise API integrations. Scaling Logic Apps Hybrid Post by Massimo Crippa Logic Apps Hybrid provides a consistent development, deployment, and observability experience across both cloud and edge applications. But what about scaling? Let's dive into that in this blog post. Calling API Management in a different subscription on LA Standard Post by Sandro Pereira Welcome again to another Logic Apps Best Practices, Tips, and Tricks post. Today, we will discuss how to call from Logic App Standard an API exposed in API Management from a different subscription using the in-app API Management connector. How to enable API Management Connector inside VS Code Logic App Standard Workflow Designer Post by Sandro Pereira If you’ve been working with Azure Logic Apps Standard in Visual Studio Code and noticed that the API Management connector is conspicuously absent from the list of connectors inside the workflow designer, you’re not alone. This is a typical behavior that many developers encounter, and understanding why it happens—and how to enable it—can save you a lot of headaches. Do you have strict security requirements for your workflows? Azure Logic Apps is the solution. Post by Stefano Demiliani Azure Logic Apps offers robust solutions for enterprise-level workflows, emphasizing high performance, scalability, and stringent security measures. This article explores how Logic Apps ensures business continuity with geo-redundancy, automated backups, and advanced security features like IP restrictions and VNET integration. Discover why Azure Logic Apps is the preferred choice for secure and scalable automation in large organizations.Introducing GenAI Gateway Capabilities in Azure API Management
We are thrilled to announce GenAI Gateway capabilities in Azure API Management – a set of features designed specifically for GenAI use cases. Azure OpenAI service offers a diverse set of tools, providing access to advanced models like GPT3.5-Turbo to GPT-4 and GPT-4 Vision, enabling developers to build intelligent applications that can understand, interpret, and generate human-like text and images. One of the main resources you have in Azure OpenAI is tokens. Azure OpenAI assigns quota for your model deployments expressed in tokens-per-minute (TPMs) which is then distributed across your model consumers that can be represented by different applications, developer teams, departments within the company, etc. Starting with a single application integration, Azure makes it easy to connect your app to Azure OpenAI. Your intelligent application connects to Azure OpenAI directly using API Key with a TPM limit configured directly on the model deployment level. However, when you start growing your application portfolio, you are presented with multiple apps calling single or even multiple Azure OpenAI endpoints deployed as Pay-as-you-go or Provisioned Throughput Units (PTUs) instances. That comes with certain challenges: How can we track token usage across multiple applications? How can we do cross charges for multiple applications/teams that use Azure OpenAI models? How can we make sure that a single app does not consume the whole TPM quota, leaving other apps with no option to use Azure OpenAI models? How can we make sure that the API key is securely distributed across multiple applications? How can we distribute load across multiple Azure OpenAI endpoints? How can we make sure that PTUs are used first before falling back to Pay-as-you-go instances? To tackle these operational and scalability challenges, Azure API Management has built a set of GenAI Gateway capabilities: Azure OpenAI Token Limit Policy Azure OpenAI Emit Token Metric Policy Load Balancer and Circuit Breaker Import Azure OpenAI as an API Azure OpenAI Semantic Caching Policy (in public preview) Azure OpenAI Token Limit Policy Azure OpenAI Token Limit policy allows you to manage and enforce limits per API consumer based on the usage of Azure OpenAI tokens. With this policy you can set limits, expressed in tokens-per-minute (TPM). This policy provides flexibility to assign token-based limits on any counter key, such as Subscription Key, IP Address or any other arbitrary key defined through policy expression. Azure OpenAI Token Limit policy also enables pre-calculation of prompt tokens on the Azure API Management side, minimizing unnecessary request to the Azure OpenAI backend if the prompt already exceeds the limit. Learn more about this policy here. Azure OpenAI Emit Token Metric Policy Azure OpenAI enables you to configure token usage metrics to be sent to Azure Applications Insights, providing overview of the utilization of Azure OpenAI models across multiple applications or API consumers. This policy captures prompt, completions, and total token usage metrics and sends them to Application Insights namespace of your choice. Moreover, you can configure or select from pre-defined dimensions to split token usage metrics, enabling granular analysis by Subscription ID, IP Address, or any custom dimension of your choice. Learn more about this policy here. Load Balancer and Circuit Breaker Load Balancer and Circuit Breaker features allow you to spread the load across multiple Azure OpenAI endpoints. With support for round-robin, weighted (new), and priority-based (new) load balancing, you can now define your own load distribution strategy according to your specific requirements. Define priorities within the load balancer configuration to ensure optimal utilization of specific Azure OpenAI endpoints, particularly those purchased as PTUs. In the event of any disruption, a circuit breaker mechanism kicks in, seamlessly transitioning to lower-priority instances based on predefined rules. Our updated circuit breaker now features dynamic trip duration, leveraging values from the retry-after header provided by the backend. This ensures precise and timely recovery of the backends, maximizing the utilization of your priority backends to their fullest. Learn more about load balancer and circuit breaker here. Import Azure OpenAI as an API New Import Azure OpenAI as an API in Azure API management provides an easy single click experience to import your existing Azure OpenAI endpoints as APIs. We streamline the onboarding process by automatically importing the OpenAPI schema for Azure OpenAI and setting up authentication to the Azure OpenAI endpoint using managed identity, removing the need for manual configuration. Additionally, within the same user-friendly experience, you can pre-configure Azure OpenAI policies, such as token limit and emit token metric, enabling swift and convenient setup. Learn more about Import Azure OpenAI as an API here. Azure OpenAI Semantic Caching policy Azure OpenAI Semantic Caching policy empowers you to optimize token usage by leveraging semantic caching, which stores completions for prompts with similar meaning. Our semantic caching mechanism leverages Azure Redis Enterprise or any other external cache compatible with RediSearch and onboarded to Azure API Management. By leveraging the Azure OpenAI Embeddings model, this policy identifies semantically similar prompts and stores their respective completions in the cache. This approach ensures completions reuse, resulting in reduced token consumption and improved response performance. Learn more about semantic caching policy here. Get Started with GenAI Gateway Capabilities in Azure API Management We’re excited to introduce these GenAI Gateway capabilities in Azure API Management, designed to empower developers to efficiently manage and scale their applications leveraging Azure OpenAI services. Get started today and bring your intelligent application development to the next level with Azure API Management.
Inbound private endpoint for Standard v2 tier of Azure API Management
Standard v2 was announced in general availability on April 1st, 2024. Customers can now configure an inbound private endpoint (preview) for your API Management Standard v2 instance to allow clients in your private network to securely access the API Management gateway over Azure Private Link. The private endpoint uses an IP address from an Azure virtual network in which it's hosted. Network traffic between a client on your private network and API Management traverses over the virtual network and a Private Link on the Microsoft backbone network, eliminating exposure from the public internet. Further, you can configure custom DNS settings or an Azure DNS private zone to map the API Management hostname to the endpoint's private IP address. Inbound private endpoint With a private endpoint and Private Link, you can: Create multiple Private Link connections to an API Management instance. Use the private endpoint to send inbound traffic on a secure connection. Use policy to distinguish traffic that comes from the private endpoint. Limit incoming traffic only to private endpoints, preventing data exfiltration. Combine with outbound virtual network integration to provide end-to-end network isolation of your API Management clients and backend services. Preview limitations Today, only the API Management instance’s Gateway endpoint supports inbound private link connections. In addition, each API management instance can support at most 100 private link connections. To participate in the preview and add an inbound private endpoint to your Standard v2 instance, you must complete a request form. The Azure API Management team will review your request and respond via email within five business days. Learn more API Management v2 tiers FAQ API Management v2 tiers documentation API Management overview documentationExpanding GenAI Gateway Capabilities in Azure API Management
In May 2024, we introduced GenAI Gateway capabilities – a set of features designed specifically for GenAI use cases. Today, we are happy to announce that we are adding new policies to support a wider range of large language models through Azure AI Model Inference API. These new policies work in a similar way to the previously announced capabilities, but now can be used with a wider range of LLMs. Azure AI Model Inference API enables you to consume the capabilities of models, available in Azure AI model catalog, in a uniform and consistent way. It allows you to talk with different models in Azure AI Studio without changing the underlying code. Working with large language models presents unique challenges, particularly around managing token resources. Token consumption impacts cost and performance of intelligent apps calling the same model, making it crucial to have robust mechanisms for monitoring and controlling token usage. The new policies aim to address challenges by providing detailed insights and control over token resources, ensuring efficient and cost-effective use of models deployed in Azure AI Studio. LLM Token Limit Policy LLM Token Limit policy (preview) provides the flexibility to define and enforce token limits when interacting with large language models available through the Azure AI Model Inference API. Key Features Configurable Token Limits: Set token limits for requests to control costs and manage resource usage effectively Prevents Overuse: Automatically blocks requests that exceed the token limit, ensuring fair use and eliminating the noisy neighbour problem Seamless Integration: Works seamlessly with existing applications, requiring no changes to your application configuration Learn more about this policy here. LLM Emit Token Metric Policy LLM Emit Token Metric policy (preview) provides detailed metrics on token usage, enabling better cost management and insights into model usage across your application portfolio. Key Features Real-Time Monitoring: Emit metrics in real-time to monitor token consumption. Detailed Insights: Gain insights into token usage patterns to identify and mitigate high-usage scenarios Cost Management: Split token usage by any custom dimension to attribute cost to different teams, departments, or applications Learn more about this policy here. LLM Semantic Caching Policy LLM Semantic Caching policy (preview) is designed to reduce latency and reduce token consumption by caching responses based on the semantic content of prompts. Key Features Reduced Latency: Cache responses to frequently requested queries based to decrease response times. Improved Efficiency: Optimize resource utilization by reducing redundant model inferences. Content-Based Caching: Leverages semantic similarity to determine which response to retrieve from cache Learn more about this policy here. Get Started with Azure AI Model Inference API and Azure API Management We are committed to continuously improving our platform and providing the tools you need to leverage the full potential of large language models. Stay tuned as we roll out these new policies across all regions and watch for further updates and enhancements as we continue to expand our capabilities. Get started today and bring your intelligent application development to the next level with Azure API Management.
Troubleshooting Azure Function App Proxy with Private Blob Container Access for Static Web App
Recently, I shared a problem I’m facing in my testing environment with a friend. I’ve decided to bring this issue to an open forum discussion to gather additional insights. I hope you can help me figure out what might be missing in my configuration. **Context:** I’m trying to replicate a solution in my test environment but encountering difficulties in a specific scenario. **Scenario:** I have a Function App acting as a proxy for a Static Web App hosted in a Blob Container. This Blob Container is set to private access, meaning public access is disabled. **The Problem:** The goal is for my Function App to authorize users and direct them correctly to the Static Web App. However, it’s not working as expected. **What I’ve tried so far:** 1. Configured Managed Identity for the Function App and granted the necessary permissions to the Blob Container. 2. Properly set up authentication and created the App Registration, which works flawlessly. 3. Verified that the proxy functions correctly when the Blob Container’s public access is enabled. **Current behavior:** - When public access to the Blob Container is enabled, everything works fine. - When public access is disabled, even with the proxy configured, access fails, and an error message "resource not found" is returned. **My questions are:** 1. Do I need to configure something additional in the proxy definition file? 2. Is there a specific setting, like a private endpoint or something similar, that I should implement to resolve this issue? **Additional considerations:** I haven’t configured a private endpoint yet, but I’m considering whether this would be the most appropriate solution for my case. My initial expectation was that granting the necessary permissions to the Function App via Managed Identity would solve the issue, but it hasn’t. I appreciate any guidance or suggestions you can provide!Introducing Azure API Management Policy Toolkit
We’re excited to announce the early release of the Azure API Management Policy Toolkit, a set of libraries and tools designed to change how developers work with API Management policies, making policy management more approachable, testable, and efficient for developers. Empowering developers with Azure API Management Policy Toolkit Policies have always been at the core of Azure API Management, offering powerful capabilities to secure, change behavior, and transform requests and responses to the APIs. Recently, we've made the policies easier to understand and manage by adding Copilot for Azure features for Azure API Management. This allows you to create and explain policies with AI help directly within the Azure portal. This powerful tool lets developers create policies using simple prompts or get detailed explanations of existing policies. This makes it much easier for new users to write policies and makes all users more productive. Now, with the Policy Toolkit, we’re taking another significant step forward. This toolkit brings policy management even closer to the developer experience you know. Elevating policy development experience Azure API Management policies are written in Razor format, which for those unfamiliar with it can be difficult to read and understand, especially when dealing with large policy documents that include expressions. Testing and debugging policy changes requires deployment to a live Azure API Management instance, which slows down feedback loop even for small edits. The Policy Toolkit addresses these challenges. You can now author your policies in C#, a language that feels natural and familiar to many developers and write tests against them. This shift improves the policy writing experience for developers, makes policies more readable, and shortens the feedback loop for policy changes. Key toolkit features to transform your workflow: Consistent policy authoring. Write policies in C#. No more learning Razor syntax and mixing XML and C# in the same document. Syntax checking: Compile your policy documents to catch syntax errors and generate Razor-based equivalents. Unit testing: Write unit tests alongside your policies using your favorite unit testing framework. CI/CD integration: Integrate Policy Toolkit into automation pipelines for testing and compilation into Razor syntax for deployment. Current Limitations While we’re excited about the capabilities of the Policy Toolkit, we want to be transparent about its current limitation: Not all policies are supported yet, but we’re actively working on expanding the coverage. We are working on making the Policy Toolkit available as a NuGet package. In the meantime, you’ll need to build the solution on your own. Unit testing is limited to policy expressions and is not supported for entire policy documents yet. Get Started Today! We want you to try the Azure API Management Policy Toolkit and to see if it helps streamlining your policy management workflow. Check out documentation to get started. We’re eager to hear your feedback! By bringing policy management closer to the developer, we’re opening new possibilities to efficiently manage your API Management policies. Whether you’re using the AI-assisted approach with Copilot for Azure or diving deep into C# with the Policy Toolkit, we’re committed to making policy management more approachable and powerful.
Update App Registration Client Secret Using Microsoft Graph REST API v1.0
Hello, I have a customer who wants to set the App registration Client Secret to 1 year. Here are the customer's requirements: For existing application registrations under ‘Certificates & Secrets’ pane, any new secrets added by owners should have the duration limited to one year. If the owner tries to set the duration greater than one year and clicks ‘Add’ button, the action should not be allowed with proper error displayed. The same behavior should also be applicable to new application registration specific secrets. It should not impact any existing secret that is present (greater or less than one year) for current application registrations. We need a way to enable and disable the global policy in case we want to disable it if something doesn’t work as expected. We don’t want to impact anything else wrt application registrations or anything in service principles. Based on the article you shared; Microsoft Entra application management policy API overview - Microsoft Graph v1.0 | Microsoft Learn Below is the script we are trying to use to add the global policy and set as default policy with isEnabled = true. As we cannot test in a different tenant, can you please confirm the snippet below will work for the above requirements? MgPolicyAppManagementPolicy | select * $policy = @{ "displayName" = "Enforce Max Lifetime for Secrets" "description" = "Policy to enforce a maximum lifetime of 1 year for any new secrets." "applicationRestrictions" = @{ "passwordCredentials" = @{ "maxLifetime" = "P365D" # ISO 8601 duration format for 1 year } } } New-MgPolicyAppManagementPolicy -BodyParameter $policy Update-MgPolicyDefaultAppManagementPolicy -id <ABOVE_POLICY_ID -IsEnabled $true I tried to test it in my own tenant, but I ran to a permission issue. Can someone please confirm if this snippet works against the customer's requirements? Thanks.
