artificial intelligence
46 TopicsAzure OpenAI Landing Zone reference architecture
In this article, delve into the synergy of Azure Landing Zones and Azure OpenAI Service, building a secure and scalable AI environment. Unpack the Azure OpenAI Landing Zone architecture, which integrates numerous Azure services for optimal AI workloads. Explore robust security measures and the significance of monitoring for operational success. This journey of deploying Azure OpenAI evolves alongside Azure's continual innovation.192KViews40likes19CommentsSecurity Best Practices for GenAI Applications (OpenAI) in Azure
This article presents an in-depth guide on security best practices for GenAI applications that use LLM models within the Azure platform. Aimed at developers and system administrators, it explores the essentials for maintaining the confidentiality, integrity, and availability of LLMs such as Azure OpenAI. It delves into practical measures for addressing security challenges, including data breaches, misuse of AI, and regulatory compliance, while also emphasizing the role of a shared responsibility model in cloud security. The guide provides a comprehensive roadmap for implementing layered security strategies, encryption protocols, access controls, and monitoring practices to ensure the robust security of LLM applications in Azure.69KViews18likes0CommentsEmpowering AI: Building and Deploying Azure AI Landing Zones with Terraform
Discover the power of deploying Azure AI Landing Zones with Terraform. Explore best practices, secure connectivity, and streamlined access to Azure AI services. Learn to create a strong cloud foundation, optimize performance, and ensure governance for your AI solutions. Join us on this practical journey to harness the true capabilities of AI.29KViews8likes17CommentsDemystifying Azure OpenAI Networking for Secure Chatbot Deployment
Embark on a technical exploration of Azure's networking features for building secure chatbots. In this article, we'll dive deep into the practical aspects of Azure's networking capabilities and their crucial role in ensuring the security of your OpenAI deployments. With real-world use cases and step-by-step instructions, you'll gain practical insights into optimizing Azure and OpenAI for your projects.27KViews6likes9CommentsAI Studio End-to-End Baseline Reference Implementation
Discover the Future of AI Deployment with Azure AI Studio’s Baseline Reference Implementation Azure AI Studio is reshaping the landscape of cloud AI integration with its commitment to operational excellence and strategic alignment with core business objectives. We are thrilled to introduce Azure AI Studio’s end-to-end baseline reference implementation—a streamlined architecture crafted for seamless, scalable, and secure AI cloud deployments. Embark on a journey to deploy sophisticated AI workloads with confidence, supported by Azure AI Studio's robust baseline architecture. Whether it's hosting interactive AI playgrounds, constructing complex AI workflows with Promptflow, or ensuring resilient and secure deployments within Azure's managed network environment, this implementation is your blueprint for success. Embrace a new era of AI innovation where security and scalability converge with organizational compliance and governance. Join us in deploying tomorrow's AI solutions, today.3.9KViews5likes0CommentsAzure OpenAI chat baseline architecture in an Azure landing zone
Unlock the potential of AI in the cloud. Our Azure OpenAI Chat Baseline Architecture provides the blueprint you need to transition from testing to a full production environment, all within an Azure landing zone. Ensure security, scalability, and governance are part of your AI strategy. Get started with our concise guide and embrace the future of AI on Azure.7.1KViews5likes0CommentsExploring AI Agent-Driven Auto Insurance Claims RAG Pipeline.
In this post, I explore a recent experiment aimed at creating a RAG pipeline tailored for the insurance industry, specifically for handling automobile insurance claims, with the goal of potentially reducing processing times.3.6KViews4likes0CommentsReducing the Environmental Impact of Generative AI: a Guide for Practitioners
Discover how to harness the power of generative AI while minimising its environmental impact in this comprehensive guide for technical practitioners. From selecting the right foundation models and optimising fine-tuning processes to leveraging retrieval augmented generation (RAG) and mastering prompt engineering, we explore key strategies for sustainable AI development and deployment. By following best practices and making informed decisions at each stage of the AI lifecycle, you can effectively integrate generative AI into your products, services, and operations while prioritising energy efficiency, reducing carbon footprint, and promoting responsible resource management. Dive in to learn how to unlock the potential of generative AI without compromising on sustainability.21KViews4likes0CommentsSecurely Integrating Azure API Management with Azure OpenAI via Application Gateway
Introduction As organizations increasingly integrate AI into their applications, securing access to Azure OpenAI services becomes a critical priority. By default, Azure OpenAI can be exposed over the public internet, posing potential security risks. To mitigate these risks, enterprises often restrict OpenAI access using Private Endpoints, ensuring that traffic remains within their Azure Virtual Network (VNET) and preventing direct internet exposure. However, restricting OpenAI to a private endpoint introduces challenges when external applications, such as those hosted in AWS or on-premises environments, need to securely interact with OpenAI APIs. This is where Azure API Management (APIM) plays a crucial role. By deploying APIM within an internal VNET, it acts as a secure proxy between external applications and the OpenAI service, allowing controlled access while keeping OpenAI private. To further enhance security and accessibility, Azure Application Gateway (App Gateway) can be placed in front of APIM. This setup enables secure, policy-driven access by managing traffic flow, applying Web Application Firewall (WAF) rules, and enforcing SSL termination if needed. What This Blog Covers This blog provides a technical deep dive into setting up a fully secure architecture that integrates Azure OpenAI with APIM, Private Endpoints, and Application Gateway. Specifically, we will walk through: Configuring Azure OpenAI with a Private Endpoint to restrict public access and ensure communication remains within a secure network. Deploying APIM in an Internal VNET, allowing it to securely communicate with OpenAI while being inaccessible from the public internet. Setting up Application Gateway to expose APIM securely, allowing controlled external access with enhanced security. Configuring VNET, Subnets, and Network Security Groups (NSGs) to enforce network segmentation, traffic control, and security best practices. By the end of this guide, you will have a production-ready, enterprise-grade setup that ensures: End-to-end private connectivity for Azure OpenAI through APIM. Secure external access via Application Gateway while keeping OpenAI hidden from the internet. Granular network control using VNET, Subnets, and NSGs. This architecture provides a scalable and secure solution for enterprises needing to expose OpenAI securely without compromising privacy, performance, or compliance. Prerequisites Before diving into the integration of Azure API Management (APIM) with Azure OpenAI in a secure, private setup, ensure you have the following in place: 1. Azure Subscription & Required Permissions An active Azure Subscription with the ability to create resources. Contributor or Owner access to deploy Virtual Networks (VNETs), Subnets, Network Security Groups (NSGs), Private Endpoints, APIM, and Application Gateway. 2. Networking Setup Knowledge Familiarity with Azure Virtual Network (VNET) concepts, Subnets, and NSGs is helpful, as we will be designing a controlled network environment. 3. Required Azure Services The following services are needed for this integration: Azure Virtual Network (VNET) – To establish a private, secure network. Subnets & NSGs – For network segmentation and traffic control. Azure OpenAI Service – Deployed in a region that supports private endpoints. Azure API Management (APIM) – Deployed in an Internal VNET mode to act as a secure API proxy. Azure Private Endpoint – To restrict Azure OpenAI access to a private network. Azure Application Gateway – To expose APIM securely with load balancing and optional Web Application Firewall (WAF). 4. Networking and DNS Requirements Private DNS Zone: Required to resolve private endpoints within the VNET. Custom DNS Configuration: If using a custom DNS server, ensure proper forwarding rules are in place. Firewall/NSG Rules: Ensure necessary inbound and outbound rules allow communication between services. 5. Azure CLI or PowerShell (Optional, but Recommended) Azure CLI (az commands) or Azure PowerShell for efficient resource deployment. Once you have these prerequisites in place, we can proceed with designing the secure architecture for integrating Azure OpenAI with APIM using Private Endpoints and Application Gateway. Architecture Overview The architecture ensures secure and private connectivity between external users and Azure OpenAI while preventing direct public access to OpenAI’s APIs. It uses Azure API Management (APIM) in an Internal VNET, an Azure Private Endpoint for OpenAI, and an Application Gateway for controlled public exposure. Key Components & Flow User Requests External users access the API via a public endpoint exposed by Azure Application Gateway. The request passes through App Gateway before reaching APIM, ensuring security and traffic control. Azure API Management (APIM) – Internal VNET Mode APIM is deployed in Internal VNET mode, meaning it does not have a public endpoint. APIM serves as a proxy between external applications and Azure OpenAI, ensuring request validation, rate limiting, and security enforcement. The Management Plane of APIM still requires a public IP for admin operations, but the Data Plane (API traffic) remains fully private. Azure Private Endpoint for OpenAI APIM cannot access Azure OpenAI publicly since OpenAI is secured with a Private Endpoint. A Private Endpoint allows APIM to securely connect to Azure OpenAI within the same VNET, preventing internet exposure. This ensures that only APIM within the internal network can send requests to OpenAI. Managed Identity Authentication APIM uses a Managed Identity to authenticate securely with Azure OpenAI. This eliminates the need for hardcoded API keys and improves security by using Azure Role-Based Access Control (RBAC). Application Gateway for External Access Since APIM is not publicly accessible, an Azure Application Gateway (App Gateway) is placed in front of it. App Gateway acts as a reverse proxy that securely exposes APIM to the public while enforcing: SSL termination for secure HTTPS connections. Web Application Firewall (WAF) for protection against threats. Load balancing if multiple APIM instances exist. Network Segmentation & Security VNET & Subnets: APIM, OpenAI Private Endpoint, and App Gateway are deployed in separate subnets within an Azure Virtual Network (VNET). NSGs (Network Security Groups): Strict inbound and outbound rules ensure that only allowed traffic flows between components. Private DNS: Required to resolve Private Endpoint addresses inside the VNET. Security Enhancements No direct internet access to Azure OpenAI, ensuring full privacy. Controlled API exposure via App Gateway, securing public requests. Managed Identity for authentication, eliminating hardcoded credentials. Private Endpoint enforcement, blocking unwanted access from external sources. This architecture ensures that Azure OpenAI remains secure, APIM acts as a controlled gateway, and external users can access APIs safely through App Gateway. Azure CLI Script for VNet, Subnets, and NSG Configuration # Variables RESOURCE_GROUP="apim-openai-rg" LOCATION="eastus" VNET_NAME="apim-vnet" VNET_ADDRESS_PREFIX="10.0.0.0/16" # Subnets APP_GATEWAY_SUBNET="app-gateway-subnet" APP_GATEWAY_SUBNET_PREFIX="10.0.1.0/24" APIM_SUBNET="apim-subnet" APIM_SUBNET_PREFIX="10.0.2.0/24" OPENAI_PE_SUBNET="openai-pe-subnet" OPENAI_PE_SUBNET_PREFIX="10.0.3.0/24" # NSGs APP_GATEWAY_NSG="app-gateway-nsg" APIM_NSG="apim-nsg" OPENAI_PE_NSG="openai-pe-nsg" # Step 1: Create Resource Group az group create --name $RESOURCE_GROUP --location $LOCATION # Step 2: Create Virtual Network az network vnet create \ --resource-group $RESOURCE_GROUP \ --name $VNET_NAME \ --address-prefix $VNET_ADDRESS_PREFIX \ --subnet-name $APP_GATEWAY_SUBNET \ --subnet-prefix $APP_GATEWAY_SUBNET_PREFIX # Step 3: Create Additional Subnets (APIM & OpenAI Private Endpoint) az network vnet subnet create \ --resource-group $RESOURCE_GROUP \ --vnet-name $VNET_NAME \ --name $APIM_SUBNET \ --address-prefix $APIM_SUBNET_PREFIX az network vnet subnet create \ --resource-group $RESOURCE_GROUP \ --vnet-name $VNET_NAME \ --name $OPENAI_PE_SUBNET \ --address-prefix $OPENAI_PE_SUBNET_PREFIX # Step 4: Create NSGs az network nsg create --resource-group $RESOURCE_GROUP --name $APP_GATEWAY_NSG az network nsg create --resource-group $RESOURCE_GROUP --name $APIM_NSG az network nsg create --resource-group $RESOURCE_GROUP --name $OPENAI_PE_NSG # Step 5: Add NSG Rules for APIM (Allow 3443 for APIM Internal VNet) az network nsg rule create \ --resource-group $RESOURCE_GROUP \ --nsg-name $APIM_NSG \ --name AllowAPIMInbound3443 \ --priority 120 \ --direction Inbound \ --access Allow \ --protocol Tcp \ --source-address-prefixes ApiManagement \ --destination-address-prefixes VirtualNetwork \ --destination-port-ranges 3443 # Step 6: Associate NSGs with Subnets az network vnet subnet update \ --resource-group $RESOURCE_GROUP \ --vnet-name $VNET_NAME \ --name $APP_GATEWAY_SUBNET \ --network-security-group $APP_GATEWAY_NSG az network vnet subnet update \ --resource-group $RESOURCE_GROUP \ --vnet-name $VNET_NAME \ --name $APIM_SUBNET \ --network-security-group $APIM_NSG az network vnet subnet update \ --resource-group $RESOURCE_GROUP \ --vnet-name $VNET_NAME \ --name $OPENAI_PE_SUBNET \ --network-security-group $OPENAI_PE_NSG # Step 7: Configure Service Endpoints for APIM Subnet az network vnet subnet update \ --resource-group $RESOURCE_GROUP \ --vnet-name $VNET_NAME \ --name $APIM_SUBNET \ --service-endpoints Microsoft.EventHub Microsoft.KeyVault Microsoft.ServiceBus Microsoft.Sql Microsoft.Storage Microsoft.AzureActiveDirectory Microsoft.CognitiveServices Microsoft.Web Creating an Azure Open AI with private endpoint # Create an Azure OpenAI Resource az cognitiveservices account create \ --name $AOAI_NAME \ --resource-group $RESOURCE_GROUP \ --kind OpenAI \ --sku S0 \ --location $LOCATION \ --yes \ --custom-domain $AOAI_NAME #Create a Private Endpoint az network private-endpoint create \ --name $PRIVATE_ENDPOINT_NAME \ --resource-group $RESOURCE_GROUP \ --vnet-name $VNET_NAME \ --subnet $SUBNET_NAME \ --private-connection-resource-id $(az cognitiveservices account show --name $AOAI_NAME --resource-group $RESOURCE_GROUP --query id -o tsv) \ --group-id account \ --connection-name "${PRIVATE_ENDPOINT_NAME}-connection" # Create a Private DNS Zone az network private-dns zone create \ --resource-group $RESOURCE_GROUP \ --name $PRIVATE_DNS_ZONE_NAME # Link Private DNS Zone to VNet az network private-dns link vnet create \ --resource-group $RESOURCE_GROUP \ --zone-name $PRIVATE_DNS_ZONE_NAME \ --name "myDNSLink" \ --virtual-network $VNET_NAME \ --registration-enabled false # Retrieve the Private IP Address from the Private Endpoint PRIVATE_IP=$(az network private-endpoint show \ --name $PRIVATE_ENDPOINT_NAME \ --resource-group $RESOURCE_GROUP \ --query "customDnsConfigs[0].ipAddresses[0]" -o tsv) # Create a DNS Record for Azure OpenAI az network private-dns record-set a add-record \ --resource-group $RESOURCE_GROUP \ --zone-name $PRIVATE_DNS_ZONE_NAME \ --record-set-name $AOAI_NAME \ --ipv4-address $PRIVATE_IP # Disable Public Network Access az cognitiveservices account update \ --name $AOAI_NAME \ --resource-group $RESOURCE_GROUP \ --public-network-access Disabled Provisioning the Azure APIM instance to an internal VNet Please follow the link to provision: Deploy Azure API Management instance to internal VNet | Microsoft Learn Create an API for AOAI in APIM Please follow the link : Import an Azure OpenAI API as REST API - Azure API Management | Microsoft Learn Configure Azure Application Gateway with Azure APIM Please follow the link : Use API Management in a virtual network with Azure Application Gateway - Azure API Management | Microsoft Learn Conclusion Securing Azure OpenAI with private endpoints, APIM, and Application Gateway ensures a robust, enterprise-grade architecture that balances security, accessibility, and performance. By leveraging private endpoints, Azure OpenAI remains shielded from public exposure, while APIM acts as a controlled gateway for managing external API access. The addition of Application Gateway provides an extra security layer with SSL termination, WAF protection, and traffic management. With this setup, organizations can: ✔ Ensure end-to-end private connectivity for Azure OpenAI. ✔ Enable secure external access via APIM and Application Gateway. ✔ Enforce strict network segmentation with VNETs, Subnets, NSGs, and Private DNS. ✔ Strengthen security with Managed Identity authentication and controlled API exposure. By following this guide, you now have a scalable, production-ready solution to securely integrate Azure OpenAI with external applications, whether they reside in AWS, on-premises, or other cloud environments. Implement these best practices to maintain compliance, minimize security risks, and enhance the reliability of your AI-powered applications.1.1KViews3likes0Comments