How to fix outbound IPs for App Service
App Service is one of Azure Compute services categorized as PaaS Platform and has been offered since early days of Microsoft Azure. App Service is continuously improved since it had been released. Now, App Service offers container orchestrator for both Windows and Linux, VNET integration, and many other useful features to leverage Azure platform. On the other hand, PaaS services are offered as multi-tenant shared services, which might pose challenges for integration with external services or legacy applications. As one of such challenges, external services sometimes require to fix IP address of an application. In this article, we will configure to fix outbound IPs of your App Service.Calculating Chargebacks for Business Units/Projects Utilizing a Shared Azure OpenAI Instance
Azure OpenAI Service is at the forefront of technological innovation, offering REST API access to OpenAI's suite of revolutionary language models, including GPT-4, GPT-35-Turbo, and the Embeddings model series. Enhancing Throughput for Scale As enterprises seek to deploy OpenAI's powerful language models across various business units, they often require granular control over configuration and performance metrics. To address this need, Azure OpenAI Service is introducing dedicated throughput, a feature that provides a dedicated connection to OpenAI models with guaranteed performance levels. Throughput is quantified in terms of tokens per second (tokens/sec), allowing organizations to precisely measure and optimize the performance for both prompts and completions. The model of provisioned throughput provides enhanced management and adaptability for varying workloads, guaranteeing system readiness for spikes in demand. This capability also ensures a uniform user experience and steady performance for applications that require real-time responses. Resource Sharing and Chargeback Mechanisms Large organizations frequently provision a singular instance of Azure OpenAI Service that is shared across multiple internal departments. This shared use necessitates an efficient mechanism for allocating costs to each business unit or consumer, based on the number of tokens consumed. This article delves into how chargeback is calculated for each business unit based on their token usage. Leveraging Azure API Management Policies for Token Tracking Azure API Management Policies offer a powerful solution for monitoring and logging the token consumption for each internal application. The process can be summarized in the following steps: ** Sample Code: Refer to this GitHub repository to get a step-by-step instruction on how to build the solution outlined below : private-openai-with-apim-for-chargeback 1. Client Applications Authorizes to API Management To make sure only legitimate clients can call the Azure OpenAI APIs, each client must first authenticate against Azure Active Directory and call APIM endpoint. In this scenario, the API Management service acts on behalf of the backend API, and the calling application requests access to the API Management instance. The scope of the access token is between the calling application and the API Management gateway. In API Management, configure a policy (validate-jwt or validate-azure-ad-token) to validate the token before the gateway passes the request to the backend. 2. APIM redirects the request to OpenAI service via private endpoint. Upon successful verification of the token, Azure API Management (APIM) routes the request to Azure OpenAI service to fetch response for completions endpoint, which also includes prompt and completion token counts. 3. Capture and log API response to Event Hub Leveraging the log-to-eventhub policy to capture outgoing responses for logging or analytics purposes. To use this policy, a logger needs to be configured in the API Management: # API Management service-specific details $apimServiceName = "apim-hello-world" $resourceGroupName = "myResourceGroup" # Create logger $context = New-AzApiManagementContext -ResourceGroupName $resourceGroupName -ServiceName $apimServiceName New-AzApiManagementLogger -Context $context -LoggerId "OpenAiChargeBackLogger" -Name "ApimEventHub" -ConnectionString "Endpoint=sb://<EventHubsNamespace>.servicebus.windows.net/;SharedAccessKeyName=<KeyName>;SharedAccessKey=<key>" -Description "Event hub logger with connection string" Within outbound policies section, pull specific data from the body of the response and send this information to the previously configured EventHub instance. This is not just a simple logging exercise; it is an entry point into a whole ecosystem of real-time analytics and monitoring capabilities: <outbound> <choose> <when condition="@(context.Response.StatusCode == 200)"> <log-to-eventhub logger-id="TokenUsageLogger">@{ var responseBody = context.Response.Body?.As<JObject>(true); return new JObject( new JProperty("Timestamp", DateTime.UtcNow.ToString()), new JProperty("ApiOperation", responseBody["object"].ToString()), new JProperty("AppKey", context.Request.Headers.GetValueOrDefault("Ocp-Apim-Subscription-Key",string.Empty)), new JProperty("PromptTokens", responseBody["usage"]["prompt_tokens"].ToString()), new JProperty("CompletionTokens", responseBody["usage"]["completion_tokens"].ToString()), new JProperty("TotalTokens", responseBody["usage"]["total_tokens"].ToString()) ).ToString(); }</log-to-eventhub> </when> </choose> <base /> </outbound> EventHub serves as a powerful fulcrum, offering seamless integration with a wide array of Azure and Microsoft services. For example, the logged data can be directly streamed to Azure Stream Analytics for real-time analytics or to Power BI for real-time dashboards With Azure Event Grid, the same data can also be used to trigger workflows or automate tasks based on specific conditions met in the incoming responses. Moreover, the architecture is extensible to non-Microsoft services as well. Event Hubs can interact smoothly with external platforms like Apache Spark, allowing you to perform data transformations or feed machine learning models. 4: Data Processing with Azure Functions An Azure Function is invoked when data is sent to the EventHub instance, allowing for bespoke data processing in line with your organization’s unique requirements. For instance, this could range from dispatching the data to Azure Monitor, streaming it to Power BI dashboards, or even sending detailed consumption reports via Azure Communication Service. [Function("TokenUsageFunction")] public async Task Run([EventHubTrigger("%EventHubName%", Connection = "EventHubConnection")] string[] openAiTokenResponse) { //Eventhub Messages arrive as an array foreach (var tokenData in openAiTokenResponse) { try { _logger.LogInformation($"Azure OpenAI Tokens Data Received: {tokenData}"); var OpenAiToken = JsonSerializer.Deserialize<OpenAiToken>(tokenData); if (OpenAiToken == null) { _logger.LogError($"Invalid OpenAi Api Token Response Received. Skipping."); continue; } _telemetryClient.TrackEvent("Azure OpenAI Tokens", OpenAiToken.ToDictionary()); } catch (Exception e) { _logger.LogError($"Error occured when processing TokenData: {tokenData}", e.Message); } } } In the example above, Azure function processes the tokens response data in Event Hub and sends them to Application Insights telemetry, and a basic Dashboard is configured in Azure, displaying the token consumption for each client application. This information can conveniently be used to compute chargeback costs. A sample query used in dashboard above that fetches tokens consumed by a specific client: customEvents | where name contains "Azure OpenAI Tokens" | extend tokenData = parse_json(customDimensions) | where tokenData.AppKey contains "your-client-key" | project Timestamp = tokenData.Timestamp, Stream = tokenData.Stream, ApiOperation = tokenData.ApiOperation, PromptTokens = tokenData.PromptTokens, CompletionTokens = tokenData.CompletionTokens, TotalTokens = tokenData.TotalTokens Azure OpenAI Landing Zone reference architecture A crucial detail to ensure the effectiveness of this approach is to secure the Azure OpenAI service by implementing Private Endpoints and using Managed Identities for App Service to authorize access to Azure AI services. This will limit access so that only the App Service can communicate with the Azure OpenAI service. Failing to do this would render the solution ineffective, as individuals could bypass the APIM/App Service and directly access the OpenAI Service if they get hold of the access key for OpenAI. Refer to Azure OpenAI Landing Zone reference architecture to build a secure and scalable AI environment. Additional Considerations If the client application is external, consider using an Application Gateway in front of the Azure APIM If "streaming" is set to true, tokens count is not returned in response. In that that case libraries like tiktoken (Python), orgpt-3-encoder(javascript) for most GPT-3 models can be used to programmatically calculate tokens count for the user prompt and completion response. A useful guideline to remember is that in typical English text, one token is approximately equal to around 4 characters. This equates to about three-quarters of a word, meaning that 100 tokens are roughly equivalent to 75 words. (P.S. Microsoft does not endorse or guarantee any third-party libraries.) A subscription key or a custom header like app-key can also be used to uniquely identify the client as appId in OAuth token is not very intuitive. Rate-limiting can be implemented for incoming requests using OAuth tokens or Subscription Keys, adding another layer of security and resource management. The solution can also be extended to redirect different clients to different Azure OpenAI instances. For example., some clients utilize an Azure OpenAI instance with default quotas, whereas premium clients get to consume Azure Open AI instance with dedicated throughput. Conclusion Azure OpenAI Service stands as an indispensable tool for organizations seeking to harness the immense power of language models. With the feature of provisioned throughput, clients can define their usage limits in throughput units and freely allocate these to the OpenAI model of their choice. However, the financial commitment can be significant and is dependent on factors like the chosen model's type, size, and utilization. An effective chargeback system offers several advantages, such as heightened accountability, transparent costing, and judicious use of resources within the organization.Announcing the General Availability of WordPress on Azure App Service
We are thrilled to announce that WordPress on Azure App Service, which was running on Public Preview since 15 February 2022, has been made Generally Available on 8 August 2022. To read the Public Preview Announcement read the blog post on The new and better ‘WordPress on App Service’ - Microsoft Tech Community.
App Service to Storage Account Connection Condition Summary
Currently, there are 4 main conditions the Azure App Service can connect to Azure Storage Account. Condition 1: App Service (Public ) --> Storage Account (Public, Same region) Condition 2: App Service (Public ) --> Storage Account (Public, Different region) Condition 3: App Service (Regional Vnet Integration) --> Storage Account (Private, Service Endpoint ) Condition 4: App Service (Regional Vnet Integration) --> Storage Account (Private, Private Endpoint) Before going deeper, here is a brief summary for your to choose the suitable design for your system. Requirements: If your security require the Firewall on the Storage Account. And the Storage Account and Azure App Service are in the same region. --> Use the above Condition 3 for 4 for your design. If your security require the Firewall on the Storage Account. And the Storage Account and Azure App Service are in the different region. --> Use the Condition 1 or Condition 4 for your design. When you would like to make the connection private, use the Condition 4. For the deeper analysis for the above 4 conditions, please see following: Background knowledge: ============ Azure Storage Account Network restricting logic is different from the Azure App Service. Even when we configured the Network restricting from the Azure Storage Account side, the tcpping will still working well like this: And the "List" request to the Azure Storage Account will not be locked. In order to verify if the Network Restricting is working or not, you can use the script to upload a file to Azure Storage Account to test. I am using the code: <?php $accesskey = "xxxx"; $storageAccount = 'xxxx'; $filetoUpload = realpath('xxxx'); $containerName = 'xxxx'; $blobName = 'xxxx'; $destinationURL = "https://$storageAccount.blob.core.windows.net/$containerName/$blobName"; function uploadBlob($filetoUpload, $storageAccount, $containerName, $blobName, $destinationURL, $accesskey) { $currentDate = gmdate("D, d M Y H:i:s T", time()); $handle = fopen($filetoUpload, "r"); $fileLen = filesize($filetoUpload); $headerResource = "x-ms-blob-cache-control:max-age=3600\nx-ms-blob-type:BlockBlob\nx-ms-date:$currentDate\nx-ms-version:2015-12-11"; $urlResource = "/$storageAccount/$containerName/$blobName"; $arraysign = array(); $arraysign[] = 'PUT'; /*HTTP Verb*/ $arraysign[] = ''; /*Content-Encoding*/ $arraysign[] = ''; /*Content-Language*/ $arraysign[] = $fileLen; /*Content-Length (include value when zero)*/ $arraysign[] = ''; /*Content-MD5*/ $arraysign[] = 'image/png'; /*Content-Type*/ $arraysign[] = ''; /*Date*/ $arraysign[] = ''; /*If-Modified-Since */ $arraysign[] = ''; /*If-Match*/ $arraysign[] = ''; /*If-None-Match*/ $arraysign[] = ''; /*If-Unmodified-Since*/ $arraysign[] = ''; /*Range*/ $arraysign[] = $headerResource; /*CanonicalizedHeaders*/ $arraysign[] = $urlResource; /*CanonicalizedResource*/ $str2sign = implode("\n", $arraysign); $sig = base64_encode(hash_hmac('sha256', urldecode(utf8_encode($str2sign)), base64_decode($accesskey), true)); $authHeader = "SharedKey $storageAccount:$sig"; $headers = [ 'Authorization: ' . $authHeader, 'x-ms-blob-cache-control: max-age=3600', 'x-ms-blob-type: BlockBlob', 'x-ms-date: ' . $currentDate, 'x-ms-version: 2015-12-11', 'Content-Type: image/png', 'Content-Length: ' . $fileLen ]; $ch = curl_init($destinationURL); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false); curl_setopt($ch, CURLOPT_HTTPHEADER, $headers); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "PUT"); curl_setopt($ch, CURLOPT_INFILE, $handle); curl_setopt($ch, CURLOPT_INFILESIZE, $fileLen); curl_setopt($ch, CURLOPT_UPLOAD, true); $result = curl_exec($ch); echo ('Result<br/>'); print_r($result); echo ('Error<br/>'); print_r(curl_error($ch)); curl_close($ch); } uploadBlob($filetoUpload, $storageAccount, $containerName, $blobName, $destinationURL, $accesskey); Preparation: ============ Write code to upload the file to Azure Storage Account for testing as mentioned in above. Create xxx.php file under the wwwroot folder for testing. Enable the "Diagnostic settings" from Azure Portal Test: ============ Condition 1: App Service (Public ) --> Storage Account (Public, Same region) By Default, Azure App Service can access the Storage Account and upload the files. But if we enable the Firewall settings under the Networking blade of Storage Account, and add the Azure App Service Outbound IP to the whitelist like following screenshot, the Azure App Service will still not able to access Azure Storage Account. If we check the Azure Storage Account log, we will see the Azure App Service was trying to use an internal IP (100.x.x.x) to access the Storage Account. Not using the Azure App Service public outbound IP: Why? I discussed this behavior with the Azure Storage Account and Azure App Service Product Group, and the result showed if the Azure App Service and Azure Storage Account are in the same datacenter, they did some optimization, so the Azure App Service will always reach the Storage Account via the fastest route, so the resource IP (Azure App Service) observed from Azure Storage Account could be an un-predicted IP (could start with 100.x.x.x or 10.x.x.x or other others) and this IP could change at any time. Question: Considering the resource IP from Azure App Service looks like start with 100.x.x.x for now, if I whitelist the 100.0.0.0/8 in the Azure Storage Account, will the Azure App Service can upload the file on it? Answer: Yes it will make the Azure App Service can access the Storage Account for now. But cannot promise it will always work, it is a not officially support scenery. Because the IP could change to 10.x.x.x and some other un-predicted IPs. Another thing is, the x.0.0.0/8 is a huge range, so it is not a good design. If you would like to enable the Firewall in the Azure Storage Account and would like to make sure the Azure App Service in the same region still could access this Storage Account, please see the details in the Condition 3 and Condition 4. Condition 2: App Service (Public ) --> Storage Account (Public, Different region) As we can see in the Azure Storage Account log, the Azure App Service is using the public IP accessing the Azure Storage Account (13.x.x.x is one of the public outbound IP of my test Azure App Service. ) And you can enable the Firewall on Azure Storage Account and whitelist all the Azure App Service outbound IP (Inbound/Outbound IP addresses - Azure App Service | Microsoft Docs). That will make sure the Azure App Service can always access the Storage Account. Condition 3: App Service (Regional Vnet Integration) --> Storage Account (Private, Service Endpoint ) Create Vnet Integration from Azure App Service to Azure Vnet Subnet. Make sure the "Microsoft.Storage" is enabled as Service Endpoint for the Subnet that App Service is integrated with. Make sure the "Route All" is enabled for the Vnet Integration: Configure Azure Storage firewalls and virtual networks | Microsoft Docs After doing above, if we check the Azure App Service resource IP, it will be one of the IP under the Vnet Subnet: Condition 4: App Service (Regional Vnet Integration) --> Storage Account (Private, Private Endpoint) Similar configuration as the Condition 3, but please make sure: Disable the Service Endpoint on the Vnet Subnet. Create Private Endpoint on the Storage Account side. Make sure the Storage Account FQDN could be resolved to private endpoint IP from Azure App Service: We can see the similar private IP records as we saw when using the Service Endpoint. But the different between the condition 3 is the Private Endpoint support cross region traffic , and the Service Endpoint only support the connection from the Azure App Service in the same region.
App Service Linux container amid conversations.
What's communicating with what? What are the various IP addresses involved when App Service Linux container talks to a remote resource (e.g., Databases)? From the remote end of the connection, would I see a private IP or Public IP as the Linux Container source address?Cloud Rendering Adobe After Effects Video with Windows Docker Container
Since I run Newbie Homemade Mashup Lab, I always have video render needs for After Effects. When there are many videos, my personal computer will spend a lot of time rendering them. During this time, I cannot do anything else. So, I came up with the idea of Cloud Rendering. This article will guide you to build your own After Effects Docker image and ultimately try rendering on Azure App Service.
