Comprehensive Guide to Monitoring Azure WAF Metrics and Logs
In today’s digital landscape, web applications are constantly under threat from various types of attacks such as SQL injection, cross-site scripting (XSS), and distributed denial-of-service (DDoS) attacks. Azure Web Application Firewall provides robust mechanisms to protect your applications and services against such threats. In this blog post, we’ll explore how to configure and monitor Azure WAF metrics and logs for both Application Gateway v2 WAF and Azure Front Door WAF and demonstrate how to fully utilize the available metrics/logs to monitor your web applications for potential threats. Understanding Azure WAF Metrics Azure WAF offers a variety of metrics that provide insights into potential threats targeting your resources. These metrics are accessible through Azure Monitor and can be used to set up alerts and automated responses. Key metrics include: Application Gateway v1 WAF Metrics: Web Application Firewall Blocked Requests Count: Count of total requests that have been blocked by the WAF engine. Web Application Firewall Blocked Requests Distribution: Total number of rules hit distribution for the blocked requests by Rule Group and Rule ID. Web Application Firewall Total Rule Distribution: Count of total matched requests distribution by Rule Group and Rule ID. Note: The Application Gateway v1 SKU (Standard and WAF) was deprecated on April 28, 2023 and will be retired on April 28, 2026. Customers on the v1 SKU are advised to migrate to the v2 SKU. For more details, please refer to - Migrate from V1 to V2 - Azure Application Gateway | Microsoft Learn Application Gateway v2 WAF Metrics: WAF Total Requests: Count of successful requests that the WAF engine has served. WAF Managed Rule Matches: Count of total managed rule matches. WAF Custom Rule Matches: Count of custom rule matches. WAF Bot Protection Matches: Count of total bot protection rule matches that have been blocked or logged from malicious IP addresses. WAF JS Challenge Request Count: Count of requests that match JavaScript challenge WAF rules. Azure Front Door WAF Metrics: Web Application Firewall Request Count: Tracks the number of requests that match WAF rules. Web Application Firewall JS Challenge Request Count: Tracks the number of requests that match JavaScript challenge WAF rules. Note: These metrics are not found in the WAF policy resource but in the Application Gateway or Azure Front Door resource themselves. Steps to Monitor WAF Metrics Navigate to Azure Monitor: Sign in to the Azure portal and go to Azure Monitor. Select Metrics: In the Azure Monitor menu, select “Metrics.” Choose Scope: Select the scope by choosing the subscription and the specific WAF resource you want to monitor. Add Metric: Click on “Add metric” and select the desired metric, such as “WAF Total Requests” or “Web Application Firewall Request Count.” Set Aggregation Type: Choose the aggregation type. Configuring Alerts Configuring alerts for Azure WAF metrics is crucial for proactive monitoring and quick response to potential threats. Here are detailed steps to set up alerts: Navigate to Azure Monitor: Sign in to the Azure portal and go to Azure Monitor. Select Metrics: In the Azure Monitor menu, select “Metrics.” Choose Scope: Select the scope by choosing the subscription and the specific WAF resource you want to monitor. Add Metric: Click on “Add metric” and select the desired metric, such as “WAF Total Requests” or “Web Application Firewall Request Count.” Set Aggregation Type: Choose the aggregation type (e.g., Sum, Average, Maximum). Configure Alert Rule: Click on “New alert rule.” Set the signal name (e.g., “WAF Request Count Alert”). Select the aggregation type (e.g., Maximum). Choose the operator (e.g., “Greater than or equal to”). Select the unit (e.g., Count). Set the threshold value based on the average traffic you see in the chart, choosing a value lower than the WAF threshold. Click “Next” and under the Actions tab, choose how you want to be notified (e.g., email, SMS, webhook). Click on “Review + create.” WAF Alert example and use case With the increasing threat of DDoS attacks, it's vital to have alerting systems in place that keep you informed about any unusual activities in your web environment. By setting targeted alerts, you can quickly identify and address potential threats. Here's a practical example to demonstrate this. Suppose you want to be alerted about potential DDoS attacks targeting your web environment. For this scenario, we have a custom rule named "RateLimitRequest." This rule is designed to limit the number of requests to 400 within a 5-minute window, grouping the traffic by client address. This helps in identifying and mitigating potential threats from specific IP addresses. To ensure you are promptly notified of any potential DDoS attacks without enabling alerts for all rules, you can create a targeted alert based on the metric "WAF Custom Rule Match." Here's how you can configure it: Select the Signal: Navigate to the Alert page and choose "WAF Custom Rule Match" as the signal. This metric tracks the matches for your custom WAF rules. Configure Dimensions: In the "Split by dimensions" section, select "Rule name" as the dimension name. This allows you to filter the alerts based on specific rules. Set the Operator and Value: Set the operator to "=" and enter "RateLimitRequest" as the dimension value. This ensures that the alert is triggered only for the specified custom rule. In this example, I’ve configured a notification to be sent only through email. However, you can also set up notifications via SMS, the Azure Mobile app, or voice calls. Besides receiving notifications from alerts, you can configure the system to trigger various actions such as an Automation runbook, Azure Function, or Logic App. This allows you to, for instance, automatically create a ticket for the SOC team immediately through the alert rule action. By configuring the alert in this manner, you will receive notifications whenever the rate limit rule is triggered beyond a certain threshold. This indicates that a few client IPs are being blocked due to rate limiting, which could be a sign of a potential DDoS attack. Being promptly alerted allows you to take necessary actions to protect your web environment from such threats. Configuring logging In addition to monitoring metrics, logging plays a crucial role in maintaining the security and performance of your web applications. While metrics provide a high-level overview of your application's health and potential threats, logs offer detailed insights into specific events and activities. By using both metrics and logs together, you can gain a comprehensive understanding of your application's behavior, quickly identify and respond to security incidents, and ensure optimal performance. Enabling Application Gateway WAF logs To configure logging for workspace analytics for Application Gateway WAF, follow these steps: Navigate to Azure Portal: Sign in to the Azure portal. Select Application Gateway Resource: Go to your Application Gateway resource. Diagnostic Settings: Under the 'Monitoring' section, select 'Diagnostic settings'. Add Diagnostic Setting: Click on 'Add diagnostic setting'. Configure Logs: Select the logs you want to collect, such as 'ApplicationGatewayAccessLog', 'ApplicationGatewayPerformanceLog', and 'ApplicationGatewayFirewallLog' and make sure to select the 'Application Gateway Firewall Log' category to get WAF logs. Destination: Choose the destination for the logs. You can send them to a Log Analytics workspace, an Event Hub, or a Storage Account. Save: Click 'Save' to apply the settings. Note: The Application Gateway supports both Azure diagnostics tables and resource-specific tables. Make sure to use the correct query. For example, if you send logs to an Azure diagnostics table and query “AGWFirewallLogs,” you will not get any results. Enabling Azure Front Door WAF logs To set up logging for workspace analytics for Azure Front Door WAF, follow these steps: Navigate to Azure Portal: Sign in to the Azure portal. Select AFD Resource: Go to your Azure Front Door resource. Diagnostic Settings: Under the 'Monitoring' section, select 'Diagnostic settings'. Add Diagnostic Setting: Click on 'Add diagnostic setting'. Configure Logs: Choose the logs you want to collect by selecting the categories: 'FrontDoor Access Log', 'FrontDoor Health Probe Log', and 'FrontDoor WebApplicationFirewall Log'. Ensure you select the 'FrontDoor WebApplicationFirewall Log' category to capture WAF logs. Destination: Choose the destination for the logs. You can send them to a Log Analytics workspace, an Event Hub, or a Storage Account. Save: Click 'Save' to apply the settings. Running KQL queries Kusto Query Language (KQL) is a powerful tool for querying and analyzing data in Azure Monitor logs. Logs can be accessed directly under the resource, such as Application Gateway or Front Door, by navigating to the Logs section. Alternatively, you can query the logs from your Log Analytics workspace. Below are some useful KQL queries for monitoring Azure WAF metrics, along with scenarios to help you understand their use cases. Identify Top IP Offenders To identify the top IP addresses that are being blocked by your Azure WAF, you can use the following query for Application Gateway WAF logs: AzureDiagnostics | where Category == "ApplicationGatewayFirewallLog" | summarize count() by clientIp_s | top 10 by count_ Scenario: Suppose you notice an unusual spike in blocked requests on your Azure WAF. You want to identify the top IP addresses that are being blocked to understand if there is a targeted attack from specific IPs. This query will help you list the top offending IP addresses, allowing you to take further action, such as blocking these IPs at the firewall level. Azure Front Door WAF Query To monitor the top IP addresses being blocked by Azure Front Door WAF, use this query: AzureDiagnostics | where Category == "FrontDoorWebApplicationFirewallLog" | summarize count() by clientIP_s | top 10 by count_ Top Five Matched Rules To understand which WAF rules are being triggered the most, you can use the following query for Application Gateway WAF logs: AzureDiagnostics | where Category == "ApplicationGatewayFirewallLog" | summarize count() by ruleId_s | top 5 by count_ Scenario: You want to identify potential focused attacks or false positives by understanding which WAF rules are being triggered the most. By running this query, you can see the top five rules with the highest number of matches. This information can help you fine-tune your WAF rules to reduce false positives or strengthen your defenses against specific attack vectors. Azure Front Door WAF Query To monitor the top rules being triggered by Azure Front Door WAF, use this query: AzureDiagnostics | where Category == "FrontDoorWebApplicationFirewallLog" | summarize count() by ruleName_s | top 5 by count_ Matched/Blocked Requests by URI To understand which parts of your web application are being targeted the most, use this query to identify the top URIs that are being matched or blocked by the Application Gateway WAF logs: AzureDiagnostics | where Category == "ApplicationGatewayFirewallLog" | project TimeGenerated, clientIp_s, requestUri_s, ruleId_s, action_s, Message | summarize count() by requestUri_s | top 10 by count_ This query helps you pinpoint vulnerable areas of your application by identifying the top URIs that are being matched or blocked by the WAF. You can then take appropriate measures to secure these areas. Azure Front Door WAF Query To monitor the top URIs being matched or blocked by Azure Front Door WAF, use this query: AzureDiagnostics | where Category == "FrontDoorWebApplicationFirewallLog" | project TimeGenerated, clientIP_s, requestUri_s, ruleName_s, action_s, details_msg_s | summarize count() by requestUri_s | top 10 by count_ Best Practices for Monitoring WAF Metrics and Logs Enable Logging: Ensure that logging is enabled for all WAF resources. This includes enabling diagnostic logs and integrating them with Azure Monitor logs. Use Managed Rules: Utilize managed rules provided by Azure WAF to protect against common threats. Regularly update these rules to ensure they cover the latest vulnerabilities. Set Up Alerts: Configure alerts for critical metrics such as “WAF Managed Rule Matches” and “WAF Custom Rule Matches” to get notified of potential threats in real-time. Regularly Review Logs: Periodically review WAF logs to identify patterns and trends in the traffic. This helps in fine-tuning the WAF rules and improving the overall security posture. Implement Exclusions: Use exclusions to fine-tune rules and exclude any data that you want to be excluded from the logs. This helps in reducing false positives and focusing on actual threats. Monitor Traffic Thresholds: Compare your real traffic to the thresholds set by Azure WAF to determine if you are close to or far from them. Adjust the thresholds as needed to suit your environment and downstream architecture. Integrate with SIEM/SOAR: Utilize Azure Sentinel for creating Workbooks and automating responses to threats. This integration allows for enhanced monitoring and automated threat response, ensuring a more robust security posture. For more details, you can refer to https://learn.microsoft.com/en-us/azure/web-application-firewall/waf-sentinel Real-World Scenarios for WAF Metrics and Logs High Volume of Requests: If you notice a sudden spike in the “WAF Total Requests” metric, it could indicate a potential DDoS attack. Setting up alerts for this metric can help you respond quickly to mitigate the attack. To configure WAF for high volume of requests: Scaling: Ensure your Application Gateway is set to auto scale. For Application Gateway v2 SKU, set the maximum instance count to the maximum possible value of 125 to handle traffic surges. Rate Limiting: Implement rate limiting to control the number of requests from specific IP addresses or geographies. Frequent Rule Matches: A high number of matches for “WAF Managed Rule Matches” or “WAF Custom Rule Matches” could indicate that your application is being targeted by specific types of attacks. Reviewing the logs and adjusting the rules can help in blocking these attacks more effectively. To configure WAF for frequent rule matches: Custom Rules: Create custom rules to block or allow traffic based on specific criteria. For example, you can block all requests from a specific IP address range. Detection Mode: Set the WAF policy to detection mode to log requests that match custom rules without blocking them, allowing you to analyze the traffic patterns. Bot Protection: If the “WAF Bot Protection Matches” metric shows a high count, it could indicate that your application is being targeted by malicious bots. Implementing bot protection rules and monitoring this metric can help in blocking these bots and protecting your application. To configure WAF for bot protection: Bot Protection Rule Set: Enable the managed bot protection rule set in your WAF policy to block or log requests from known malicious IP addresses. JavaScript Challenge: Enable the JavaScript challenge action for bot rules. This helps in blocking bots and reducing false positives by challenging requests that match specific criteria. JavaScript Challenge: The “WAF JS Challenge Request Count” metric tracks the number of requests that match JavaScript challenge WAF rules. A high count could indicate that your application is being targeted by automated scripts. Monitoring this metric and adjusting the challenge rules can help in mitigating such attacks. Key Takeaways Azure Web Application Firewall offers robust protection mechanisms for web applications. By monitoring Azure WAF metrics, you can gain valuable insights into the security and performance of your web applications. Proactively monitoring and analyzing WAF logs using Kusto Query Language (KQL) helps identify and mitigate potential threats. This approach ensures your web applications remain secure and performant, providing a seamless experience for your users. Understanding and fine-tuning WAF policies and rules reduce false positives and focus on actual threats. Regularly reviewing WAF logs helps identify patterns and trends, improving your security strategies continuously. In conclusion, staying vigilant and proactive in your WAF protection strategy is essential for ensuring uninterrupted service delivery and safeguarding your web applications from potential threats. By fully utilizing the available metrics and logs, you can effectively monitor and protect your resources, providing a secure and reliable experience for your users. References Monitoring metrics for Azure Application Gateway Web Application Firewall metrics | Microsoft Learn Azure Monitor metrics for Application Gateway | Microsoft Learn Azure Web Application Firewall monitoring and logging | Microsoft Learn https://learn.microsoft.com/en-us/azure/web-application-firewall/waf-sentinelAzure WAF’s Bot Manager 1.1 and JavaScript Challenge: Navigating the Bot Threat Terrain
Discover how Azure WAF’s Bot Manager 1.1 and the JavaScript Challenge elevate your bot protection strategy. In this blog, we explore how these advanced features in Azure Front Door and Application Gateway provide a robust defense against malicious bots while ensuring legitimate traffic flows smoothly. Learn how to fine-tune your web application’s security to safeguard against evolving bot threats without compromising on performance.Independent Configuration of Size Enforcement and Inspection Limits in Application Gateway WAF
Explore how the latest update to Azure's Application Gateway WAF enhances your security options by allowing independent configuration of size enforcement and inspection limits. This blog delves into the flexible security adjustments you can make to meet the specific needs of your applications, ensuring robust protection against a variety of threats while maintaining optimal performance. Learn how to navigate these new features for an improved security management experience.
