SSL/TLS connection issue troubleshooting guide
You may experience exceptions or errors when establishing TLS connections with Azure services. Exceptions are vary dramatically depending on the client and server types. A typical ones such as "Could not create SSL/TLS secure channel." "SSL Handshake Failed", etc. In this article we will discuss common causes of TLS related issue and troubleshooting steps.Managing Security Center at scale using ARM templates and Azure Policy
*** Update: This blogpost has been updated with a new ARM template and new Azure Policy definitions which covers the new Security Center bundle pricings. The new ARM template can be found here, the new Azure Policy definitions can be found here *** Recently we have been receiving several customer questions on how to manage Security Center at scale in a continuous integration (CI) and continuous delivery (CD) scenario, better known as CI/CD. How do you make sure that when a new subscription is instantiated, Security Center is configured correctly and is enabled to monitor new and existing resources? How do you manage hundreds of subscriptions within your organizattion? How do you enforce your security policies? This blogpost covers two scenarios: Configure Security Center using an ARM template to support a CI/CD scenario and management at scale Enforce a Security Center configuration within your organization, using Azure Policy Since both ARM templates and Azure Policy talk to the Resource Manager API in Azure, by making a JSON formatted request, you can re-use the deployment section of an ARM template to author an Azure Policy definition. If you are new to ARM templates and are looking for guidance on authoring, go here. In case you want to leverage PowerShell for configuring ASC, go here. The Security Center ARM Template Every ARM template consists of these 7 elements (not all of them have to be used): It’s the “resources” element that we are interested in since the ARM template reference for Security Center hasn’t been documented yet. We are in the process of publishing which makes it easier to find which types and values are allowed and will also provide IntelliSense. I will update this post when that becomes available. In this blogpost I will cover the two most asked questions: How can I make sure that Security Center is configured for the Standard pricing tier, which unlocks all the Security Center features How can I enable auto provisioning, which enables automatic installation of the Microsoft Management Agent (MMA) VM extension for new resources. Please refer to the pricing tier documentation for the difference between the Free and Standard tier. Set the ASC pricing tier in an ARM template The new pricing tier API is fast and efficient to use. We are using the Microsoft.Security/pricings type to set our pricing tier. How does that look in a template? The ARM template can be found here. If you want to deploy this template, please make sure you target the subscription instead of a resource group. This is a common made mistake, since ASC lives at the subscription level, not at the resource group level. A deployment would look like this (using PowerShell): New-AzDeployment -Name myAscDeploy -Location <yourLocation> -TemplateFile ‘<yourTemplateFileAndPathHere>' -Verbose Since you have probably noticed the usage of a parameter, so you can flip it from “Free” to “Standard” or the other way around, you will be prompted for this value. Please note that we are only allowing the two values specified and it is case sensitive. You can verify the deployment in the activity log: Leveraging the ARM template to create an Azure Policy Now that you have a working ARM template, you can use it to create a deployIfNotExists type of Azure policy which allows you to remediate if the policy definition is non-compliant. If you are just starting with Azure Policy, I would recommend to explore our documentation for guidance on different policy definitions. Leveraging the ARM template we just created, you can construct a deployIfNotExists Azure Policy. You first need to create the section that allows you to find the pricingTier field and value (Standard). For this to work, you need to target your Policy at the right scope (subscriptions or higher), then you define the “effect” (deployIfNotExists) and you set the type to Microsoft.Security/pricings, where the field and value lives which we are looking for. If the existenceCondition returns false and you want to remediate it, you need to define a deployment section under the resources section, where you define your target type and properties. You can find the Azure Policy shown above here. Having your ARM Policy defined, you can now create and assign your new Azure Policy. Please note that you need the appropriate permissions to create a managed identity. This ensures that the policy has the appropriate permissions to change the Security Center configuration at the subscription level. It takes around 30 minutes for the policy to take effect as mentioned in the confirmation. Your policy will remain in this state for a while: Then after around 30 minutes, you will notice a non-compliant assignment, as shown below. If you click on the assignment, you can explore which resource is non-compliant. You can click on Create Remediation Task to remediate it. Automatically create a remediation task In case you want to use automation, you can leverage the Policy Insights API to create a remediation task. One of the options you have is to invoke the API using PowerShell. The script below shows an example how to accomplish that. Please note that besides the subscriptionId, you need to pass a remediationName and policyID: You can find the script to create a remediation task here. If you have succesfully executed the script, you should see something similar in your activity log: Your ASC pricing tier should be changed to Standard and after a policy refresh cycle, your compliance state should be updated to compliant. Configure Security Center auto provisioning using a custom workspace The following ARM template enables auto provisioning so that the Microsoft Management Agent VM extension will be deployed automatically to new created virtual machines. You will also configure a custom Log Analytics workspace instead of the default Security Center one. The ARM template for enabling auto provisioning can be found here. Please note that you need to pass specific parameters values for configuring the Log Analytics workspace like workspace name, the resource group of the workspace and the Azure subscriptionID of where the workspace is created. Deployment of the ARM template is straightforward as you have seen before while deploying the pricing tier template and should look like this:
Enable HTTPS setting on Azure App service using Azure policy
Use Case: By default, clients can connect to Azure App Service endpoints by using both HTTP or HTTPS. However it is always recommended to redirect HTTP to HTTPs because HTTPS uses the SSL/TLS protocol to provide a secure connection, which is both encrypted and authenticated. In this post , we will learn how to enable HTTPS Only setting on the Azure App service using Azure Policy. Background: Azure App Service is a fully managed platform as a service (PaaS) offering for developers. It is a HTTP-based service for hosting web applications, REST APIs, and mobile back ends. App Service not only adds the power of Microsoft Azure to your application, such as security, load balancing, autoscaling, and automated management , you can also take advantage of its DevOps capabilities, such as continuous deployment from Azure DevOps, GitHub, Docker Hub, and other sources, package management, staging environments, custom domain, and TLS/SSL certificates. Azure Policy is a service in Azure which helps to enforce organizational standards and to assess compliance. Policy evaluates resources in Azure by comparing the properties of those resources to business rules which are defined in the Policy. These business rules, described in JSON format, are known as policy definitions. To learn more about policy and how to create a policy you can refer this document. To ensure proper governance and considering the organization has a requirement to enforce HTTPS on both new and existing App services within your environment we are going to achieve this using Azure Policy. Prerequisites: An Azure Subscription. If you don't have an Azure subscription, create an Azure free account before you begin. You need to have required permissions to create and manage Azure policy definitions in your subscription. You can refer to Resource Policy Contributor or Security Admin role. Since this policy is using DeployIfNotExists effect, policy assignments will require a managed identity to do remediation. Create a custom policy to Enable HTTPS on App Service: In the Azure portal, select All services > search for Policy > go to Definitions > click + Policy definition. Select Definition location by clicking on the ellipsis [...] and choose your management group or subscription. Fill Name field with your policy definition name like 'Enable HTTPS on App Service' and add the Description. Select appropriate Category for your policy, you can create a new one or use existing one like 'App Service'. Here is the JSON Policy definition which you need to add in the Policy Rule section and then click on Save. DISCLAIMER: Below sample policy definition is not supported under any Microsoft standard support program or service. This is intended to be used in non-production environment only. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, owners of this GitHub repro, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages. { "mode": "All", "policyRule": { "if": { "allOf": [ { "field": "type", "equals": "Microsoft.Web/sites" }, { "field": "kind", "like": "app*" } ] }, "then": { "effect": "[parameters('effect')]", "details": { "type": "Microsoft.Web/sites", "name": "[field('name')]", "existenceCondition": { "field": "Microsoft.Web/sites/httpsOnly", "equals": "true" }, "roleDefinitionIds": [ "/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c" ], "deployment": { "properties": { "mode": "incremental", "parameters": { "webAppName": { "value": "[field('name')]" }, "location": { "value": "[field('location')]" }, "kind": { "value": "[field('kind')]" } }, "template": { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "webAppName": { "type": "string" }, "location": { "type": "string" }, "kind": { "type": "string" } }, "resources": [ { "type": "Microsoft.Web/sites", "apiVersion": "2021-02-01", "name": "[parameters('webAppName')]", "location": "[parameters('location')]", "kind": "[parameters('kind')]", "properties": { "httpsOnly": "true" } } ] } } } } } }, "parameters": { "effect": { "type": "String", "metadata": { "displayName": "Effect", "description": "Enable or disable the execution of the policy" }, "allowedValues": [ "DeployIfNotExists", "AuditIfNotExists", "Disabled" ], "defaultValue": "DeployIfNotExists" } } } Ensure that the roleDefinitionIds contains enough permissions to enable HTTPS settings on the resource, you can select Contributor role. If the ExistenceCondition evaluates to true, the policy doesn't trigger the deployment and if the condition evaluates to false the HTTPS deployment happens. Refer to this document to learn more about DeployIfNotExists and ExistenceCondition. Assign the custom policy: Open the Azure portal ,select All services > search for Policy > go to Assignments> select Assign policy. On the Assign Policy page, set the Scope by selecting the ellipsis and then selecting either a management group or subscription. Optionally, select a resource group. A scope determines what resources or grouping of resources the policy assignment gets enforced on. Then use the Select button at the bottom of the Scope page. Resources can be excluded based on the Scope. Exclusions start at one level lower than the level of the Scope. Exclusions are optional, so leave it blank for now. Select the Policy definition ellipsis to open the list of available definitions. Search for the custom policy 'Enable HTTPS on App Service' that you have created in the previous step and then click on Select. The Assignment name is automatically populated with the policy name you selected, but you can change it. For this example, leave Enable HTTPS on App Service. You can also add an optional Description. The description provides details about this policy assignment. Assigned by will automatically fill based on who is logged in. This field is optional, so custom values can be entered. Leave policy enforcement Enabled. For more information, see Policy assignment - enforcement mode. Select Next at the bottom of the page or the Parameters tab at the top of the page to move to the next segment of the assignment wizard. If the policy definition selected on the Basics tab included parameters, they are configured on this tab. Since our definition has parameters (DeployIfNotExists , AuditIfNotExists , Disabled) , by default DeployIfNotExists is selected. In the Remediation tab, you can see that "Create a Managed Identity" is selected by default as the policy effect is DeployIfNotExists. Policies with the deployIfNotExists and modify effect types need the ability to deploy or modify the resources To do this, choose between an existing user assigned managed identity or creating a system assigned managed identity. This identity will also be given the Contributor permissions. For more information , refer managed identities. To evaluate the existing resources, you can select "Create a remediation task" as shown below. In the next page, set your desired message in Non-compliance message field. This custom message is displayed when a resource is denied or for non-compliant resources during regular evaluation. Then click Review + Create. Review the selected options, then select Create at the bottom of the page. Verify if the policy evaluation triggered: It takes around 30 minutes for the policy assignment to be applied to the defined scope and then the evaluation cycle begins for resources within that scope against the newly assigned policy and depending on the effects used by the policy or initiative, resources are marked as compliant, non-compliant, or exempt. Also for every 24 hours, there is a standard compliance evaluation cycle which will trigger and assignments are automatically reevaluated. Also you can trigger an On demand evaluation scan. You're now ready to identify non-compliant resources to understand the compliance state of your environment. Identify non-compliant resources: Select Compliance in the left side of the page. Then locate the 'Enable HTTPS on App Service' policy assignment you created. If there are any existing App Service resources that don't have HTTPS enabled, they appear under Non-compliant resources. Remember that when a condition is evaluated against your existing App Service resources and HTTPS is not enabled, then those resources are marked as non-compliant but no action is taken on these existing resources. For this reason, we have created a remediation task during the policy assignment. On the left side, click on Remediation and locate with your Policy definition name and you can see that the existing non-compliant resources got remediated successfully. And after the remediation run, all the App Services in your environment will get marked as compliant. Now that you can verify the HTTPS setting is enabled on your App Service now. Go to your App Service > TLS/SSL settings > HTTPS Only Note : This Policy is used to enable HTTPS setting for App Service resources only. But you can use the same policy rule for function app by modifying the below field. { "field": "kind", "like": "functionapp*" } This way, we can use Azure Policy to Enable HTTPS setting on Azure App Services. To learn more about Azure Policy, refer to the following documentation Azure Policy
Managing Azure Policies through Python SDK
Managing Azure Policies through Python SDK Azure Policy helps to enforce organizational standards and to assess compliance at-scale. It also helps to bring your resources to compliance through bulk remediation for existing resources and automatic remediation for added resources. Common use cases for Azure Policy include implementing governance for resource consistency, regulatory compliance, security, cost, and management. Policy definitions for these common use cases are already available in your Azure environment as built-ins to help you get started. Specifically, some useful governance actions you can enforce with Azure Policy include: Ensuring your team deploys Azure resources only to allowed regions. Enforcing the consistent application of taxonomic tags Requiring resources to send diagnostic logs to a Log Analytics workspace References: https://learn.microsoft.com/en-us/azure/governance/policy/overview https://learn.microsoft.com/en-us/azure/governance/policy/policy-glossary https://learn.microsoft.com/en-us/azure/developer/python/sdk/azure-sdk-overview We can manage Azure policies through portal, PowerShell, CLI, REST API, Bicep, ARM Templates, Terraform and SDKs. This blog will cover the policy management through Python SDK and we can use any IDE that supports Python SDK for Azure. We are using Visual Studio Code here. (https://azure.microsoft.com/en-in/products/visual-studio-code) Azure Python SDK Authentication Reference : https://learn.microsoft.com/en-us/azure/developer/python/sdk/authentication-overview Import Libraries import os from azure.identity import ClientSecretCredential from azure.identity import AzureCliCredential from azure.mgmt.authorization import AuthorizationManagementClient from azure.mgmt.resource import PolicyClient, ResourceManagementClient from azure.mgmt.resource.subscriptions import SubscriptionClient from azure.mgmt.policyinsights import PolicyInsightsClient Define and Assign Variables subscription_id = "exyx-exyx4e-4xyx9-axyz9c-45be63c6a8ad" # your subscription ID tenant_id = "xyzxyz-abc-cd-avc-48ebcd07d17c" # Your tenant ID client_id = "123abc-cdd-4f09-ad9f-abcdef" # Your Client ID client_secret = "wiufhuiw24874946497fff" # Your Client Secret POLICY_NAME = "KeyVaultDIAGDINEpolicy" SUBSCRIPTION_ID = "exyx-exyx4e-4xyx9-axyz9c-45be63c6a8ad" GROUP_NAME = "resourceGroupName" MANAGEMENTGROUP_ID= "abcd-123-4e7b-a446-233cvd" POLICY_ASSIGNMENT_NAME = "KVDiagDINEpolicy" Creation of objects resource_client = ResourceManagementClient( credential=ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret), subscription_id=subscription_id ) policyinsights_client = PolicyInsightsClient( credential=ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret), subscription_id=subscription_id ) policy_client = PolicyClient( credential=ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret), subscription_id=subscription_id ) Creation of Policy Definition at the Subcription Scope definitionatsubscription = policy_client.policy_definitions.create_or_update(policy_definition_name=POLICY_NAME, parameters= { "displayName": "KVDiag", "policyType": "Custom", "mode": "Indexed", "parameters": { "effect": { "type": "String", "metadata": { "displayName": "Effect", "description": "Enable or disable the execution of the policy" }, "allowedValues": [ "DeployIfNotExists", "Disabled" ], "defaultValue": "DeployIfNotExists" }, "profileName": { "type": "String", "metadata": { "displayName": "Profile name", "description": "The diagnostic settings profile name" } }, "logAnalytics": { "type": "String", "metadata": { "displayName": "Log Analytics workspace", "description": "Select Log Analytics workspace from dropdown list. If this workspace is outside of the scope of the assignment you must manually grant 'Log Analytics Contributor' permissions (or similar) to the policy assignment's principal ID.", "strongType": "omsWorkspace", "assignPermissions": "true" } }, "azureRegions": { "type": "Array", "metadata": { "displayName": "Allowed Locations", "description": "The list of locations that can be specified when deploying resources", "strongType": "location" } }, "metricsEnabled": { "type": "String", "metadata": { "displayName": "Enable metrics", "description": "Whether to enable metrics stream to the Log Analytics workspace - True or False" }, "allowedValues": [ "True", "False" ], "defaultValue": "True" }, "logsEnabled": { "type": "String", "metadata": { "displayName": "Enable logs", "description": "Whether to enable logs stream to the Log Analytics workspace - True or False" }, "allowedValues": [ "True", "False" ], "defaultValue": "True" } }, "policyRule": { "if": { "allOf": [ { "field": "type", "equals": "Microsoft.KeyVault/vaults" }, { "field": "location", "in": "[parameters('azureRegions')]" } ] }, "then": { "effect": "[parameters('effect')]", "details": { "type": "Microsoft.Insights/diagnosticSettings", "existenceCondition": { "allOf": [ { "field": "Microsoft.Insights/diagnosticSettings/logs.enabled", "equals": "[parameters('logsEnabled')]" }, { "field": "Microsoft.Insights/diagnosticSettings/metrics.enabled", "equals": "[parameters('metricsEnabled')]" }, { "field": "Microsoft.Insights/diagnosticSettings/workspaceId", "equals": "[parameters('logAnalytics')]" } ] }, "roleDefinitionIds": [ "/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa", "/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293" ], "deployment": { "properties": { "mode": "incremental", "template": { "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "resourceName": { "type": "string" }, "location": { "type": "string" }, "logAnalytics": { "type": "string" }, "metricsEnabled": { "type": "string" }, "logsEnabled": { "type": "string" }, "profileName": { "type": "string" } }, "variables": {}, "resources": [ { "type": "Microsoft.KeyVault/vaults/providers/diagnosticSettings", "apiVersion": "2017-05-01-preview", "name": "[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('profileName'))]", "location": "[parameters('location')]", "dependsOn": [], "properties": { "workspaceId": "[parameters('logAnalytics')]", "metrics": [ { "category": "AllMetrics", "enabled": "[parameters('metricsEnabled')]", "retentionPolicy": { "enabled": "false", "days": 0 } } ], "logs": [ { "category": "AuditEvent", "enabled": "[parameters('logsEnabled')]" } ] } } ], "outputs": {} }, "parameters": { "location": { "value": "[field('location')]" }, "resourceName": { "value": "[field('name')]" }, "logAnalytics": { "value": "[parameters('logAnalytics')]" }, "metricsEnabled": { "value": "[parameters('metricsEnabled')]" }, "logsEnabled": { "value": "[parameters('logsEnabled')]" }, "profileName": { "value": "[parameters('profileName')]" } } } } } } }}) print("Created policy definition ID : {}".format(definitionatsubscription.id)) print("Created policy definition Name : {} \n".format(definitionatsubscription.name)) print (" ********* ******** ******** ********** ******** ******* \n") Creation of Policy Definition at Management Group Scope definitionatmanagementgroup = policy_client.policy_definitions.create_or_update_at_management_group(policy_definition_name=POLICY_NAME, parameters= { "displayName": "KVDiag", "policyType": "Custom", "mode": "Indexed", "parameters": { "effect": { "type": "String", "metadata": { "displayName": "Effect", "description": "Enable or disable the execution of the policy" }, "allowedValues": [ "DeployIfNotExists", "Disabled" ], "defaultValue": "DeployIfNotExists" }, "profileName": { "type": "String", "metadata": { "displayName": "Profile name", "description": "The diagnostic settings profile name" } }, "logAnalytics": { "type": "String", "metadata": { "displayName": "Log Analytics workspace", "description": "Select Log Analytics workspace from dropdown list. If this workspace is outside of the scope of the assignment you must manually grant 'Log Analytics Contributor' permissions (or similar) to the policy assignment's principal ID.", "strongType": "omsWorkspace", "assignPermissions": "true" } }, "azureRegions": { "type": "Array", "metadata": { "displayName": "Allowed Locations", "description": "The list of locations that can be specified when deploying resources", "strongType": "location" } }, "metricsEnabled": { "type": "String", "metadata": { "displayName": "Enable metrics", "description": "Whether to enable metrics stream to the Log Analytics workspace - True or False" }, "allowedValues": [ "True", "False" ], "defaultValue": "True" }, "logsEnabled": { "type": "String", "metadata": { "displayName": "Enable logs", "description": "Whether to enable logs stream to the Log Analytics workspace - True or False" }, "allowedValues": [ "True", "False" ], "defaultValue": "True" } }, "policyRule": { "if": { "allOf": [ { "field": "type", "equals": "Microsoft.KeyVault/vaults" }, { "field": "location", "in": "[parameters('azureRegions')]" } ] }, "then": { "effect": "[parameters('effect')]", "details": { "type": "Microsoft.Insights/diagnosticSettings", "existenceCondition": { "allOf": [ { "field": "Microsoft.Insights/diagnosticSettings/logs.enabled", "equals": "[parameters('logsEnabled')]" }, { "field": "Microsoft.Insights/diagnosticSettings/metrics.enabled", "equals": "[parameters('metricsEnabled')]" }, { "field": "Microsoft.Insights/diagnosticSettings/workspaceId", "equals": "[parameters('logAnalytics')]" } ] }, "roleDefinitionIds": [ "/providers/microsoft.authorization/roleDefinitions/749f88d5-cbae-40b8-bcfc-e573ddc772fa", "/providers/microsoft.authorization/roleDefinitions/92aaf0da-9dab-42b6-94a3-d43ce8d16293" ], "deployment": { "properties": { "mode": "incremental", "template": { "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "resourceName": { "type": "string" }, "location": { "type": "string" }, "logAnalytics": { "type": "string" }, "metricsEnabled": { "type": "string" }, "logsEnabled": { "type": "string" }, "profileName": { "type": "string" } }, "variables": {}, "resources": [ { "type": "Microsoft.KeyVault/vaults/providers/diagnosticSettings", "apiVersion": "2017-05-01-preview", "name": "[concat(parameters('resourceName'), '/', 'Microsoft.Insights/', parameters('profileName'))]", "location": "[parameters('location')]", "dependsOn": [], "properties": { "workspaceId": "[parameters('logAnalytics')]", "metrics": [ { "category": "AllMetrics", "enabled": "[parameters('metricsEnabled')]", "retentionPolicy": { "enabled": "false", "days": 0 } } ], "logs": [ { "category": "AuditEvent", "enabled": "[parameters('logsEnabled')]" } ] } } ], "outputs": {} }, "parameters": { "location": { "value": "[field('location')]" }, "resourceName": { "value": "[field('name')]" }, "logAnalytics": { "value": "[parameters('logAnalytics')]" }, "metricsEnabled": { "value": "[parameters('metricsEnabled')]" }, "logsEnabled": { "value": "[parameters('logsEnabled')]" }, "profileName": { "value": "[parameters('profileName')]" } } } } } } }},management_group_id= 'abcd-233df-4e7b-a446-2333xcvc' ) print("Created policy definition ID : {}".format(definitionatmanagementgroup.id)) print("Created policy definition Name : {} \n".format(definitionatmanagementgroup.name)) print (" ********* ******** ******** ********** ******** ******* \n") Defining Scopes for Policy Assignment scopeRG = '/subscriptions/{}/resourceGroups/{}'.format( SUBSCRIPTION_ID, GROUP_NAME ) scopeSubscription = '/subscriptions/{}'.format( SUBSCRIPTION_ID ) scopeMG = '/providers/Microsoft.Management/managementGroups/{}'.format( MANAGEMENTGROUP_ID ) Creating Policy Assignments at different Scope - Resource Group, Subscription and Management Group assignmentatRG = policy_client.policy_assignments.create( scopeRG, POLICY_ASSIGNMENT_NAME + " at RG", { 'policy_definition_id': definitionatmanagementgroup.id, "identity": {"type": "SystemAssigned"}, "location" :"eastus", 'parameters' : { "profileName": { "value": "KV_DIAG_Settings1" }, "logAnalytics": { "value": "/subscriptions/e818bd2d-e44e-4a99-a89c-45be63c6a8ad/resourcegroups/psdonotdeletessimportant/providers/microsoft.operationalinsights/workspaces/lnos-demo-1-log-sb" }, "azureRegions": { "value": ["westus"] } }, } ) print("Createed policy assignment: {}".format(assignmentatRG.id)) print("Created policy assignment: {}\n".format(assignmentatRG.name)) print (" ********* ******** ******** ********** ******** ******* \n") assignmentatsubscription = policy_client.policy_assignments.create( scopeSubscription, POLICY_ASSIGNMENT_NAME + "at Subscription", { 'policy_definition_id': definitionatsubscription.id, "identity": {"type": "SystemAssigned"}, "location" :"eastus", 'parameters' : { "profileName": { "value": "KV_DIAG_Settings2" }, "logAnalytics": { "value": "/subscriptions/e818bd2d-e44e-4a99-a89c-45be63c6a8ad/resourcegroups/psdonotdeletessimportant/providers/microsoft.operationalinsights/workspaces/lnos-demo-1-log-sb" }, "azureRegions": { "value": ["westus"] } } } ) print("Created policy assignment: {}".format(assignmentatsubscription.id)) print("Created policy assignment: {} \n".format(assignmentatsubscription.name)) print (" ********* ******** ******** ********** ******** ******* \n") assignmentatmanagementgroup = policy_client.policy_assignments.create( scopeMG, POLICY_ASSIGNMENT_NAME + "at MG", { 'policy_definition_id': definitionatmanagementgroup.id, "identity": {"type": "SystemAssigned"}, "location" :"eastus", 'parameters' : { "profileName": { "value": "KV_DIAG_Settings3" }, "logAnalytics": { "value": "/subscriptions/e818bd2d-e44e-4a99-a89c-45be63c6a8ad/resourcegroups/psdonotdeletessimportant/providers/microsoft.operationalinsights/workspaces/lnos-demo-1-log-sb" }, "azureRegions": { "value": ["westus"] } } } ) print("Created policy assignment: {}".format(assignmentatmanagementgroup.id)) print("Created policy assignment: {}\n".format(assignmentatmanagementgroup.name)) print (" ********* ******** ******** ********** ******** ******* \n") Trigger manual policy evaluation and get the resource compliance/non-compliance count : Refernce Article : https://learn.microsoft.com/en-us/powershell/module/az.policyinsights/start-azpolicycompliancescan?view=azps-10.4.1 triggerevaluation = policyinsights_client.policy_states.begin_trigger_subscription_evaluation( subscription_id='abc1233-e44e-4a99-a89c-123dfg') print("Manual policy evaluation triggered :\n{}".format(triggerevaluation.status)) policyevaluationresult = policyinsights_client.policy_states.list_query_results_for_subscription_level_policy_assignment( subscription_id='abc123-e44e-4a99-a89c-12333df', policy_states_resource="latest", policy_assignment_name= '621dc2655dd74ed68a37286d' ) for item in policyevaluationresult: print(' Resource ID : '+ item.resource_id) print(' Policy Assignment ID : '+item.policy_assignment_id) print(' Policy Assignment Scope : '+item.policy_assignment_scope) print(' Compliance State : '+item.compliance_state) print('\n') if item.compliance_state == 'Compliant': Compliant= int(Compliant +1) elif item.compliance_state == 'NonCompliant': NonCompliant = int(NonCompliant +1) else: 1==1 print('Number of Resources') print('Compliant :' + str(Compliant)) print('NonCompliant :' + str(NonCompliant)) Creation of Remediation Task at different Scopes: remediation = policyinsights_client.remediations.create_or_update_at_management_group("1233-dfdf-4e7b-a446-223233", "mymgremediation" , { "policy_assignment_id": "/providers/microsoft.management/managementgroups/6cf4d5f3-dfdf-4e7b-a446-48ebcd07d17c/providers/microsoft.authorization/policyassignments/logagents", ## "policy_definition_reference_id" : "4037797753452518688" ## (optional) Needed when we need to remediate the non-compliant resources of policy initiative assignment }) print("Create remediation:\n{}".format(remediation)) remediationRG = policyinsights_client.remediations.create_or_update_at_resource_group( "myremediationatRG", "aro-rererer", { "policy_assignment_id": "/providers/microsoft.management/managementgroups/12345-dfdf-4e7b-a446-48ebcd07d17c/providers/microsoft.authorization/policyassignments/logagents", "resource_discovery_mode" : 'ExistingNonCompliant' ## ReEvaluateCompliance - Re-evaluate the compliance state of resources and then remediate the resources found to be non-compliant. ##ExistingNonCompliant - Remediate resources that are already known to be non-compliant. }) print("Create remediation:\n{}".format(remediationRG)) remediationsubscription = policyinsights_client.remediations.create_or_update_at_subscription( "mymgremediationatsubsc" , { "policy_assignment_id": "/providers/microsoft.management/managementgroups/12333-dfdf-4e7b-a446-48ebcd07d17c/providers/microsoft.authorization/policyassignments/logagents", "resource_discovery_mode" : 'ReEvaluateCompliance' ## ReEvaluateCompliance - Re-evaluate the compliance state of resources and then remediate the resources found to be non-compliant. ##ExistingNonCompliant - Remediate resources that are already known to be non-compliant. ## "policy_definition_reference_id" : "4037797753452518688" }) print("Create remediation:\n{}".format(remediationsubscription)) Note - resource_discovery_mode is only for the policy assignments at the subscription and the resource group scope.AKS Security Dashboard
In today’s digital landscape, the speed of development and security must go hand in hand. Applications are being developed and deployed faster than ever before. Containerized application developers and platform teams enjoy the flexibility and scale that Kubernetes has brought to the software development world. Open-source code and tools have transformed the industry - but with speed comes increased risk and a growing attack surface. However, in vast parts of the software industry, developers and platform engineering teams find it challenging to prioritize security. They are required to deliver features quickly and security practices can sometimes be seen as obstacles that slow down the development process. Lack of knowledge or awareness of the latest security threats and best practices make it challenging to build secure applications. The new Azure Kubernetes Service (AKS) security dashboard aims to alleviate these pains by providing comprehensive visibility and automated remediation capabilities for security issues, empowering platform engineering teams to secure their Kubernetes environment more effectively and easily. Consolidating security and operational data in one place directly within the AKS portal allows engineers to benefit from a unified view of their Kubernetes environment. Enabling more efficient detection, and remediation of security issues, with minimal disruption to their workflows. Eventually reducing the risk of oversight security issues and improving remediation cycles. To leverage the AKS security dashboard, navigate to the Microsoft Defender for Cloud section in the AKS Azure portal. If your cluster is already onboarded to Defender for Containers or Defender CSPM, security recommendations will appear on the dashboard. If not, it may take up to 24 hours after onboarding before Defender for Cloud scans your cluster and delivers insights. Security issues identified in the cluster, surfaced in the dashboard are prioritized to risk. Risk level is dynamically calculated by an automatic attack path engine operating behind the scenes. This engine assesses the exploitability of security issues by considering multiple factors, such as cluster RBAC (Role Based Access Control), known exploitability in the wild, internet exposure, and more. Learn more about how Defender for Cloud calculates risk. Security issues surfaced in the dashboard are divided into different tabs: Runtime environment vulnerability assessment: The dynamic and complex nature of Kubernetes environments means that vulnerabilities can arise from multiple sources, with different ownership for the fix. For vulnerabilities originating from the containerized application code, Defender for Cloud will point out every vulnerable container running in the cluster. For each vulnerable container Defender for cloud will surface remediation guidelines that include the list of vulnerable software packages and specify the version that contains the fix. The scanning of container images powered by Microsoft Defender Vulnerability Management (MDVM) includes scanning of both OS packages and language specific packages see the full list of the supported OS and their versions. For vulnerabilities originating from the AKS infrastructure, Defender for cloud will include a list of all identified CVEs (common vulnerabilities and exposures) and recommend next steps for remediation. Remediation may include upgrading the Node pool image version or the AKS version itself. Since new vulnerabilities are discovered daily, even if a scanning tool is deployed as part of the CI/CD process, runtime scan can’t be overlooked. Defender for cloud makes sure Kubernetes workloads are scanned daily compared to an up-to-date vulnerability list. Security misconfigurations: Security misconfigurations are also highlighted in the AKS security dashboard, empowering developers and platform teams to execute fixes that can significantly minimize the attack surface. In some cases, changing a single line of code in a container's YAML file, without affecting application functionality, can eliminate a significant attack vector. Each security misconfiguration highlighted in the AKS security dashboard includes manual remediation steps, and where applicable, an automated fix button is also available. For containers misconfigurations, a quick link to a built-in Azure policy is included for easily preventing future faulty deployments of that kind. This approach empowers DevOps & platform engineering teams to use the “Secure by Default” method for application development. To conclude - automated remediation and prevention can be a game changer in keeping the cluster secure- a proactive approach that can help prevent security breaches before they can cause damage, ensuring that the cluster remains secure and compliant with industry standards. Ultimately, automated remediation empowers security teams to focus on more strategic tasks, knowing that their Kubernetes environment is continuously monitored and protected. Assigning owners to security issues Since cluster administration and containers security issues remediation is not always the responsibility of a single team or person, it is recommended to use the “assign owner” button in the security dashboard to notify the correct owner about the issue need to be handled. It is also possible to filter the view using the built-in filters and assign multiple issues to the same person quickly. Get Started Today To start leveraging these new features in Microsoft Defender for Cloud, ensure either Defender for Container or Defender CSPM is enabled in your cloud environments. For additional guidance or support, visit our deployment guide for a full subscription coverage, or enable on a single cluster using the dashboard settings section. Learn More If you haven’t already, check out our previous blog post that introduced this journey: New Innovations in Container Security with Unified Visibility and Investigations. This new release continues to build on the foundation outlined in that post. With “Elevate your container posture: from agentless discovery to risk prioritization”, we’ve delivered capabilities that allow you to further strengthen your container security practices, while reducing operational complexities.
Change Azure Policy assignment's system assigned managed identity location
After adding a managed identity to a policy assignment, it is possible to edit only some managed identity related settings of the policy assignment. For instance, if a system assigned managed identity has been selected and created before, its location can’t be changed and a new policy assignment needs to be created with the desired location. This article presents a custom script that creates a new policy assignment from an existing one with a system managed identity created in a specified location.
Microsoft's inconsistent implementation of tagging in Azure
We revamped our Azure resource tagging strategy several years ago and rely on them heavily for #Governance and #FinOps. We not only enforce #tags via #AzurePolicy, we also enforce tag values based on a set of permissible values for each tag. Even with that in place we experience some drift due to exclusions required in the policy definition or exemptions in the policy assignments. I won't get into why this flexibility is needed here, that's a whole separate discussion. Establishing a sound tag hygiene process becomes a vital component of your overall governance and FinOps strategies. One method we employ for tag hygiene is to surface the non-compliant resources in a #PowerBi report using an #AzureResourceGraph (ARG) query. Yes, you can do this in the Compliance section of Azure Policy as well however it lacks ease of use. For example, flipping back and forth between policies, filtering by subscriptions, surfacing other linked metadata is a cumbersome experience in the Azure Policy blade. Now onto my frustrations with how Microsoft has implemented tagging across Azure. 1. Inconsistent application of Tag case-sensitivity across tools - In Azure Policy and in the Azure portal, tag names are case-insensitive whereas tag values are case-sensitive. - In Azure Resource Graph Explorer, both tag names and tag values are case-sensitive. - Why is there inconsistency with case-sensitivity of tag names? 2. Inconsistent Tag validation across Resource Types - When deploying a Storage Account, Azure validates my tag policy before I am able to hit the create button (before it's submitted to ARM) whereas when deploying a resource like a Public IP Address, that validation only occurs after you hit the create button. This likely happens with other resource types as well. By the way, my tagging policy specifies "Indexed" for mode, so in effect it should apply to any and all resources that support tagging in Azure. - Why is does the evaluation of the tag policy differ based on the resource being deployed? 3. Inconsistent Tag UX across Resource Types - When deploying a Storage Account, the tags input is a drop-down list. However, when deploying an Azure Virtual Machine, the tags input is a textbox. Although the latter makes use of predictive text, it's still clearly a different experience. This inconsistency is found across multiple Azure resources. - Why is the tag UX different between resource types? I realize some of this is addressed or is less of a concern when using IaC but that may not be for everyone, or work in all scenarios. It would be great if Microsoft could standardize their implementation of tagging resources uniformly across the entire Azure estate. In my opinion I don't think that's a huge ask.Configure Security Center bundle pricing with Azure Policy
With the new Security Center pricing tier options per resource type, customers have asked us how to configure these at the (Root) Management Group scope so that any new (or existing) subscription will be automatically configured for the Standard pricing tier, allowing Security Center to automatically protect your resources. As you may know, we have recently added Storage accounts protection in Security Center: The most efficient way to achieve that objective is to leverage Azure Policy. With the new Azure Policy aliases for Security Center you can author Azure Policy definitions for each of the 4 resource types. To get you going, I've written 4 Azure Policy definitions which you can add to 1 single initiative to either enforce it on new subscriptions, or to set it on existing subscriptions. The Azure Policy definition (deployIfNotExists) for setting the Standard pricing tier for Storage Accounts looks like this: Add the 4 policy definitions for each bundle pricing tier: Once you have added the 4 Policy definitions, you can add them to 1 single initiative: Finally we assign the Initiative: It will take around 30 minutes for a new assignment to become active: After a while we can see the compliance state for the Initiative: Clicking on one of the definitions shows us why it is not compliant. From here we can "remediate": Remediation is in progress and then done: The 4 Policy definitions (deployIfNotExists) for the bundle resources can be found here.
