What’s new: Monitor the health of your automation rules and playbooks

Special thanks to BenjiSec for creating the Automation Health workbook and Yechiel_Levin who contributed to this article.

 

These days, when organizations are looking to do more with less resources, automation capabilities are a critical asset for organizations. Microsoft Sentinel automation rules and playbooks allow customers to turn repetitive manual processes to automatic flows. They save time and resources and allow security teams to focus on their main goal: protecting the organization from threats and investigating the important incidents that require their attention.

 

While Microsoft Sentinel automation features rely on the robust and powerful Azure engines, we understand that SOC engineers still want to make sure their automations run as expected. Having the ability to monitor automation rules and playbooks at scale allows teams to automate more scenarios and ensure everything happens as expected behind the scenes.

 

We are pleased to introduce new monitoring capabilities for automation rules and playbooks. These are part of Microsoft Sentinel's health monitoring features that let customers monitor active content in their workspace.

 

Now available:  

• keep track of the health of your automation rules and playbooks by monitoring their execution logs.
• Visualize data using an out-of-the-box workbook.
• Audit who triggered playbooks on demand.
• Correlate Azure Logic Apps and Microsoft Sentinel health logs.
• Set up notifications of health events for relevant stakeholders, who can then take action.

 

Get started

• Turn on health monitoring for Microsoft Sentinel.
  A new category is now available: Automation.
• Recommended: Turn on Logic Apps diagnostics logs for your playbooks.
  This will allow you to correlate the trigger and run results of playbooks, as explained later in this post.

 

Monitor execution logs of automation rules and playbooks

After enabling the automation category under Microsoft Sentinel health settings, the SentinelHealth Log Analytics table will collect new events:

• Automation rule run. Logged whenever an automation rule's conditions are met, causing it to run.
  
• Playbook was triggered. Logged whenever a playbook is triggered on an incident manually from the portal, through the API, or by an automation rule. 

 

 

 

 

SentinelHealth 
| where SentinelResourceType == "Automation rule" or SentinelResourceType == "Playbook"

 

 

Get more details about automation rule run

In addition to general audit details, automation rule run logs include:

• Number of actions the automation rule successfully triggered
• Number of actions configured in this automation rule
• Which incident triggered this rule
• List of playbooks this automation rule triggered successfully and their run Ids (for correlation with Logic Apps diagnostics logs)

 

 

SentinelHealth
 | where OperationName == "Automation rule run"

 

 

 

Audit who triggered playbooks on-demand

You can now see for each on-demand playbook triggering the user details.

 

 

 

SentinelHealth 
| where SentinelResourceType == "Playbook"
| extend triggeredBy = ExtendedProperties.TriggeredByName.UserDisplayName, triggerdByUPN = ExtendedProperties.TriggeredByName.UserPrincipalName
| project-away WorkspaceId, SentinelResourceId, Type, TenantId, ExtendedProperties

 

 

 

Resolve issues when errors occur

Each log indicates success, partial success or failure. When errors occur, the description field may supply further information, and Microsoft Sentinel documentation supplies suggested actions.

 

 

Automation health workbook

The Automation health workbook (created by the great BenjiSec) helps you visualize your health data, as well as the correlation between the two types of logs that we just mentioned.

 

 

Track playbooks triggering and runs results

Microsoft Sentinel's health monitoring table allows you to track the triggering of playbooks, but to monitor what happens inside your playbooks and their results when they're run, you must also turn on Azure Logic Apps diagnostics.

Correlate Azure Logic Apps diagnostic logs to know the result status of playbooks triggered by Microsoft Sentinel.

 

 

 

 

 

SentinelHealth 
| where SentinelResourceType == "Automation rule"
| mv-expand TriggeredPlaybooks = ExtendedProperties.TriggeredPlaybooks
| extend runId = tostring(TriggeredPlaybooks.RunId)
| join (AzureDiagnostics 
    | where OperationName == "Microsoft.Logic/workflows/workflowRunCompleted"
    | project
        resource_runId_s,
        playbookName = resource_workflowName_s,
        playbookRunStatus = status_s)
    on $left.runId == $right.resource_runId_s
| project
    RecordId,
    TimeGenerated,
    AutomationRuleName= SentinelResourceName,
    AutomationRuleStatus = Status,
    Description,
    workflowRunId = runId,
    playbookName,
    playbookRunStatus

 

 

 

Learn more

Find full instructions in Microsoft Sentinel documentation:

• About health monitoring in Microsoft Sentinel
• Enable health monitoring in Microsoft Sentinel
• Monitor the health of your automation rules and playbooks
• SentinelHealth table schema