Integrating API data into Microsoft Security Copilot using custom logs and KQL plugins

Microsoft Security Copilot (Copilot) is a generative Artificial Intelligence (AI) system for cybersecurity use cases. Copilot is not a monolithic system but is an ecosystem running on a platform that allows data requests from multiple sources using a unique plugin mechanism. Currently, it supports API, GPT, and KQL-based plugins, API and KQL-based plugins can be used to pull external data into Security Copilot. In this blog post we will discuss how both methods can be combined and how we can get data only available through the API to Security Copilot, while benefiting from KQL plugin simplicity.

KQL vs. API plugins

KQL-based plugins can gather insights from Microsoft Sentinel workspaces, M365 Defender XDR, and Azure Data Explorer clusters. Such plugins do not require any development skills beyond ability to write KQL queries, do not require any additional authentication mechanism and can be easily extended with new features/queries as needed. API plugins give Copilot the capability to pull data from any external data source if it supports REST API, e.g. allowing Copilot to make Graph API calls.

KQL and API plugins each have their specific use cases. The KQL option is often chosen for its simplicity and due to certain limitations associated with API plugins:

• API plugin request body schemas are limited to a depth of 1, which means they cannot handle deeply nested data structures.
• Output from APIs often needs to be parsed before Security Copilot can ingest it, as Security Copilot, like all other large language models (LLMs) based applications, has limits on how much information it can process at once, known as a "token limit".
• API must be publicly available for Copilot to access it, which means API endpoint must be properly secured and authentication method must be supported by Copilot.

The best of both worlds

A possible solution is to integrate data available only through the API into the Log Analytics workspace, allowing for subsequent querying via KQL. The solution consists of two parts:

1. Logic App to query API data and send it to Log Analytics (Sentinel) workspace.
2. Custom KQL plugin for Security Copilot to query custom tables.

As an example, we will build a solution that allows querying Defender XDR Secure Score historical data, which is currently only available through Graph API.

Create Logic App to store data retrieved via API in Log Analytics workspace

We will start with building a simple Logic App to get API data and send it to Log Analytics. While we use Secure Score data, as an example, the same method can be used for any other data that does not change often and suitable for KQL table storage. The Logic App will do the following:

Figure 1: Logic App to send Secure Score data to Log Analytics

1. Logic App is triggered once a day, in accordance with Secure Score update schedule (once in 24 hours).
2. It gets the latest Secure Score via HTTP call to Graph API:
   • HTTP GET to https://graph.microsoft.com/v1.0/security/secureScores?$top=1:
   • Graph API call is authenticated via Managed Identity:
   • 
   • Managed Identity will require SecurityEvents.Read.All permission to get access to Secure Score data:

     Connect-AzAccount
     $GraphAppId = "00000003-0000-0000-c000-000000000000"
     $NameOfMSI = "LOGIC_APP_NAME”
     $Permission = "SecurityEvents.Read.All"
     
     $GraphServicePrincipal = Get-AzADServicePrincipal -AppId $GraphAppId
     $AppRole = $GraphServicePrincipal.AppRole | Where-Object {
         $_.Value -eq $Permission -and $_.Origin -contains "Application"
     }
     
     New-AzADServicePrincipalAppRoleAssignment `
             -ServicePrincipalDisplayName $NameOfMSI `
             -ResourceDisplayName $GraphServicePrincipal.DisplayName `
             -AppRoleId $AppRole.Id 
     

      

3. Received Secure Score data is sent to Log Analytics workspace using built-in Azure Log Analytics Data Collector connector. For convenience we will split data returned by Secure Score API into two categories to store them in different custom log tables: overall Secure Score values and specific values for each security control.
   • Managed Identity assigned to the Logic App will have to be granted Log Analytics Contributor role on the chosen Log Analytics workspace:

      

4. As a result of running Logic App for the first time, two custom logs will be generated in the selected Log Analytics workspace.
   • SecureScoreControlsXDR_CL log will contain all information about specific controls:

      

   • SecureScoreXDR_CL will contain just one entry per day, but it is handy when it comes to tracking Secure Score changes in the organization.

Create custom KQL Plugin for Security Copilot

Now when we have our data conveniently stored in Log Analytics workspace, we can proceed to creation of custom KQL plugin. Below is an example of such plugin with some basic skills, but thanks to simplicity of KQL plugins, it can easily be extended and adjusted to ones needs:

Descriptor:
  Name: SecureScoreXDRPlugin
  DisplayName: Defender XDR Secure Score plugin
  Description: Skills to query and track Microsoft Defender XDR Secure Score 

SkillGroups:
  - Format: KQL
    Skills:
      - Name: GetSecureScoreXDR
        DisplayName: Get Defender XDR Secure Score for specific date
        Description: Queries Defender XDR Secure Score current status for Apps, Identity, Devices, Data and Total
        ExamplePrompts:
          - 'Get Defender Secure Score for today'
          - 'Get Secure Score for 2022-01-01'
          - 'What is the current Secure Score'
          - 'What was Secure Score 7 days ago'
        Inputs:
          - Name: date
            Description: The date to query the Secure Score for
            Required: true
        Settings:
          Target: Defender
          Template: |-
            let specifieddate = todatetime('{{date}}');
            SecureScoreControlsXDR_CL
            | where TimeGenerated between (startofday(specifieddate) .. endofday(specifieddate))
            | summarize IdentityScore = sumif(score_d, controlCategory_s == "Identity"), AppsScore = sumif(score_d, controlCategory_s == "Apps"), DeviceScore = sumif(score_d, controlCategory_s == "Device"), DataScore = sumif(score_d, controlCategory_s == "Data") by bin(TimeGenerated, 1d)
            | extend TotalScore = (IdentityScore + AppsScore + DeviceScore + DataScore)

      - Name: GetSecureScoreXDRChanges
        DisplayName: Get Defender XDR Secure Score controls changes for the past 7 days
        Description: Queries Defender XDR Secure Score and shows changes during the past 7 days
        ExamplePrompts:
          - 'How did secure score change in the past week'
          - 'What are secure score controls changes'
          - 'Show recent changes across secure score controls'
          - 'Show secure score changes for the past 7 days'
        Inputs:
          - Name: date
            Description: The date to query the Secure Score for
            Required: true
        Settings:
          Target: Defender
          Template: |-
            let specifieddate = todatetime('{{date}}');
            let Controls = SecureScoreControlsXDR_CL
            | project TimeGenerated, RecommendationCategory=controlCategory_s, ControlName=controlName_s, Recommendation=description_s, ImplementationStatus=implementationStatus_s, ControlScore = score_d
            | where TimeGenerated >= specifieddate;
            Controls
            | summarize distinctScoreCount = count_distinct(ControlScore) by ControlName
            | where distinctScoreCount > 1
            | join kind=inner (
                Controls
            ) on ControlName
            | summarize TimeGenerated = max(TimeGenerated) by ControlName
            | join kind=inner (
                Controls) on TimeGenerated, ControlName
            | project TimeGenerated, ControlName, RecommendationCategory, Recommendation, ImplementationStatus, ControlScore

Now we need to save text above to YAML file and add it as custom KQL plugin:

Once plugin is deployed, we can query and track Secure Score data using Security Copilot:

Conclusion

Storing non-log data within a Log Analytics workspace is an established practice. This method has been used to allow security analysts easy access to supplementary data via KQL, facilitating its use in KQL queries for detection enrichment purposes. As illustrated in the scenario above, we can still generate alerts based on this data, such as notifications for declining Secure Scores. Additionally, this approach now enables further AI-powered Security Copilot scenarios.