Importing AWS Security Hub Findings into Microsoft Sentinel
This blog explores how to ingest AWS Security Hub findings into Microsoft Sentinel using native solutions. Although a GitHub reference for deploying an Azure Function-based solution is included, my experience assisting a customer with its implementation provided valuable insights. Instead of step-by-step instructions, I’ll provide a high-level overview and guidance to navigate potential challenges. Let’s dive in! Ingest AWS Security Hub Events to Azure Sentinel https://github.com/Azure/Azure-Sentinel/blob/master/DataConnectors/AWS-SecurityHubFindings/README.md#ingest-aws-security-hub-events-to-azure-sentinel What is AWS Security Hub? AWS Security Hub is a cloud security posture management (CSPM) service that performs automated, continuous security best practice checks against your AWS resources to help you identify misconfigurations, and aggregates your security alerts (i.e. findings) in a standardized format so that you can more easily enrich, investigate, and remediate them. As of November 2024, we already have an S3 connector that ingests logs from specific AWS services: VPC Flow Logs, Amazon GuardDuty, CloudTrail, and CloudWatch, by pulling them from an S3 bucket. We will rely on this connector to receive findings from AWS Security Hub into the AWSCloudWatch table in Microsoft Sentinel, as these findings are sent to the CloudWatch service. While the data ingested into the AWSCloudWatch table may not be parsed exactly as expected, a KQL transformation rule will help address this — more on that later. The flow would look like this AWS services are configured to send their logs to S3 (Simple Storage Service) storage buckets. The S3 bucket sends notification messages to the SQS (Simple Queue Service) message queue whenever it receives new logs. The Microsoft Sentinel AWS S3 connector polls the SQS queue at regular, frequent intervals. If there is a message in the queue, it will contain the path to the log files. The connector reads the message with the path, then fetches the files from the S3 bucket. To connect to the SQS queue and the S3 bucket, Microsoft Sentinel uses a federated web identity provider (Microsoft Entra ID) for authenticating with AWS through OpenID Connect (OIDC), and assuming an AWS IAM role. The role is configured with a permissions policy giving it access to those resources. Reference https://learn.microsoft.com/en-us/azure/sentinel/connect-aws?tabs=s3#architecture-overview To forward findings from AWS Security Hub to CloudWatch, we will use EventBridge. First, we will create a CloudWatch log group to serve as the destination for these events before setting up the EventBridge rule. Creating a CloudWatch log group Follow the steps here for creating a CloudWatch log groups Working with log groups and log streams — Amazon CloudWatch Logs To be able to add a CloudWatch log group as a target in the EventBridge rule. The log group must start with /aws/events. For reference Configuring an EventBridge rule for Security Hub findings — AWS Security Hub Create a new EventBridge rule with event pattern In my example I am not filtering out any findings but if you like to filter for example based on severity like INFORMATIONAL, LOW you can update the event pattern. Refer here for Configuring an EventBridge rule for Security Hub findings — AWS Security Hub Exporting logs from CloudWatch log group to S3 bucket Logs arriving in the CloudWatch log group are in GZIP format, which is accepted by Microsoft Sentinel. These logs can now be sent over to Sentinel. Exporting log data to Amazon S3 — Amazon CloudWatch Logs We can even automate the process which is defined here. Automate! Export of Cloudwatch Logs to S3 Bucket Using Lambda with Eventbridge Trigger — DEV Community Now we can rely on the instructions of setting up S3 Connector in Sentinel Connect Microsoft Sentinel to Amazon Web Services to ingest AWS service log data | Microsoft Learn We have already configured an AWS service(CloudWatch) to export logs to an S3 bucket so skip the mentioned step. Logs arriving in Microsoft Sentinel will appear in the AWSCloudWatch table but won't be parsed as expected. The finding is stored in the Message column. KQL Transformation We can use this KQL transformation query to parse the logs. AWSCloudWatch | where isnotempty(Message) // Ensure message is not empty | extend CleanMessage = replace_regex(Message, @"^\S+\s", "") // Remove the timestamp at the beginning | extend ParsedMessage = parse_json(CleanMessage) // Parse the cleaned message as JSON | where isnotempty(ParsedMessage) // Filter only rows where parsing was successful | extend FindingsArray = ParsedMessage.detail.findings // Extract 'findings' array | mv-expand FindingsArray // Expand findings array into rows | extend Findings = parse_json(FindingsArray) // Parse each finding as JSON | extend Region = tostring(ParsedMessage.region), // Extract region from the top-level message Severity = tostring(Findings.Severity.Label), // Extract Severity from findings Account = tostring(ParsedMessage.account), // Extract account from the top-level message Product = tostring(Findings.ProductName) // Extract Product from findings | project Title = tostring(Findings.Title), Region, Severity, Account, Product // Extract Title, Region, Severity, Account, and Product Tranformation query for parsing AWS Security Hub findings. Debugging Tips #1 Not able to add a CloudWatch log group as a target in the eventbridge rule. To add a CloudWatch log group as a target, you can either create a new log group or use an existing log group. The log group must start with /aws/events. #2 Cannot search the CloudWatch log group. Ensure that AWS service is selected in the target group. #3 Error enabling and configuring event notifications using the Amazon S3 console When configuring your S3 bucket to send notification messages to your SQS queue as part of the S3 connector setup, you might encounter an issue related to the queue name. In my experience, I received an access policy error even though the queue name was visible. If you face a similar issue, I recommend loosening the access policy attached to the queue temporarily and testing the setup again. Once it works, you can refine the policy to meet your security requirements.
AWS Chime based apps (Slack or 3CX) calls drop-out - Only on Intune enrolled MacOS 15 + MS Defender
Hi Intune_Support_Team , I have recently come across with an Issue. Issue: Call Dropout, Network freeze on AV Calls for Apps / Platforms Description: I have noticed this issue on only MacOS Devices enrolled on Intune; that are later updated to MacOS15 Sequioa using Intune policy Mac Update policy + MS Defender for Endpoint Enrolled, with MS Defender Network Filter added to the list, hangs / freezes AV calls for 2-3 seconds like a network glitch on Slack Huddles. This also happens on 3CX Telephone app in bit different way as 3CX agent's audio is not heard by far-end Customer. Both of these only happens on Device upgraded to MacOS 15 + Defender + Network Filter with just Slack and 3CX. Google Meet, Zoom, Teams works well. NOTE : Compared to a Device which is not on Intune /Defender with MacOS 15 Slack Huddle and 3CX is a Charm. I also tried initially to look into Apple MacOS bugs, didnt find much, then raised a request to Slack Support, In Response I got this Hi there Swapnil, Thanks for contacting Slack support. What is happening here is that users are losing media connectivity to the huddles server, causing them to drop and then be reconnected. This can happen for a number of reasons, but if you've recently updated to macOS 15 Sequoia, there is a macOS networking bug which is highly likely to be the cause in this case (https://support.apple.com/en-au/102281). The issue is as follows: Overall the connection may be completely fine. Suddenly the media connection to the huddles server stops completely (even if the rest of the internet connection is fine). After the huddles server detects a period of no data being sent/received, it forces the client to reconnect to the huddle. This can help for some time but it may eventually repeat again through each huddle. Unfortunately in each case we cannot help explain the exact underlying cause is as it occurs on the end of each users network environment. In your case however, if users are experiencing the issue after upgrading to macOS 15, the aforementioned networking bug is the most likely cause. Normally the causes of these kinds of issues are as follows: Firewall or other network configuration closing websockets media connections. The macOS Sequoia bug causes this specific kind of problem. Overzealous modem/router throttling media connections. ISP throttling media connections. On the another response they also mentioned about something is probably not right with MS Defender Network Filter blocking out traffic for AWS Chime Server. Hi Swapnil, Thanks for your reply. Because there are so many variables we aren't going to be tracking this on our side. One thing I would say is that you should just be sure that there are no third party dependencies in your macOS environment which might be in need of an update. I'll give you a random example: Organisations using the Zscaler client connector would have encountered a variation of this issue (https://help.zscaler.com/client-connector/firewall-posture-check-failure-macos-sequoia). The macOS updates alone would not have addressed it, Zscaler needed to issue an update to their client connector software. Until users were running the Zscaler client with the relevant fix, no amount of system updates would have prevented them from running into the compatibility issue. So all I am saying is that you should be keeping an eye out for updates to both macOS and any relevant 3rd party dependencies - it's possible you will need to take manual action in some way first. The public facing macOS updates tend to be quite vague, so it is probably best to start with MS Defender and any other relevant 3rd party configurations before waiting on a macOS update to ultimately fix the issue. You may also prefer to pre-emptively seek confirmation from their respective support services so you know exactly what your next steps are. I hope this gives you a better idea on how to approach the issue and plan for updates Swapnil, and apologies I couldn't provide more guidance. After reading about this I tried to dig little more and understood, 3CX is also using AWS Chime A/V Servers. My users are stuck and losing their Slack Huddles which is day to day quick AV. Any insightful info on this one will be helpful. Thanks Swapnil email address removed for privacy reasons
Windows 10 90-day Evaluation on Public Cloud
Hi Everyone, I'm wondering if I can, without breaking any licensing, download 90-days Windows 10 Evaluation and deploy it as a VM on a Public Cloud Infrastructure (for example Google Cloud or AWS) ? Only as a Proof of Concept, with no production what-so-ever. VM would be running less than 1 day, after that it would be destroyed. The end-goal would be to exercise such operation + create an step-by-step tutorial of how to deploy OS by bringing your own ISO file.
Strange Traffic - MS Edge to AWS Destination
My firewall logs show where MS Edge has tried to make TCP connections outbound to port 9573. The destination hosts are AWS IPs (EC2s) e.g. 18.209.11[.]79 . I was able to narrow it down to process ID 4852 as seen in snips - that process ID points to the Network Service. I was either on Version 91.0.864.70, or earlier - I'm not sure if my Edge has upgraded since last week. Can anyone say what might have caused this?
Admin Center on AWS w/managed active directory
Hi There, We are running a large set of Win 2019 distributed servers with a managed Active Directory on AWS. I would like to use the Admin center to remotely manage those servers + the AD extension to manage the AWS managed AD. I this supported? at this early investigation point, I am in a stage that the AD extension is not loaded and coming up in the menu. All other basic function looks working. Thx, Nir.AWS or Azure or GCP?
Hi All, I am new to the cloud and have started preparing for Azure Fundamentals certification to begin my Azure cloud journey. I am looking for some clarification on the queries I have with regard to Cloud learning. Please share your thoughts. I chose Azure and not AWS for three main reasons: 1) 90% of Fortune 500 companies run on Azure 2) Suit of products from Microsoft that are being widely used by organizations across the world. So, there would be more demand for IT professionals with Azure knowledge 3) AWS market is too crowded. Too many people with 7x certifications and competition is fierce (per my observation) To be honest, I have zero knowledge about the features that AWS brings to the table. Neither Azure nor GCP. But, I preferred Azure for the aforementioned reasons. Lately, I saw this post on Twitter where Andrew mentions the advantages for the beginners in choosing AWS in his replies. Would be really helpful if you can provide your insights on the below: 1) Being a Linux user, will I have any difficulty in gaining Azure knowledge 2) I understand that Azure provides Azure CLI (bash shell). Despite that, is it required for me to gain knowledge on power shell? 3) How easy for Azure professions to switch to AWS or GCP? Also, as market share of Google Cloud platform showing considerable growth with time, and as there is not much expertise available, can oneself make more valuable by preparing for GCP certifications? The new president of Google cloud, Thomas Kurian in one of his interviews was talking about the efforts being made to get major accounts to GCP and how AI, ML can be used in their advantage to deliver effective solutions. So, keeping that in mind, should I switch to GCP? Thanks and Regards Akshay
2019 Datacenter in AWS as Terminal Server
Hi All, We are running a TS in AWS, its a 2019 Data Centre License, and we have 15 2019 per device CALS for users to access the TS. The licenses are showing fine - and users can connect and log into the server fine - however the CAL's aren't allocating and because of this users are kicked every 60min. The License manager seems to complain that the host server is 2016 when it is absolutely 2019? WE aren't using a broker or gateway or anything, just a single instance. Any ideas?
Wrong classification of administrative events for AWS cloudtrail logs
Hi, I'm trying to understand for what reason the below -raw data presented- event is classified as "Administrative Activity". This is causing millions of internal AWS API calls to be classified as Administrative Activity and triggers alarms. Is the eventName field considered and possible values are grouped based on the risk? When we filter in cloudtrail itself, we apply basic filtering of readOnly = false, then we get all changes by administrative activity. is there any way to filter out based on the readOnly field? "eventType": "AwsApiCall", "eventTime": "2019-03-20T09:10:57.0000000Z", "awsRegion": "eu-central-1", "eventName": "Decrypt", "readOnly": true,
