Event banner
Event details
- I'm not sure if this was covered, is there a way to better articulate the query speed of each tier other than fast, slower and slowest? Any metrics that can be provided to assess which tier we should select if query time is a requirement?
- TrevorRusher
Community Manager
We are moving into the Q&A portion of the session! The presenters will do their best to answer questions as they come in down below.
- If we want to ingest the same logs into auxiliary logs instead of the current analytics logs, will they be stored in different tables? Are there any other considerations we need to be aware of when making the switch from analytics logs to auxiliary logs?
- Matt_Lowe
Microsoft
Entire tables can be set to be aux tier. This would keep the data within the native table. Custom tables will be needed for aux logs if you are looking to split data between analytics tier and aux tier.
- Is there any plan to support Azure Data Explorer with Sentinel Summary Rules ?
What are the differences between basic logs and auxiliary logs? Are there any use cases that will prefer basic logs over auxiliary logs?
- Is there a complete list of operators that are permitted or prohibited in Auxiliary logs?
- Didier-MSFTSummary rule for Auxiliary logs: KQL Supports all KQL commands on a single table. You can join up to five Analytics tables using the lookup operator https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/lookup-operator
- Are there any best practices for gathering these requirements, and are there calculators available to help determine the best data management options to meet the use case needs?
- Matt_LoweLeveraging use cases (MITRE Tactics, attack scenarios, industry requirements) is a good way to determine which data is needed and where it should land in terms of data tier. From there, it's about determining how often a SOC analyst may need that data. Is it for detections? Is it for monitoring in workbooks? Is it needed for investigation? That will help figure out if the data should be analytics (hot), basic (cool), or aux (cold). Regarding calculations, we recommend just using the public Azure calculator for Microsoft Sentinel/Azure Monitor. Please see: https://azure.microsoft.com/en-us/pricing/calculator/?msockid=2fe98aff6088637619829e7a61a562bd
- GBusheyThere are no calculators although the SOC Optimization will say if a table hasn't been used and could be switched. It will really depend on the amount of data being ingested and what that data will be used for.
- Hi everyone, this auxiliary logs is really useful for many clients, but I still have some questions: 1. - Can we use the CommonSecurityLog table and transition it to auxiliary logs? Since this is the table used to ingest logs from firewall, network devices and so on on the CEF format (It was said this isn't supported now, will it be?) - This is the main way I see my clients ingestion firewall logs to Sentinel 2.- How does the ingestion work? Can we still use the same AMA agent and just have a different DCR? (Seems like CEF is still not available), can you provide an example on how to ingest the logs then? 3. - How exactly does the summary rules work? Once a day, hourly?, if the same summary rule has equal data on the CustomTable does it overwrite it and only save the latest timestamp?
For the question 1 and 2) - You can ingest CEF (Firewall) logs directly to a new custom table (i.e., CommonSecurityLog_CL) using DCR ingestion-time transformation once transformation becomes supported. The custom table (i.e., CommonSecurityLog_CL) has the Auxiliary Tier plan enabled. Then, you create a Summary Rule, which you can schedule to run every couple of hours or day on the (i.e., CommonSecurityLog_CL), then send the summary logs to another custom table with the Analytic tier plan. Check Summary Rule documentation: https://learn.microsoft.com/en-us/azure/sentinel/summary-rules#create-a-summary-rule
- Matt_LoweAt the moment CommonSecurityLog would have to directly go to the aux logs table. This will be addressed in the future to allow ingestion time transformation to split the data between the analytics tier table and the aux table. The recommendation would be to filter out less valuable data from the analytics table to be sent to the aux table via DCR based table splitting. Regarding summary rules, you can determine how often that it runs but the minimum limit is every 20 minutes. We have gotten feedback regarding lowering the time but no further details at this time. For limits, please see: - https://learn.microsoft.com/en-us/azure/azure-monitor/logs/create-custom-table-auxiliary#public-preview-limitations - https://learn.microsoft.com/en-us/azure/azure-monitor/logs/summary-rules?tabs=api#create-or-update-a-summary-rule
- TrevorRusherDon't be shy! Ask your questions down here in the comment section and our off-camera team will answer via text or our on-camera team will get to it after their presentation!
- What is the story of the Basic Logs, in this case? It's not mentioned here. Is the Basic Logs tier going away in the future?
- Didier-MSFTThere are no plans to retire basic logs, but we expect customers to choose auxiliary logs over basic due to cost as we work on the limitations we have today for Auxiliary, such as no transformations
- Thanks, Didier, for the clarification. So, for directing customers to choose auxiliary logs over basic logs due to cost, once the current limitations for auxiliary logs are addressed, we don't see a scenario for using Basic logs anymore, except it's a bit faster in terms of performance.
