Securing financial data with private networking
FinOps toolkit security posture Ensuring the security of FinOps hubs and the FinOps toolkit in general has been a top priority from the very beginning. The FinOps toolkit is designed to help organizations manage and optimize their cloud costs, and it is crucial that these tools operate securely to protect sensitive financial data and maintain trust. This blog post outlines the steps we have taken to secure the FinOps toolkit solutions, from design to ongoing maintenance, with a focus on the newly introduced network aspects of FinOps hubs. Microsoft's Secure Future Initiative Microsoft's Secure Future Initiative (SFI), launched in November 2023, is a comprehensive, multi-year commitment to advance cybersecurity across all Microsoft products and services. The initiative is built on three core principles: Secure by design: Security is prioritized from the initial design phase of any product or service. Secure by default: Security features are enabled and enforced by default, requiring no additional effort from users. Secure operations: Continuous improvement of security controls and monitoring to address current and future threats. These principles guide Microsoft's approach to protecting identities, networks, engineering systems, and more, ensuring a robust defense against cyberattacks. Secure by design From the outset, Microsoft adopted a "secure by design" approach, embedding security into every phase of the development process. This includes: Threat modeling: We conducted thorough threat modeling sessions to identify potential security risks and vulnerabilities. Secure coding practices: Our development team follows secure coding practices, including input validation, output encoding, and proper error handling, to prevent common vulnerabilities. Secure development lifecycle We integrated security into our development lifecycle to ensure that security is continuously addressed throughout the product's evolution: Code reviews: All code changes are available for public review to ensure they meet our security standards. This helps catch potential security issues early in the development process. Static analysis: We use static analysis tools to automatically scan our code for security vulnerabilities. This provides an additional layer of assurance that our code is secure. Secure deployment and configuration Securing the deployment and configuration of the FinOps toolkit is critical to maintaining its security posture: Infrastructure as code (IaC): We use IaC tools like Azure Resource Manager (ARM) templates and Bicep to define and manage our infrastructure. This ensures that our infrastructure is consistently deployed and configured securely. Private networking: We use private networking to ensure that traffic between FinOps hubs components remain private, reducing exposure to the public internet. Managed identities: We use managed identities for services and deployment scripts, providing secure, reliable authentication without needing to store or rotate secrets. Role-based access control (RBAC): We use RBAC to restrict access to FinOps hubs resources and data based on the principle of least privilege. This ensures that users only have access to the resources they need to perform their jobs. Introducing default secure deployments When deploying a new FinOps hub instance, you’ll see a few new options in the deployment form (or template parameters, if deploying programmatically). You’ll find networking options on the Advanced tab where you can set Access to either Public or Private, depending on your needs. Note: If you’re deploying the 0.7 version of the toolkit the “public” option includes some private networking components. In version 0.8 this behavior has been reverted so the "public" option mirrors the behavior of previous toolkit versions. The behavior of the “private” option remains unaffected. Estimating the cost of private networking Private networking comes at an increased cost. The following estimates are based on current list costs. Service category Service Description Estimated monthly cost Analytics Azure Data Factory Azure Data Factory V2 Type, Data Pipeline Service Type, Azure Integration Runtime: 0 activity run(s) 0 data movement unit(s) 100 pipeline activities 100 pipeline activities – External Azure vNet integration runtime: 0 activity run(s) 100 data movement unit(s) 100 pipeline activities 100 pipeline activities – External Data Flow: 1 x 8 general purpose vCores x 100 hours 0 x 8 memory optimized vCores x 730 hours $444.13 Networking Azure Private Link 5 endpoints x 730 hours 100 GB Outbound data processed 100 GB Inbound data processed $38.50 Comparing network access options Public Private Benefit of private networking Storage Resources are accessible over the open internet. (Still protected by RBAC.) Resource access is restricted to the FinOps hub network, peered networks (e.g., corporate vNet), and trusted Azure services. Private endpoints are created. Financial data can only be accessed when at work or on the corporate VPN. Azure Data Explorer Key vault Keys and secrets are never accessible via to the open internet. Azure Data Factory Uses public compute pool. Managed integration runtime deployed managed private network. Managed private endpoints created for Data Explorer, data lake and key vault. All data processing happens inside the network. Virtual Network Not applicable in v0.8. FinOps hub traffic happens within an isolated vNet. Everything remains private, ideal for regulated environments. How public access works Public access in v0.8 follows the connectivity model of previous FinOps hubs releases. Access is controlled via RBAC and communications encrypted via TLS. Storage is accessible via public IP addresses (firewall set to public). Data Explorer (if deployed) is accessible via public IP addresses (firewall set to public). Key Vault is accessible via public IP addresses (firewall set to public). Azure Data Factory is configured to use the public integration runtime. How private access works Private access is the most secure approach but comes at an increased cost for Azure Data Factory as dedicated compute is deployed when running the ETL pipelines. Public network access is disabled by default. Storage is accessible via private IP address and trusted Azure services - firewall is set to default deny with bypass for services on trusted list. Data Explorer (if deployed) is accessible via private IP address - firewall is set to default deny with no exceptions. Key vault is accessible via private IP address and trusted azure services - firewall is set to default deny with bypass for services on trusted list. Azure Data Factory is configured to use the public integration runtime, which helps reduce costs. A virtual network is deployed to ensure communication between all components during deployment and at runtime remains private. FinOps hub virtual network When private access is selected, your FinOps hub instance will include a virtual network to ensure communication between its various components remain private. The virtual network should be a /26 (64 IP addresses) in size. This is to accommodate the minimum required subnet sizes for Container Services (used during deployments for running scripts) and Data Explorer. The IP range can be set at the time of deployment and defaults to 10.20.30.0/26. If required, you can pre-create the virtual network and subnets (and optionally peer it with your hub network) provided you follow these requirements: The virtual network should be a /26 (64 IP addresses in size). The name should be <HubName>-vNet. The virtual network must be divided into 3 subnets with the service delegations as specified: private-endpoint-subnet (/28) – no service delegations configured - hosts private endpoints for storage and key vault. script-subnet (/28) – delegated to container services for running scripts during deployment. dataExplorer-subnet (/27) – delegated to Azure Data Explorer. Private endpoints and DNS Communication between the various FinOps hub components is encrypted using TLS. For TLS certificate validation to succeed when using private IP addressing reliable DNS name resolution is required. During private deployments DNS zones will be created and bound to the VNet, and the necessary private endpoints and DNS entries for the hub components will be created to guarantee name resolution between them. privatelink.blob.core.windows.net – for Data Explorer and storage used by deployment scripts privatelink.dfs.core.windows.net – for Data Explorer and the data lake hosting the FinOps data and pipeline configuration privatelink.table.core.windows.net – for Data Explorer privatelink.queue.core.windows.net – for Data Explorer privatelink.vaultcore.azure.net – for Azure Key Vault privatelink.<location>.kusto.windows.net – for Data Explorer ⚠️ Altering the DNS configuration of the FinOps hub virtual network is not recommended. FinOps hub components require reliable name resolution for deployments and upgrades to succeed. ETL pipelines in Azure Data Factory also require reliable name resolution between components. Network peering, routing, and name resolution When private access is selected the FinOps hub workload is deployed to an isolated spoke virtual network. Multiple options exist to enable private connectivity to the FinOps hub virtual network including: Peering the FinOps hub network with another Azure vNet. Peering the FinOps hub network with an Azure vWAN hub. Extending the FinOps hub network address space and deploying a VPN gateway. Extending the FinOps hub network address space and deploying a Power BI data gateway. Allowing one’s corporate firewall and VPN IP ranges access over the public internet via the storage and Data Explorer firewalls. To enable private access to FinOps hub data from outside the virtual network (when peering to another virtual network) only the private IP address of Data Explorer and storage need to be resolved to a DNS name. The A records are required. The CNAME records may also be required depending on your DNS solution: Required? Name Description Required <storage account name>.privatelink.dfs.core.windows.net A record for Azure Data Lake Optional <storage account name>.dfs.core.windows.net CNAME to A record Required <data explorer name>.privatelink.<azure location>.kusto.windows.net A record for Azure Data Explorer Optional <data explorer name>.<azure location>.kusto.windows.net CNAME to A record In the above diagram The FinOps hub virtual network is peered to a network hub Azure firewall acts as core the router. DNS entries for storage and Data Explorer have been added to Azure DNS Resolver to ensure reliable name resolution A route table has been attached to the network gateway subnet to ensure traffic from on-premise can route to the peered vNet. This network topology follows the Hub-Spoke network architecture outlined in the Cloud Adoption Framework for Azure and the Azure Architecture Center. In conclusion Implementing private networking for Microsoft FinOps hubs ensures secure and efficient cloud resource access. This enhances security, protects sensitive financial data, and maintains trust, aligning with the principles of the Microsoft Secure Future Initiative for security by design, default, and operations. Private access provides a high level of security but incurs higher costs, especially for Azure Data Factory. The advantages of increased security and decreased exposure to the public internet may justify these costs, making private networking a practical option for organizations using FinOps hubs in regulated environments. To learn more, refer to the FinOps hubs overview.What's new in FinOps toolkit 0.7 – November 2024
In November, the FinOps toolkit 0.7 introduces a few highly anticipated updates: support for large datasets at scale with Azure Data Explorer, EA and MCA cost savings, private endpoints, and infrastructure encryption in FinOps hubs; more accurate effective cost in Power BI; and more!Managing Azure OpenAI costs
As AI adoption accelerates across industries, organizations are increasingly integrating these technologies into their core operations. With the growing reliance on AI, it has become essential for our customers to manage their AI spend. This blog post covers how you can leverage Cost Management to analyze, monitor, and optimize your Azure OpenAI costs. Please note that the tools mentioned below are also applicable for other Azure services.
