Fuel AI Innovation with Microsoft Databases
If data is the fuel that powers AI, then AI is only as good as the data behind it. Right now, it’s never been more important to have a strong data analytics and management foundation in place. To help our customers achieve real transformation with AI, we’ve invested heavily across the Microsoft databases portfolio. That includes developing a comprehensive vision for our databases focused on one goal: enabling you to build the next generation of intelligent applications. This week at Microsoft Ignite 2024, you’ll hear about how Azure helps you create fast, secure, and scalable applications powered by the latest advances in AI. Let me share a quick summary of our top database announcements. Delivering the best enterprise databases At the heart of this vision is our commitment to providing the best enterprise databases, with a strong emphasis on reliability, resiliency, and security and the performance and availability needed to support modern, intelligent applications. We’re proud of the ground-to-cloud flexibility we offer for your workloads, and that’s especially true when it comes to our SQL databases—they’re enterprise-ready and built on the same proven, industry-leading SQL engine, so you have a consistent SQL experience whether you’re on-premises or in the cloud. The latest release, SQL Server 2025, is now in private preview and features built-in AI to simplify intelligent application development and RAG patterns. We also announced the general availability of instance pools in Azure SQL Managed Instance. Instance pools let you provision small, cost-effective 2-vCores instances within a pre-provisioned pool, helping you right-size your workloads when migrating or modernizing in the cloud. In our flagship NoSQL database Azure Cosmos DB, dynamic autoscaling is now generally available, providing cost optimization for nonuniform workloads. With dynamic autoscaling, partitions and regions scale independently so you can scale all the data your AI applications are using in the most cost-efficient way. We’ve also invested in our fully managed open-source databases. This includes new features in Azure Database for MySQL such as zonal resiliency by default, which will be generally available in December 2024. This feature helps you ensure seamless server recovery and business continuity in the face of zonal outages. Also coming to public preview in December, you can use Azure Migrate to discover MySQL instances and their attributes within your environment, assess their readiness for migration, and obtain recommendations on suitable compute and storage options. In addition, we’ve made recent investments in Azure Database for PostgreSQL, including the public preview of elastic clusters on Azure Database for PostgreSQL – Flexible Server. This enables horizontal scaling through row-based and schema-based sharding, making it easy to build multitenant apps by offloading shard management and operations—such as tenant isolation, split, or online rebalancing of shards—to the service. Unlocking the power of AI with SaaS-ified databases Built on a SaaS foundation, Fabric Databases are a new class of cloud databases that bring together transactional and analytical workloads to create a truly unified data platform. Now in preview, SQL database is the first database engine to come to Fabric, enabling customers to: Build intelligent AI applications faster with built-in vector search, RAG support, and Azure AI integration. Boost productivity with auto-optimizing and auto-scaling databases. Accelerate innovation with Copilot assistance. Support CI/CD using GitHub integration for source control. Ready to give SQL database in Fabric a try? Starting December 3rd, you can join live sessions with database experts and the Microsoft product team and see just how easy it is to get started. View the schedule and register for the series here. You can also register today to join us from March 29 to April 3, 2025, at the Microsoft Fabric Community Conference in Las Vegas, Nevada to learn more. Rounding out our Fabric Databases news, we’re also pleased to announce the public preview of Fabric integration with Microsoft Purview Information Protection, extending the benefits of central, policy-based governance to Fabric items, including the new SQL database in Fabric. Meeting the needs of modern AI developers Beyond a SaaS-ified experience, we also want to provide the best databases for AI developers. We have a lot of exciting developments to share that demonstrate our commitment to building an ecosystem that’s integrated with Azure AI services and tool support to make building AI applications even easier. Let’s start with a recent innovation in Azure SQL Database and now in SQL database in Fabric – native vector support. We’re excited to announce the public preview of a vector data type that gives developers the ability to handle vector data, which is foundational when it comes to building scalable AI-enabled applications. This announcement also includes essential new vector functions, like VECTOR_DISTANCE, VECTOR_NORM and VECTOR_NORMALIZE to support advanced operations, particularly when the embedding model does not return normalized vectors. And, we’re working hard to make vector indexing on Azure SQL even faster with DiskANN, coming in the future. DiskANN is one of the fastest vector indexing algorithms on the market, and its performance and reliability characteristics are a great fit for our customers’ demanding requirements. While Azure Cosmos DB is already fueling some of the most powerful AI applications on the market, we’re committed to making it even better. We're making a number of announcements to boost performance even further, to include the general availability of DiskANN vector index in Azure Cosmos DB for NoSQL. We’re also announcing two new search features available in public preview: full-text search, which enables efficient text searches and text-based ranking with BM25, and hybrid search, which combines the benefits of semantic vector search, with the text-based relevance of BM25. These new capabilities help retrieve the most accurate data from the database to power generative AI applications. Azure Database for PostgreSQL is optimized for AI developers, and we continue to expand its capabilities with the addition of DiskANN, now available in preview. The new Semantic Ranker Solution Accelerator, now generally available, provides automated deployment scripts that can be used to provision a semantic ranker model as an Azure Machine Learning inference endpoint. Finally, we introduced graph processing capabilities within Azure Database for PostgreSQL with the Apache AGE graph extension and enhanced the accuracy of your generative AI applications with a solution accelerator that integrates GraphRAG with PostgreSQL graph query capabilities. Finally, automatic parameter tuning is coming soon to Azure Database for PostgreSQL. Using machine learning to optimize workload parameters, your AI applications will exhibit significantly higher performance and scalability. We also have some exciting enhancements for MySQL developers. Azure Database for MySQL now supports the MySQL 9.1 Innovation release, which includes exciting new capabilities, such as JavaScript for stored procedures and vector datatype support, expanding your options for application development and advanced data processing. Integrating your data estate Microsoft Fabric is the foundation of an integrated data estate, bringing together everything from data science, engineering, and warehousing to real-time intelligence and operational databases in one environment. Mirroring provides a modern way of accessing and ingesting data continuously and seamlessly from any database into Microsoft Onelake in Fabric. This capability is now generally available for Azure SQL Database and in preview for Azure SQL Managed Instance. Mirroring is also in private preview for SQL Server 2016-2022, and you can sign-up to participate here. The future of databases is now Microsoft databases are empowering developers with essential tools to create groundbreaking AI applications. Whether you’re a large enterprise or a startup building your first application, Microsoft databases provide the performance, security, and reliability to help you make the most of your investments—and unlock valuable insights that drive business growth. Embracing these technologies today sets the stage for a more innovative, efficient, and secure tomorrow.Zero to Hero: Your Guide to Getting Skilled on Azure Data and AI
Technology is changing fast. With these guides you will ramp-up and level up your skill set to keep the upper hand in this competitive market. Our Zero to Hero guides ensure that you have a solid foundation as you begin to explore Azure.Automating the Publishing of Workspace Artifacts in Synapse CICD
New features have been recently introduced in Synapse Workspace Deployment task V2 to facilitate CICD automation in Synapse. These features will give users the ability to do one touch deployments. Before introducing these features, users had to manually hit the “Publish” button from the Synapse Studio, to persist their changes in Synapse Service (Live Mode) and generate the ARM templates for deployment in the publish branch. This was a showstopper for a fully automated CICD lifecycle. With the introduction of these new features, users will no longer require the manual intervention from the UI, thus allowing a fully automated CICD in Synapse. Adding to this, these features to validate as well as generate the ARM templates for deployment using any user branch.Deploying Synapse SQL Serverless objects across environments using SSDT
The long-awaited feature for all Synapse CICD fans is here! SqlPackage now supports serverless SQL pools in Extract and Publish operations. In this article, I will demonstrate how you can run this utility in a DevOps pipeline to replicate your SQL serverless objects across different environments.Azure Data Factory CI-CD using YAML template
Purpose of this Blog 1. Defined the end to end Git integration and Deployment flow using YAML template from a project implementation perspective 2. This can be easily implemented as you just need include this YAML template and do few changes as this has been successfully implemented for Deployments in some projects.Save money and increase performance with intelligent cache for Apache Spark in Azure Synapse
Data professionals can now save money and increase the overall performance of repeat queries in their Apache Spark in Azure Synapse workloads using the new intelligent cache, now in public preview. This feature lowers the total cost of ownership by improving performance up to 65% on subsequent reads of files stored in the available cache for Parquet files and 50% for CSV files.
