Replicated Tables now generally available in Azure SQL Data Warehouse
We are excited to announce that Replicated Tables, a new type of table distribution, are now generally available in Azure SQL Data Warehouse (SQL DW). SQL DW is a fully managed, flexible, and secure cloud data warehouse tuned for running complex queries fast and across petabytes of data. The key to performance for large-scale data warehouses is how data is distributed across the system. When queries join across tables and data is distributed differently, data movement is required to complete the query. The same can be said when transforming data to load, enrich, and apply business rules. With Replicated Tables, the data is available on all compute nodes, hence data movement is eliminated, and queries run faster. In some cases, such as small dimension tables, choosing a Replicated Table versus a Round Robin table, can increase performance because data movement is reduced. As with all optimization techniques, performance gains may vary and should be tested. Read about it in the Azure blog.
How to use values from Data Source as parameter to other Data Source
I have two linked services: 1. A SQL database 2. A Rest API. I'm retrieving data from SQL to my Staging area incrementally. This is no problem. However, for retrieving information from the Rest API I need values from the records I just moved to the staging area. So let's say I get product data from SQL and the Rest API gives me additional info for these products from an external system. So I need to call the Rest API multiple times with as parameter each of the product ID's I've just imported. What is the most efficient way to this in ADF?. So in the way ADF was intended to be used. Thanks for any tips.Azure SQL Data Warehouse now supports automatic creation of statistics
We are pleased to announce that Azure SQL Data Warehouse (Azure SQL DW) now supports automatic creation of column level statistics. Azure SQL DW is a fast, flexible, and secure analytics platform for the enterprise. Modern systems such as Azure SQL DW, rely on cost-based optimizers to generate quality execution plans for user queries. Even though Azure SQL DW implements a cost-based optimizer, the system relies on developers and administrators to create statistics objects manually. When all queries are known in advance, determining what statistics objects need to be created is an achievable task. However, when the system is faced with ad-hoc and random queries which is typical for the data warehousing workloads, system administrators may struggle to predict what statistics need to be created leading to potentially suboptimal query execution plans and longer query response times. One way to mitigate this problem is to create statistics objects on all the table columns in advance. However, that process comes with a penalty as statistics objects need to be maintained during table loading process, causing longer loading times. Read about it in the Azure blog.Explore SaaS analytics with Azure SQL Database, SQL Data Warehouse, Data Factory, and Power BI
Continuing our series of tutorials on SaaS application patterns with SQL Database, we are delighted to announce an additional cross tenant analytics tutorial. This new tutorial shows how to extract and load tenant data into Azure SQL Data Warehouse (SQL DW) using Azure Data Factory (ADF) and then analyze it in Power BI. Read more about it in the Azure blog.Turbocharge cloud analytics with Azure SQL Data Warehouse
Data is transformative. The ability to turn data into breakthrough insights is foundational to remain relevant in an increasingly competitive market. To help our customers deliver fast insights from exponentially growing data, today we are announcing the general availability of the Compute Optimized Gen2 tier of Azure SQL Data Warehouse. With this performance optimized tier, we are bringing the best of Microsoft software and hardware innovations to dramatically accelerate query performance and concurrency for our customers. Read more about it in the Azure blog.
Azure Analysis Services - Windows Authentication
How does Azure Analysis Services support Windows Authentication? Does it translate the account to the matching account in Azure Active Directory? I am trying to use the Windows Authentication option but I keep getting an error: password_required_for_managed_user: Password is required for managed user (Microsoft.IdentityModel.Clients.ActiveDirectory).Super Charge Power BI with Azure Analysis Services
In April we announced the general availability of Azure Analysis Services, which evolved from the proven analytics engine in Microsoft SQL Server Analysis Services. The success of any modern data-driven organization requires that information is available at the fingertips of every business user, not just IT professionals and data scientists, to guide their day-to-day decisions. Self-service BI tools have made huge strides in making data accessible to business users. However, most business users don’t have the expertise or desire to do the heavy lifting that is typically required, including finding the right sources of data, importing the raw data, transforming it into the right shape, and adding business logic and metrics, before they can explore the data to derive insights. With Azure Analysis Services, a BI professional can create a semantic model over the raw data and share it with business users so that all they need to do is connect to the model from any BI tool and immediately explore the data and gain insights. Azure Analysis Services uses a highly optimized in-memory engine to provide responses to user queries at the speed of thought. Read about it on the Azure blog.
Are Azure database DTU and Azure search Index related?
Are Azure SQL database DTU and Azure search Index related? because in my application DTU % is going till 100% however in the code we have written a logic to update searchservice with datasource, Index and Index. Updation is done with first delete then create datasource, Index and Indexer. When we call Search service manually by hitting controller for searchservice it deletes Index but does not create again and it is happening on staging level. I can not debug the code. I have tried the same scenario locally but it works fine on local. Thanks & Regards Sindhuja Azure DeveloperLightning fast query performance with Azure SQL Data Warehouse
Azure SQL Data Warehouse is a fast, flexible and secure analytics platform for enterprises of all sizes. Today we announced significant query performance improvements for Azure SQL Data Warehouse (SQL DW) customers enabled through enhancements in the distributed query execution layer. Analytics workload performance is determined by two major factors, I/O bandwidth to storage and repartitioning speed, also known as shuffle speed. In this previous blog post, we described how SQL DW caches relevant data to take advantage of NVMe based local storage. In this blog post, we will go under the hood of SQL DW, to see how the shuffling speed has improved. Read about it in the Azure blog.Azure sets new performance benchmarks with SQL Data Warehouse
As the amount of data grows exponentially, the pressure to quickly harness it for insights to share across the organization also increases rapidly. As Microsoft continues to evolve our analytics portfolio, we are committed to delivering a data warehouse solution that provides a fast, flexible, and secure analytics platform in the cloud. Today we are excited to announce that Azure SQL Data Warehouse has set new performance benchmarks for cloud data warehousing by delivering at least 2x faster query performance compared to before. The key to this technical innovation is instant data movement, a capability that allows for extremely efficient movement between data warehouse compute nodes. At the heart of every distributed database system is the need to align two or more tables that are partitioned on a different key to produce a final or intermediate result set. Instant data movement in SQL Data Warehouse now accelerates this movement, resulting in faster query performance. You can learn more about how your query performance will improve from this blog. Read more about it in the Azure blog.
