Join the Fabric Partner Community for this Week's Fabric Engineering Connection calls!
Are you a Microsoft Partner that is interested in data and analytics? Be sure to join us for this week's Fabric Engineering Connection call, now offered at two different times! 🎉 The Fabric Engineering Connection call, taking place Wednesday, March 5, from 8-9 am PST, and the APAC Fabric Engineering Connection call, taking place Thursday, March 6, from 1-2 am UTC/Wednesday, March 5, from 5-6 pm PST, will feature presentations from Matthew Hicks on Iceberg Virtualization in OneLake; Diana Tram Nguyen and Andrew Boe on SMB Paths to Azure Solutions Partner Designations & AMM and Azure Innovate Access; and Sanaullah Sanai and Rita ABDELLATIFI on Sustainability Data Solutions in Fabric. This is your opportunity to learn more, ask questions, and provide feedback. To join the call, you must be a member of the Fabric Partner Community Teams channel. To join, complete the participation form at https://aka.ms/JoinFabricPartnerCommunity. We can't wait to see you tomorrow/Thursday!Join the Fabric Partner Community for this Week's Fabric Engineering Connection calls!
Are you a Microsoft partner that is interested in data and analytics? Be sure to join us for this week's Fabric Engineering Connection call, now offered at two different times! 🎉 The Fabric Engineering Connection call, taking place Wednesday, February 26, from 8-9 am PST, and the APAC Fabric Engineering Connection call, taking place Thursday, February 27, from 1-2 am UTC/Wednesday, February 26, from 5-6 pm PST, will feature presentations from Salil Kanade and Venkata Satya Pradeep Srikakolapu on Data Warehouse Developer Experiences and Ted Vilutis on Spark RLS/CLS. This is your opportunity to learn more, ask questions, and provide feedback. To join the call, you must be a member of the Fabric Partner Community Teams channel. To join, complete the participation form at https://aka.ms/JoinFabricPartnerCommunity. We can't wait to see you Wednesday/Thursday!Join us at the Microsoft Fabric Community Conference in Las Vegas, NV from March 30 - April 2, 2025!
FabCon offers you the chance to learn the latest from Microsoft when it comes to Fabric product features and allows you and your teams to get hands-on with the product through workshops on March 30 and April 3. Throughout FabCon, there will be opportunities for you to connect with Microsoft executives, meet with customers, and connect with peers. On March 30, Microsoft will be hosting the FabCon Partner Pre-Day. This is a special day dedicated to our Microsoft partners and is free for registered FabCon attendees. Our Partner Pre-Day will feature keynotes by Wangui McKelvey and Ashley Asdourian and afternoon breakouts for both systems integrator and software company attendees. We are also offering a 48-hour group discount window exclusively for customers through our partners from Feb 18-19. For questions about available discount codes that you can send directly to your 5 selected customers, please reach out to the fabricpartnersteam@microsoft.com. Register for FabCon and join the FabCon Partner Pre-Day today!Join the Fabric Partner Community for this Week's Fabric Engineering Connection calls!
Are you a Microsoft Partner that is interested in data and analytics? Be sure to join us for this week's Fabric Engineering Connection call, now offered at two different times! 🎉 The Fabric Engineering Connection call, taking place Wednesday, February 12, from 8-9 am PST, will feature Teddy Bercovitz discussing ISV Workloads in Fabric, followed by Idris Motiwala presenting on SQL DB in Fabric. Idris Motiwala will return with his presentation for the APAC Fabric Engineering Connection call, followed by Sanjay Soni presenting on Dream Demos. The APAC Fabric Engineering Connection is taking place Thursday, February 13, from 1-2 am UTC/Wednesday, February 12, from 5-6 pm PST. This is your opportunity to learn more, ask questions, and provide feedback. To join the call, you must be a member of the Fabric Partner Community Teams channel. To join, complete the participation form at https://lnkd.in/g_PRdfjt. We can't wait to see you Wednesday/Thursday!Join the Fabric Partner Community for this Week's Fabric Engineering Connection calls!
Are you a Microsoft partner that is interested in data and analytics? Be sure to join us for this week's Fabric Engineering Connection call, now offered at two different times! 🎉 The Fabric Engineering Connection call, taking place Wednesday, February 5, from 8-9 am PST, will feature Varun Jain and Dandan Zhang presenting on Workspace Monitoring and Workspace Private Links. Amir Netz will be hosting his very first AMA (Ask Me Anything) session with the APAC Fabric Engineering Connection call, taking place Thursday, January February 6, from 1-2 am UTC/Wednesday, February 5, from 5-6 pm PST. Join the call to personally ask Amir your Fabric related questions! This is your opportunity to learn more, ask questions, and provide feedback. To join the call, you must be a member of the Fabric Partner Community Teams channel. To join, complete the participation form at https://lnkd.in/g_PRdfjt. We can't wait to see you on the call!Join the Fabric Partner Community for this Week's Fabric Engineering Connection calls!
Are you a Microsoft partner that is interested in data and analytics? Be sure to join us for this week's Fabric Engineering Connection call, now offered at two different times! 🎉 The Fabric Engineering Connection call, taking place Wednesday, January 29, from 8-9 am PST, will feature the first AMA (Ask Me Anything) session of 2025 with Amir Netz. Join the call to personally ask Amir your Fabric related questions! The APAC Fabric Engineering Connection call, taking place Thursday, January 30, from 1-2 am UTC/Wednesday, January 29, from 5-6 pm PST, will feature presentations from Varun Jain and Dandan Zhang on Workspace Monitoring and Workspace Private Links. This is your opportunity to learn more, ask questions, and provide feedback. To join the call, you must be a member of the Fabric Partner Community Teams channel. To join, complete the participation form at https://lnkd.in/g_PRdfjt. We can't wait to see you Wednesday!Join the Fabric Partner Community for this Week's Fabric Engineering Connection calls!
Are you a Microsoft Partner that is interested in data and analytics? Be sure to join us for this week's Fabric Engineering Connection call, now offered at two different times! 🎉 Tamer Farag will be joining both calls to discuss the latest and greatest related to partner offerings and updates, including certifications, Fabric Featured Partners, trainings, incentives, and more. The Fabric Engineering Connection call taking place Wednesday, January 15, from 8-9 am PST. The APAC Fabric Engineering Connection call taking place Thursday, January 16, from 1-2 am UTC/Wednesday, January 15, from 5-6 pm PST. This is your opportunity to learn more, ask questions, and provide feedback. To join the call, you must be a member of the Fabric Partner Community Teams channel. To join, complete the participation form at https://aka.ms/JoinFabricPartnerCommunity. We can't wait to see you this week!Join the Fabric Partner Community for this Week's Fabric Engineering Connection calls!
Are you a Microsoft Partner that is interested in data and analytics? Be sure to join us for this week's Fabric Engineering Connection call, now offered at two different times! 🎉 The Fabric Engineering Connection call taking place Wednesday, January 8, from 8-9 am PST will feature presentations from Miguel Llopis on recently shipped and upcoming Data Factory features and Misha Desai on Data Science. The Fabric Engineering Connection call taking place Thursday, January 9, from 1-2 am UTC/Wednesday, January 8, from 5-6 pm PST will feature presentations from Emily Chen and Leo Li on recently shipped and upcoming Data Factory features and Misha Desai on Data Science. This is your opportunity to learn more, ask questions, and provide feedback. To join the call, you must be a member of the Fabric Partner Community Teams channel. To join, complete the participation form at https://aka.ms/JoinFabricPartnerCommunity. We can't wait to see you this week!A Practical Guide to Implement End-to-End Data Warehousing PoC Using Microsoft Fabric-Part1
Sometime back we had the opportunity to help a customer implement a PoC using Microsoft Fabric. In a series of blogs, we will demonstrate how we helped our customer from the SfMC team to implement a Data Warehouse PoC using the ELT (Extract, Load, Transform) approach. For this PoC, we used sample data from SQL Server’s well-known sample databases, giving us a straightforward and accessible way to illustrate the capabilities of Microsoft Fabric in a real-world data warehousing scenario. The following were the customer requirements:​ Build the enterprise grade Data Warehouse solution in Microsoft Fabric​ Ingest data from 59 diverse sources across 130 different countries Source data from ADLS (JSON)​ Source data from Azure and on-prem SQL Server databases​ Other data sources. ​ ​PoC scope:​ ELT approach ingest (OLTP DB & ADLS Gen2) data directly into the Warehouse​ Transformation using stored procedures to load the production schema Solution Summary: Ingest OLTP data from source SQL Server (full load)​ Meta data driven copy activity to load data into staging schema of the Data Warehouse.​ Stored procedures to process the staging data along with data from ADLS shortcuts.​ Load the production Dimension tables.​ Application’s customer data arrives into ADLS storage as JSON files.​ Stored Procedures to process the data and load Customer dimension and Fact data using incremental load.​ Built 6 Dimensions and 1 Fact.​ Build reports.​ Sample overview of items that were implemented: Sample report from the PoC: Prerequisites: To implement this PoC, the following are the prerequisites: you may need to download the following files: You can download them from: Download all the required files to local drive (eg: C:\temp). We assume that you already have a Fabric tenant setup with a Fabric Capacity (you will need a F64 capacity to test the co-pilot feature else a F2 capacity will be sufficient). If Fabric is not enabled, use the link below to enable Fabric for your organization. Step by Step Guide to Enable Microsoft Fabric for Microsoft 365 Developer Account You will also need: A logical SQL Server (how to deploy one, how to add local IP address to the firewall exception) ADLS Gen2 Account (how to deploy one, how to create a container) Once created, Once the account is created, navigate to the “Data Storage” section and create a container name it levelup or a name of choice Open the container “levelup”and create a folder called “JSON_FILES” Install Storage Explorer, configure to connect to your Azure subscription, and navigate to storage account container. Upload five JSON files from the downloaded folder “JSON_FILES” to ADLS Gen2 in the "levelup" container under the "JSON_FILES" folder. Upload the following folders by choosing “Upload Folder” from Azure Storage Explorer. Sales_SalesOrderDetail Sales_SalesOrderHeader The above two folders containing Delta files, will be used for creating shortcuts in the Lakehouse which will be used for building the tables within the Warehouse in the subsequent tasks. After the upload, you should have the folders below inside your Levelup container. Next, create a database on the Azure SQL Server by using the bacpac files downloaded earlier: Connect to Azure SQL Logical Server using SSMS (if you don’t have, you can download here) Right click on Databases and select the option “Import Data-Tier Application”. Follow the screen captures below to complete the database bacpac import. Note: Depending upon DTU/V-core chosen, the Import activities might take up 30 mins. Continue to the next blogA Practical Guide to Implement End-to-End Data Warehousing PoC Using Microsoft Fabric-Part2
In this continuation (part 2) of the previous blog, you will create the required Fabric items within a workspace and create the tables in Lakehouse and Data Warehouse. Task1: Create a Microsoft Fabric Workspace, Data Warehouse and Lakehouse Before you can start building the solution, you'll first need to create a workspace where you'll create the rest of the items. Follow the steps below to create the workspace. Sign in to https://app.fabric.microsoft.com/ Select Workspaces > New Workspace. Fill out the Create a workspace dialog as follows: Name: LevelUpFabric1 (or a name of choice). Name should be unique. Expand the Advanced section. In the License Mode choose Trial or Fabric Capacity. Click Apply. The workspace will be created and opened. In the upper left corner, select New Item> search and click on Warehouse to create Data Warehouse and name it “DW_Levelup”. This will create the Data Warehouse and open it. Click on the workspace icon the left navigation to navigate to the workspace page. In the upper left corner of the Workspace, select New Item > search and click on Lakehouse to create a Lakehouse and name it, “LH_Levelup”. This will create the Lakehouse and open it. Task2: Create table Shortcuts in the Lakehouse from ADLS Gen2 We will now create shortcuts in the Lakehouse pointing to the two delta folders (Sales_SalesOrderHeader and Sales_SalesOrderDetail) which you had uploaded into the ADLS Gen2 store account in the pre-requisite steps. In the Lakehouse explorer, hover over the Tables folder (not Files folder) and click on the ellipsis and click “New Shortcut”. In the New Shortcut dialog box, select ADLS Gen2 under External sources. In the next screen, provide the required details as shown in the below snip. You may refer to your storage account settings for details. It is discouraged to use Account key for authentication but is done so for ease of use. The URL should be similar to: https://xxxxx.dfs.core.windows.net/levelup Ensure the shortcut name “Sales_SalesOrderDetail” and “Sales_SalesOrderHeader” matches the delta folder name. Important: Please use Sales_SalesOrderHeader and Sales_SalesOrderDetail as shortcut names. These names are used in the stored procedures. If you choose to use different shortcut names in step 5, you may need to update the SPs that refer to it, to avoid errors. The shortcuts should show up within the tables section of the Lakehouse like the below. We will use these shortcuts later in the solution. You may have to refresh the tables section for the shortcuts to appear (you may also have to refresh the tables section if the shortcuts appear as under “Unidentified” too. If it continues to be unidentified, there could be some steps above which were not followed correctly.) Task3: Create the required Tables and SPs in the Fabric Data Warehouse Now, go to Fabric workspace “LevelUpFabric1” and Navigate to the Data Warehouse and open a new query window and run below scripts. Note: The above scripts are available in the “FabricLevelUp_Files” folder that was downloaded during the prerequisite steps within “TSQL_Script” folder. Scripts Details 01_create_AllStgDimTbls.sql Creates staging (stg) schema and all the staging related tables. 02_TablesToCopy.sql Creates metadata tables to dynamic full load activity. 03_del_AllRecordsOfStgDimTbls.sql Creates stored procedure to delete all the records of the staging dim tables. 04_crt_Prd_Sch_Tbl_Sp.sql Create production (prd) schema, fact table, and SP for incremental update to fact table. 05_crt_Prd_Dim_Tbls.sql Create stored procedure to create all the production dim tables. 06_upd_prd_dim_Customer.sql Creates SP to update incremental stg.DimCustomer records to prd.DimCustomer tables. So far, we have created workspace, all the required items in the Lakehouse and Data Warehouse which will be used to load the Data Warehouse. Continue to the next blog...
