Migrating from Azure Data and AI Stack to Microsoft Fabric: A practical overview
Reading Time: 2 minutes
As the landscape of data platforms continues to evolve, Microsoft has introduced Microsoft Fabric as a comprehensive solution for data integration, analytics, and AI. As a primer, refer to our blog on MS Fabric capabilities for implementing Data Mesh.
The question of migrating to Fabric may arise for organizations already invested in Microsoft Azure’s Data and AI stack. In this blog, we’ll walk through considerations and essential steps for migrating from Azure’s existing services to Fabric, leveraging the latest capabilities of Fabric to streamline data operations.
As Fabric continues to evolve and add more capabilities, we expect a more complete mapping of capabilities to the native Azure stack to allow complete migration to the SaaS model.
For workloads with scores of 1 or 2, we recommend a wait-and-watch approach as Fabric adds capabilities over the next few quarters.
For workloads with scores of 3, 4, or 5, enterprises can consider undertaking PoCs to test capabilities before evolving a concrete migration plan.
Migration Methodology
For Azure workloads with amenability of >=3 and for workloads on other CSPs, robust planning is key to a successful migration. Sigmoid has helped several clients through migrations of their data, analytics, and AI workloads. We cover the critical elements of a migration plan below.
Step 1: Evaluate Current Workloads
Before starting the migration process, it’s crucial to evaluate your current workloads running on Azure / other CSP services. Determine which services are most critical to your operations and map them to the corresponding Fabric services using the table above.
A plot of the business criticality vs. migration amenability can help prioritize candidate workloads for migration. The workloads towards the top right are the most suitable for migration. A sample of such a prioritization matrix is below. However, this has to be moderated by the upstream and downstream dependencies for each workload. For example, Analytics Reports & Dashboards on Power BI are easy to migrate in isolation but would depend on the migration of the DE Pipelines, Storage, and Semantic data models.
Step 2: Prepare for Data Migration
- Provision MS Fabric Capacity
- Prepare Environments and DevOps – Dev, Test, Production. Azure DevOps
- Detailed analysis of Existing workloads
- Architecture
- Schema
- Security
- Operational Dependencies
- PoC / MVP Plan
- Timelines
- Fabric capacity
- Tools & accelerators
- Test Plan and quality assurance
- Contingency plan
Migration Amenability Assessment
Capability
Data Lake Storage
Native Azure Feature
ADLS Gen2
Fabric Feature
OneLake – on ADLS Gen2
Short term – Data movement from ADLS to Fabric is not necessary. Creating ‘Shortcuts’ can provide access to ADLS Long Term – Phased data migration is required to decommission PaaS and get the full benefits of SaaS model.
Data Warehouses / Data Marts – SQL Workloads
Native Azure Feature
Serverless and Dedicated SQL Pools
Fabric Feature
Synapse Data Warehouse
Migrating from Synapse Analytics Dedicated SQL Pools requires robust planning and migration methodology. For small data marts (GBs), options are available to migrate using Fabric Data Factory Copy Wizard For large data warehouses (TBs), convert schemas to Fabric with Copy Wizard, then export data to Lakehouse using CETAS, then ingest this data into Fabric Warehouse using COPY INTO or Fabric Data Factory activities.
