Lakehouse Architecture and Medallion Design in Fabric

Instructor: Alaa Hady · Language: English or Arabic · Suggested duration: 2 days · Level: Intermediate to Advanced (Level 200–300)

Objectives

• Design a scalable Medallion Architecture (Bronze → Silver → Gold) on OneLake.
• Master Delta Lake table mechanics, schema evolution, time travel, and table maintenance (OPTIMIZE, VACUUM, V-Order).
• Leverage OneLake Shortcuts and Database Mirroring for zero-copy data virtualization.
• Query and model Lakehouse data using the auto-generated SQL Analytics Endpoint.

Outline and contents

1. Lakehouse paradigm: combining data lake flexibility (unstructured files) with data warehouse ACID transactions (managed Delta tables)
2. Structuring the Medallion Architecture: Bronze (raw/immutable), Silver (cleansed/conformed/deduplicated), and Gold (dimensional Star Schema)
3. Multi-Lakehouse vs. Schema-enabled Lakehouse workspace topologies across Dev/Test/Prod
4. Zero-copy data integration: OneLake Shortcuts (Internal OneLake, ADLS Gen2, AWS S3, GCS, Dataverse) and Near-Real-Time Database Mirroring (Snowflake, Azure SQL, Cosmos DB)
5. Inside Delta Lake on Fabric: transaction logs (_delta_log), Parquet files, time travel, and schema evolution
6. Performance optimization for Direct Lake & Spark: V-Order compression, Liquid Clustering / Z-Ordering, file compaction (OPTIMIZE), and retention (VACUUM)
7. Using the SQL Analytics Endpoint: creating SQL views, functions, and managing table-level security
8. Governance & Security: OneLake data access roles (RBAC), folder-level permissions, and workspace roles
9. Hands-on Architecture Lab: Building a three-layer Medallion Lakehouse with Shortcuts, Delta optimizations, and Gold dimensional tables

Target audience

Analytics engineers, data engineers, solution architects, and experienced BI developers.

Expected outcomes

Participants can architect, organize, and maintain a high-performance Fabric Lakehouse following Medallion best practices that serves both data science and Direct Lake BI workloads effortlessly.