Those who work in the data world, especially with a modern platform like Microsoft Fabric, are likely familiar with medallion architecture, where data is carefully moved from its raw bronze state to a cleaned and conformed silver layer and finally to a business-ready gold layer.
Each layer serves a purpose, but once data reaches the analytical silver and gold layers, the focus shifts from transformation to performance and efficiency. How do we build tables so they’re fast to query, easy to maintain and make the best use of Fabric’s compute capacity?
For years, we’ve had two common (but imperfect) methods:
- Standard views: These are always “live,” showing the most current data, but every time a user loads a report, Fabric must re-run the entire query. This can slow things down and use more compute resources than necessary.
- Full tables (truncate/reload): These are quick to query but clunky to maintain. Wiping and rebuilding large tables is disruptive, wasteful and inefficient when only a small portion of data changes each time.
What if there were a better way? A method that provides the performance of a table with the simplicity of a view. Enter the materialized lake view, a modern take on a classic concept that brings new efficiency and reliability to the lakehouse.
What’s old is new again: the materialized view
Materialized views have existed for decades. A standard view is just a stored query, while a materialized view stores both the query and its results as a physical table.
Historically, the challenge was keeping those results fresh. When the data underneath changed, the stored results did not automatically update, meaning teams had to manage complex refresh schedules or accept stale data. That extra overhead often made them more trouble than they were worth.
Fabric’s approach changes that story entirely.
Why materialized lake views matter in Fabric
Microsoft Fabric’s materialized lake views (MLVs) take the idea of materialized views and reimagine it for a lakehouse environment, where efficiency and performance matter at scale.
Protiviti recognizes that both the current GA features as well as Microsoft’s roadmap for materialized lake views helps address many of the challenges teams face when moving to more modern lakehouse architectures, allowing teams to build faster, manage cloud compute costs, decrease complexity and incorporate data quality directly into their MLV view definitions.
With MLV adoption exceeding 90% in gold layers for many Protiviti clients, continued advancements are driving even greater gold layer use and enabling simpler, more efficient silver architectures. Some benefits Protiviti and Protiviti clients are seeing with MLVs are enumerated below.
Simplify data pipelines
The traditional truncate-and-reload model is heavy-handed. It often means dropping and recreating large datasets just to capture a few new or updated records.
MLVs automate that process. Define the logic needed and Fabric manages how it’s maintained behind the scenes. There’s no need for custom merge logic or orchestration pipelines to detect what changed. And because MLVs stay available during refreshes, analysts never lose access to the data while it’s being updated.
Make better use of compute resources
Even though Fabric abstracts away direct costs, all work still runs within shared compute capacity. Running the same complex queries repeatedly can slow other workloads or extend refresh times. MLVs reduce that overhead.
When reports use a standard view, each refresh runs the query in full. With MLVs, the heavy query runs once during the refresh and everyone else reads from pre-computed results. Reports load faster, concurrency improves and available compute power goes further.
This makes MLVs especially valuable for the gold layer, where data doesn’t change constantly but is accessed frequently.
The real magic: smart incremental updates
Traditional materialized views required full rebuilds, which could take hours for large datasets. Fabric’s MLVs are becoming far more efficient by supporting incremental updates.
That means when new or changed data arrives, only those specific records are processed, rather than rebuilding the entire dataset from scratch.
For datasets with a billion rows and only a thousand have changed, just those thousand with changes are refreshed.
It’s the difference between rebuilding an entire house every night versus just touching up the paint on the front door. The result is faster refreshes, lower compute usage and more up-to-date data for end users.
Building trust: data quality at refresh time
Performance is critical, but so is data confidence. Fabric now allows data quality checks to run as part of each materialized lake view refresh.
When MLV is defined, it is possible to include rules that validate whether data meets specific expectations, including factors like required values being present, data falling within expected ranges or relationships staying consistent. If something doesn’t pass, Fabric flags it during the refresh process, providing visibility into data that might not meet predefined business standards.
This adds a new layer of trust. Instead of discovering data issues only after they reach reports or dashboards, it is possible to surface and measure quality right where and when data is materialized. It’s a practical way to ensure the data powering gold and silver layers is both fast and reliable.
Where MLVs fit in the medallion architecture
The primary use case: the gold layer
This is where MLVs shine. Gold-layer data is often queried constantly but changes less frequently. Using MLVs here provides:
- Speed: Reports load faster thanks to pre-computed data.
- Efficiency: Transformations run once per refresh instead of every time a user queries.
- Reliability: Data quality checks reinforce confidence in what users see.
An emerging use case: the silver layer
There’s also growing opportunity in the silver layer, particularly for source systems that only provide full daily snapshots of data. MLVs can handle that data more efficiently, comparing new snapshots to existing results and automatically applying incremental refreshes. As Fabric’s capabilities evolve, we expect to see even more creative ways to use MLVs for mid-tier transformations and validations.
What’s next
Materialized lake views in Microsoft Fabric are a foundational building block for a high-performing, reliable data lakehouse. They simplify refresh management, reduce workload demands and now bring data quality into the refresh process itself.
They represent a simple but powerful principle: do the heavy lifting once, keep data trusted and everyone benefits from faster, more consistent results.
To learn more about Protiviti’s Microsoft consulting services, contact us.
