Data Lakehouse: It's a design choice, not an evolution

Data Lakehouse is a relatively new term, but not a new desire. It’s about flexibility, structure, and long-term scalability, all consolidated into an architecture.

Let’s unpack what this architecture is solving and why it should be seen as a conscious architectural choice, not just a natural progression from traditional data warehouses.

Foundation: Data Lake vs Data Warehouse

Before diving into what a Lakehouse is, we need to revisit the foundations that shaped it, which are Data Lakes and Data Warehouses. These are not just storage solutions but also reflect different philosophies of handling data at scale.

Data Warehouse

A data warehouse is a centralized repository optimized for analytical queries and reporting. It enforces schema-on-write, meaning data must be cleaned and structured before being loaded. This approach ensures consistency, integrity, and performance.

Think of data warehouses as well-structured libraries. Every book (or dataset) is carefully categorized, indexed, and easy to find. This makes them ideal for use cases like:

• Business intelligence dashboards
• Financial reporting
• Forecasting models
• Regulatory audits

Warehouses often support high concurrency, low-latency queries, and strong transactional guarantees (ACID). They are typically backed by robust engines like Snowflake, BigQuery, Redshift, or SQL Server.

However, this level of control comes at a cost—literally and operationally. Warehouses are expensive, especially as data volume and complexity increase. Ingesting semi-structured or rapidly changing data requires heavy ETL pipelines to transform raw inputs into a usable format.

Data Lake

On the other hand, a data lake is a large repository that stores data in its raw form, often in formats like JSON, CSV, or Parquet. It accepts structured, semi-structured, and unstructured data with minimal upfront processing—known as schema-on-read.

Data lakes are often built on top of object storage systems like Amazon S3, Azure Data Lake Storage, or Hadoop Distributed File System (HDFS). They’re cost-effective, scalable, and excellent for storing:

• Application logs
• Clickstream data
• Relational CDC events
• Sensor/IoT data
• Image, audio, and video files

The main advantage here is flexibility. You don’t need to know the exact structure of the data when you ingest it. But this also introduces challenges. Querying raw data can be slow. Data quality may be inconsistent. And without governance, a lake can quickly turn into a swamp.

Comparing the Two

Each approach solves different problems. Warehouses give you structure and performance. Lakes give you flexibility and scale. But until recently, teams had to choose one or maintain both—duplicating data and pipelines.

That’s where the Lakehouse comes in.

Enter: Data Lakehouse

Lakehouse is a term that emerged to bridge the longstanding gap between data lakes and data warehouses. For years, organizations have faced the pain of maintaining two separate systems: one optimized for cheap, scalable storage and another for fast, reliable querying. The Lakehouse architecture was introduced to unify these capabilities into a single platform.

At its core, a Lakehouse brings warehouse-like data management features—such as transactions, schema enforcement, and indexing—directly to a data lake environment. This is made possible by modern table formats like Apache Iceberg, Apache Hudi, and Delta Lake, which sit atop object storage systems and bring transactional semantics to traditionally append-only files.

Key characteristics of Lakehouse include:

• ACID Transactions: Changes to data are atomic, consistent, isolated, and durable. This was previously only guaranteed by warehouse systems.
• Schema Evolution: Ability to evolve data models over time without complete rewrites or manual intervention.
• Data Versioning & Time Travel: Accessing past states of a dataset becomes possible, aiding debugging, auditing, or point-in-time analytics.
• Streaming + Batch Support: Many Lakehouse-compatible formats allow ingestion of both batch and streaming data into the same destination table.
• Open Table Formats: Unlike traditional warehouse vendors with proprietary systems, Lakehouse tables are typically open and compatible with multiple engines.

This convergence of flexibility and structure allows engineers to build pipelines where raw, intermediate, and curated datasets all live in the same unified storage layer, queried by a variety of compute engines like Trino, Spark, or Presto.

But it’s not just about merging capabilities. The Lakehouse introduces new ways of thinking about data systems:

• Instead of exporting cleaned data from a warehouse into a lake for machine learning, data scientists can now run notebooks directly on curated tables in the Lakehouse.
• Instead of duplicating transformation logic for batch and streaming jobs, engineers can write once and serve both.
• Instead of worrying about vendor lock-in with proprietary warehouses, teams can adopt open governance and storage standards.

The Lakehouse architecture isn’t a silver bullet. But it is a bold step toward simplification in a landscape that has long been fragmented.

Pros and Cons of Lakehouse

Let’s be honest: Lakehouse sounds like a silver bullet. But we should treat it as an option, not the de facto evolution.

Pros

• Unified architecture: One place for raw and curated data—removing the dual-maintenance of lake + warehouse.
• Cost efficiency: Object storage is significantly cheaper than warehouse compute-based storage.
• Decoupled compute: Query engines like Trino or Spark can be scaled independently based on workload types.
• Versioning and time travel: Thanks to advanced table formats.

Cons

• Operational complexity: It’s a system composed of many moving parts—catalogs, query engines, storage formats, compaction jobs, schema evolution tooling, and more.
• Query performance: Still lags behind mature warehouse engines like Snowflake or BigQuery, especially under mixed or high-concurrency workloads.
• Tooling and ecosystem maturity: Warehouse tools are often more integrated and polished, especially for enterprise users.

Key Insight

Lakehouse is not a clear upgrade path—it’s a trade-off. For many teams, especially those with complex semi-structured data, or a need to scale cost-effectively, it’s a strong candidate. But if you’re only serving BI use cases and want simplicity, a data warehouse might still be the right call.

Observation

At one company I worked with, the team originally maintained a Redshift warehouse for all analytical needs. As business expanded, new data sources arrived: clickstreams, logs, third-party JSON blobs. Redshift struggled to scale affordably with these formats.

They moved raw data into S3 and tried querying with Athena—but hit performance and governance issues. Eventually, they adopted a Lakehouse stack: Hudi + Trino + Apache Hive metastore. This unlocked low-cost raw storage, incremental ingestion via CDC, and fast queries for curated models.

However, it came with its own challenges: managing clustering jobs, handling schema evolution, and tuning query performance for BI users. In the end, it worked, but not because it was easier, because it was better for their use case.

Conclusion

Data Lakehouse is not the “next step” after Data Warehouse. It’s a design choice, which is an architecture tailored for flexibility, scalability, and multi-modal data.

You don’t adopt Lakehouse because it’s trendy. You adopt it because your data demands it.