Databricks Blog·· 17 hr agoAI score36
Push-based CDC into Unity Catalog with Zerobus Ingest and Debezium
Push-based CDC into Unity Catalog with Zerobus Ingest and Debezium
AI summary
Debezium Server 3.7 ships an open-source Zerobus Ingest sink, built by Databricks with NVIDIA's contribution of envelope mode, that pushes change events directly into Unity Catalog Delta tables without a Kafka cluster between capture and the lakehouse. Envelope mode lands each full Debezium change event in a fixed nine-column table with source position and an idempotency key, and the sink waits for Zerobus to confirm durability before Debezium advances its offset, giving at-least-once delivery. Landed events can then be shaped into Type-1 current-state tables via AUTO CDC sequenced by source LSN, or Type-2 history tables.
Source: Databricks Blog · databricks.com