Managed data
What is Data as a Service (DaaS) and why it matters
A clear definition of Data as a Service (DaaS): what it includes, how it differs from building your own infrastructure and why more companies adopt it.
Read articleKey takeaways
- CDC detects and replicates only the changes in source data.
- It reduces load on source systems versus copying everything.
- It enables near real-time synchronisation with low latency.
- It is key to keeping data fresh without penalising operations.
- It is far more scalable than full reloads.
Keeping a copy of the data up to date without saturating production systems is one of the classic integration challenges. Change Data Capture solves it elegantly.
What it is
Change Data Capture (CDC) is a set of techniques that identify and capture only the changes (inserts, deletes and updates) in a data source, to replicate them to a destination without copying the whole set each time.
Why it matters
- Efficiency: it moves only what changed, not the whole set.
- Low impact: it reduces load on the source system.
- Freshness: it enables near real-time synchronisation.
- Cost: less transfer and processing than a full reload.
Full reload
Copy everythingSlow, costly
CDC
Only changesLow impact
Result
Fresh dataNo saturation
Versus a full reload
The alternative to CDC is periodically reloading the whole dataset, which is slow, costly and penalises the source, especially with large volumes. CDC avoids that by capturing only the change delta, making it far more scalable for frequently changing data.
Where it fits
CDC is especially useful for feeding data lakes, warehouses and analytical processes with fresh operational data, and for synchronising systems in near real time. In a managed architecture, incremental CDC-based replication is one of the most efficient ways to keep data available without affecting daily operations.
CDC keeps data fresh by moving only what changed, not the whole dataset.
In summary
Change Data Capture replicates only the changes in a source, not the whole set, keeping data fresh in near real time with minimal load on production systems. It is far more scalable than full reloads and ideal for feeding lakes, warehouses and analytics.
Sources & further reading
Frequently asked questions
Is CDC the same as replication?
CDC is an efficient form of replication: instead of copying the whole set, it captures and replicates only the changes in the source.
Does it penalise source systems?
Far less than a full reload. By moving only changes, it reduces load and impact on production systems.
What is it mainly used for?
To keep data lakes, warehouses and analytical processes fresh, and to synchronise systems in near real time efficiently.
Why is it better than a full reload?
A full reload is slow, costly and penalises the source. CDC captures only the change delta, scaling far better with frequently changing data.
Does CDC enable real-time data?
It enables near real-time synchronisation with low latency, which a periodic full reload cannot match.
Where does CDC fit in an architecture?
As incremental replication feeding lakes, warehouses and analytics with fresh operational data without affecting daily operations.
Turn this data into results
Tell us what you want to achieve. Data Layer connects, processes and delivers the result up and running, with no infrastructure for you to manage.