Managed data
What is Data as a Service (DaaS) and why it matters
A clear definition of Data as a Service (DaaS): what it includes, how it differs from building your own infrastructure and why more companies adopt it.
Read articleKey takeaways
- Batch ingestion captures data at intervals; streaming captures it as it happens.
- Batch is simpler and cheaper; streaming adds immediacy at the cost of complexity.
- The acceptable latency of the use case drives the choice.
- Many architectures combine both modes by need.
- Latency rules the decision.
Before transforming or analysing data, you have to capture it. That first step — ingestion — shapes the freshness of the whole chain. The big choice is between two modes: batch or streaming.
The two modes
Batch ingestion collects data grouped at defined intervals: hourly, nightly, per close. Streaming ingestion captures each event as it occurs, with a latency of seconds or less.
Batch
GroupedBy interval
Streaming
ContinuousPer event
Decide by
Acceptablelatency
Pros and cons
- Batch — for: simple, cheap, enough for most reporting.
- Batch — against: data is as old as the last cycle.
- Streaming — for: near real-time data for immediate decisions.
- Streaming — against: greater technical complexity and operational cost.
How to decide: latency rules
The key question is not "which is better?" but "how old can the data be without losing value?". A monthly financial report gains nothing from streaming; fraud detection or real-time logistics need it.
Combined approaches
In practice, many architectures use a mix: streaming for critical flows that demand immediacy and batch for the bulk of analytical loads. A managed service selects the most efficient mode for each source.
The question is not which is better, but how old the data can be without losing value.
In summary
Batch ingestion groups data by intervals; streaming captures events continuously with low latency. Batch is simpler and cheaper and suits most reporting; streaming is needed when immediacy adds value. Latency rules the choice, and many architectures combine both.
Sources & further reading
Frequently asked questions
Is streaming always better because it is faster?
No. It only adds value when the decision depends on immediacy. For periodic reporting, batch is simpler and cheaper without losing usefulness.
Can I combine batch and streaming?
Yes, it is common: streaming for critical flows and batch for the rest of analytical loads.
What does streaming cost?
Always-on infrastructure and greater operational and monitoring complexity, which means more cost than an equivalent batch process.
How do I decide between them?
Ask how old the data can be without losing value. If immediacy matters (fraud, logistics), streaming; if not, batch.
What is the downside of batch?
The data is only as fresh as the last cycle, so it is unsuitable when decisions depend on the moment.
Who picks the mode in a managed service?
The provider selects the most efficient mode per source, combining streaming and batch as each case needs.
Turn this data into results
Tell us what you want to achieve. Data Layer connects, processes and delivers the result up and running, with no infrastructure for you to manage.