<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=1705902170274878&amp;ev=PageView&amp;noscript=1">
Skip to content
Talk to Us
  • There are no suggestions because the search field is empty.

Data Analytics Pipeline

Feeding a cloud analytics platform means moving large, sustained data volumes reliably. A private connection over IZO™+ Multi Cloud Connect gives the pipeline predictable throughput and keeps egress on results private and cheaper than internet transfer.

What this is

Analytics pipelines pull data from on-premises systems – transactional databases, logs, sensor and telemetry streams – into a cloud data platform where it is stored, processed and queried. The connection between the two is where pipelines quietly succeed or fail: an under-sized or jittery path turns a nightly load into a load that never finishes.

How it works

Data moves in two shapes, and the pipeline usually has both. Bulk loads are large, scheduled transfers – a historical backfill, a nightly batch – that need raw throughput for a bounded window. Streaming ingest is a continuous trickle of events that needs low, steady latency rather than peak bandwidth. A IZO™+ Multi Cloud Connect Direct connection carries both privately; per-connection bandwidth shaping keeps the bulk load from starving the stream.

Throughput on a single flow is bounded by latency and packet loss, not just link speed, so two design choices matter. First, use jumbo frames where the path supports them (below 9000 bytes, and noting that some cloud paths cap the effective size – for example Azure ExpressRoute presents an effective 1500-byte WAN-side MTU); larger frames mean fewer packets and better throughput on bulk transfers. Second, run parallel streams for bulk loads so a single TCP flow’s ceiling does not cap the whole transfer.

Egress is the cost lever. Analytics generate results – extracts, model outputs, dashboards’ backing data – that flow back on-premises or onward to another cloud. A private connection lowers the per-gigabyte cost of that egress compared with the public internet, which is why egress-heavy analytics is a common reason to move onto IZO™+ Multi Cloud Connect in the first place.

Example

A logistics operator collects vehicle and warehouse telemetry across its network and wants to analyse it in Google Cloud. Streaming ingest carries live telemetry continuously over a IZO™+ Multi Cloud Connect Direct connection, while a nightly bulk load moves the day’s consolidated records using parallel streams and jumbo frames where supported. Query results and daily extracts return over the same private path, keeping egress cost predictable.

MCC Visual Reference Architecture & Use Cases-13.6 Data Analytics Pipeline.drawio

Bandwidth planning

Size for the bulk window, not the daily average. If a nightly load must move a fixed volume inside a few hours, work backwards from volume and window to the required sustained throughput, then add the streaming baseline on top. Validate with a real transfer before committing – achievable throughput on a long-distance path is often below the link rate until frame size and parallelism are tuned.

IZO™+ Multi Cloud Connect components in this architecture

An analytics pipeline is bandwidth-led; it often fans a single on-ramp out to several cloud landings:

IZO™+ Multi Cloud Connect Direct (private MPLS underlay):

  • Fabric Port – the on-ramp where your network meets the service (Hosted or Dedicated; L3 Private access is typical). Present in every solution.

  • Virtual Cloud Connection (often several) – high-bandwidth landings, one per analytics cloud; multi-cloud fan-out from one Fabric Port is common. 

IZO™+ Multi Cloud Connect Flex (internet underlay with an in-path VNF):

  • Fabric Port – the on-ramp where your network meets the service (Hosted or Dedicated; L3 Private access is typical). Present in every solution.

  • Edge Connect – the short leg that carries traffic from the Fabric Port to the VNF

  • VNF – based on your needs as an in-path function. where an in-path function (for example inspection) is required on the pipeline.

  • Virtual Cloud Connection(s)

From the IZO™+ Multi Cloud Connect side, this architecture uses a Fabric Port + one or more Virtual Cloud Connections; the Tata Communications-billed parts run to the Virtual Cloud Connections, while the analytics cloud’s port/attachment and egress sit on your cloud bill.

Considerations

  • Size for the bulk window, not the average - Work backwards from volume and window to the sustained throughput, then add the streaming baseline on top.

  • Throughput tuning - Use jumbo frames where the path supports them (below 9000 bytes) and parallel streams for bulk loads; note Azure’s effective 1500-byte MTU and Oracle’s 9000-byte direct-path jumbo.

  • Egress is the cost lever - A private Virtual Cloud Connection lowers per-gigabyte egress on results versus the public internet.

  • Multi-cloud fan-out - One Fabric Port can carry several Virtual Cloud Connections – size the port for the aggregate of all cloud legs.

  • Validate before committing - Achievable long-distance throughput is often below the link rate until frame size and parallelism are tuned.

What’s on the cloud side

The analytics platform and its cloud connection are created in the cloud provider’s console. This page covers how the private ingest and egress paths are delivered on the Tata Communications side; for the cloud-side connection request and routing, see the relevant per-cloud section.

Related pages