More than 20% of the web sits behind Cloudflare's network, and the company's Data team owns the pipeline that moves that data from tens of thousands of globally distributed servers to central data centers. That pipeline feeds Cloudflare Logs, the SQL API, observability tooling and internal analytics. In May, delivery reached roughly two billion events per second, combining customer traffic with data from internal teams. The recurring problem is exponential traffic growth against linear hardware growth, so the team has spent five years offsetting volume increases with engineering rather than procurement — the equivalent of deferring two and a half years of hardware expansion.
The concrete changes described in the post:
- Breaking up a monolithic data schema in 2022, which let the pipeline treat different data shapes independently instead of forcing one layout on everything.
- Adjusting compression in 2024, trading CPU cycles for smaller payloads on the wire and on disk.
- Rearchitecting pipeline replication in spring 2026, changing how data is copied between components to reduce redundant movement.
The through-line for developers is that these are pipeline-internal optimisations, not new customer-facing APIs or configuration flags. Nothing in the post announces a new endpoint, SDK version or pricing change; the payoff is that existing products such as Cloudflare Logs and the SQL API keep their delivery characteristics while the underlying fleet avoids proportional growth. That matters because capacity planning is usually the ceiling on log retention windows, query latency and observability granularity. When hardware allocation grows linearly but event volume does not, teams can extend what they store and query without the platform passing on a cost step change. Anyone designing high-volume ingestion — schema layout, compression trade-offs, replication topology — can read this as a case study in buying time through architecture instead of hardware.
Source: https://blog.cloudflare.com/rewinding-the-exponential/