Case Study

Supabase: to a billion (requests) and beyond

Supporting 200x growth on half the infrastructure with Platformatic Watt.

Supabase

With Great Scale Comes Great Responsibility

If you're a developer, you're likely already familiar with Supabase and what they do: they're the team behind the insanely popular and scalable Postgres-based “everything you need for your backend” platform that lets teams “build in a weekend and scale to millions,” all without needing to set up their own supporting database infrastructure.

In 2021, Supabase introduced object storage as “Supabase Storage,” which has since evolved to support S3 compatibility, Row-Level Security policies, CDN-based asset caching, and full Postgres metadata.

Like Supabase as a whole, Storage has seen explosive growth over the last year, with traffic surging as much as 200x due to an influx of AI-native builders adopting the platform. Across all their regions, the service now handles more than 400 million uploads a day, with total requests topping a billion.

Like many teams building infrastructure for the next generation of AI-enabled builders, Fabrizio Fenoglio (Team Lead for Supabase Storage) and his team realized they would need to rethink the architecture behind their service.

Supabase Storage Last Year Growth200xTraffic surging due to an influx of AI-native builders adopting the platform.
Daily Uploads400MHandled by the service across all their regions.

You can't scale what you can't see

The original Storage architecture used Node.js for its API, chosen for the runtime's capacity to scale I/O and data streaming via its unique Event Loop mechanism. However, at 200x the traffic you originally architected for, you'll inevitably begin to see the common signs of stress for a Node.js service: Event Loop instability, climbing latency, runaway garbage collection, and a throughput wall you can't get past, no matter how much infrastructure you throw at it.

Fabrizio's team could see that something was wrong. What they couldn't see was why:

We had metrics — Grafana metrics — and we could see sudden spikes and instability in the runtime. Response time was growing exponentially when these issues happened. The problem was, we didn't know exactly what was going on under the hood. We could see the instability, but we didn't know why.

Fabrizio FenoglioFabrizio Fenoglio

Given the scale their service runs at, reproducing these errors outside of production was effectively impossible — simulating the failure would mean simulating millions of tenants. If they wanted to find the bottleneck, they would have to profile in production.

Now, performance profiling in production is notoriously difficult, because generating the profile itself typically comes with a significant performance hit to your throughput (think on the order of 15%), which isn't something you can tolerate if your service is already experiencing stability issues.

This is the old “Quis custodiet ipsos custodes?” (“who watches the watchmen”) dilemma: you can't safely monitor or profile from within the same Event Loop.

Watt solves this via its management plane and platformatic/flame, which together allow profiles to be collected safely from the worker threads without disturbing their Event Loop.

This means Fabrizio and his team could finally diagnose these performance issues in production:

The only way to figure it out was in production, with Platformatic.

The first step was adding Watt to the Node.js stack, which let us do very powerful profiling and snapshotting on live production traffic.

Fabrizio FenoglioFabrizio Fenoglio

The flamegraph revealed the hotspots in the call stack, one of them being an outdated ORM that spun up a connection pool for every customer database and tore it down when idle — churn that, across millions of tenants, carried a heavy memory footprint and drove the garbage collector into overdrive.

Architecting for the AI Era with Platformatic and Watt

Fabrizio and his team at Supabase worked with Platformatic on a new, Watt-based architecture — one designed to get far more out of every ECS task they were already paying for.

One of the core ideas behind Watt — the same one that made production profiling possible — is running your services as worker threads, each with its own dedicated Event Loop.

Fabrizio and the Storage team used this to increase their compute density and throughput. They consolidated their single-CPU ECS tasks into fewer containers packed with more compute, and used Watt to run multiple copies of their applications as worker threads inside each ECS task.

Watt's built-in load balancing, based on SO_REUSEPORT, enables each worker thread to efficiently consume the allocated compute, while the management plane watches Event Loop Utilization on each thread, hot-swapping any worker that starts to degrade.

The outcomes thus far have more than validated this approach:

Our metrics are much flatter now — especially the garbage-collection runtime.

A single container, at peak on heavy operations, was handling about 50 MB/s per core and having a very hard time keeping up.

Now we're running 300 MB/s per second per container with 2 cores — so we've halved the number of containers we run, and the system is much more stable.

Fabrizio FenoglioFabrizio Fenoglio
Before Watt50 MB/sA single container, at peak
With Watt300 MB/sPer container with 2 cores
Throughput Improvement4x

This 4x improvement in throughput was a huge win for the folks at Supabase managing (and paying for) the underlying AWS infrastructure, and an even bigger one for the builders who rely on Storage, as Fabrizio explains:

Customers won't experience spikes of latency anymore when heavy or bursty traffic hits our service — their latency is very stable overall, and we can scale our instances up very quickly.

Performance is much improved too: they can expect to download and upload files much, much faster now.

Fabrizio FenoglioFabrizio Fenoglio

The Road Ahead

With the Storage Service delivering consistent performance, even during peak traffic, Fabrizio and the team can focus on using Watt to further optimize their system.

The next step is to further leverage this multi-worker capability to share the tenant database connection pool across multiple threads.

This is the kind of separation of concerns you'd normally split into microservices, except that with Watt, you have the option to simplify your architecture by running these services as threads in one machine instead of separate containers.

Next, several improvements are already mapped out from the architecture sessions: extracting connection pooling into its own dedicated system, allowing the service to maintain a fixed set of connections and multiplex queries across tenants to avoid the runtime cost of constantly reopening connections.

That, coupled with a new S3 upload path that trims internal buffering and cuts garbage collection even further, could push per-container throughput significantly higher still.

We're not done yet. Once we complete the last three or four improvements, we think we can get our Node.js system to 500–600 MB/s per second per container.

Fabrizio FenoglioFabrizio Fenoglio

Incredible DX at incredible scale is central to the mission for Fabrizio and the team at Supabase, and with Watt at the core of their architecture, they are ready for the next billion requests and beyond.

Platformatic

Platformatic Inc.

548 Market St PMB 85033 San Francisco,

California 94104-5401 US

Copyright © | All rights reserved

Follow us on:

Fastify™ & Node.js© are trademarks of OpenJS Foundation