You probably don't start questioning auto-scaling or shared infrastructure because of anything you see at baseline performance. The dashboards look healthy, and the load tests pass.
The questions start when bid density climbs, whether that's during a campaign peak, a sustained load test, or a Monday morning win rate review that starts with something unexpectedly off.
That assumption holds until the first major campaign push. Going into it, the configuration looks right: auto-scaling is tuned, warm pools are pre-initialized, and the architecture is prepared to handle the volume.
What you don't see in the pre-event review is the provisioning gap. Hyperscaler auto-scaling default warm-up is around 300 seconds. With warm pools pre-initialized, that can come down to around 30 seconds under favorable conditions.
Auction timeouts run at 100 milliseconds or less. The gap between a scaling trigger and a new instance taking live auction traffic is measured in minutes. Thousands of auctions close during that window.
In the meantime, your existing nodes are carrying the full auction load at higher utilization. Every bid being processed on those nodes either makes the exchange timeout window or misses it based on what those nodes can deliver.
Auto-scaling solves a capacity problem, but RTB is often a latency problem. New capacity coming online in 30 to 300 seconds doesn't change the auctions that already happened during the provisioning window.
Standard monitoring captures what your bidding platform did, not what the exchange did with the response.
When a bid arrives after the auction window closes, the exchange drops it and moves on. That shows up as a missed auction on the exchange side, not as a latency error in your logs. Your infrastructure dashboard still shows average response time within acceptable ranges.
The problem is discovered unexpectedly: tracking p99 bid response latency against win rate by exchange over time isnât a comparison most monitoring configurations are designed to surface. By the time someone pulls it, win rates have already dropped.
Your baseline load test looks healthy. At 50,000 bids per second, p99 response latency sits at 6 milliseconds.
Then the campaign launches. The platform runs at 90,000 bids per second during a major campaign event, and the physical hardware is now more loaded than it was during the test.
Average response latency stays within spec, but p99 starts drifting on the bids that happen to hit moments of resource contention.
Nothing in the dashboards suggests a problem at first. Then p99 starts moving, and a little later, so do win rates.
When latency starts climbing, the instinct is usually to scale out: more instances, more capacity, and the problem resolves.
More instances increase total throughput capacity, but they don't change what's happening on the nodes already serving live traffic. The bids on those nodes are meeting the exchange timeout window or missing it based on what those nodes can deliver right now, and new capacity coming online in 30 to 300 seconds doesn't change those outcomes.
You can have enough capacity for the traffic and still lose bids because of p99 latency on the nodes already handling live auctions.
The tuning path is real and worth running: profiling the bid pipeline, optimizing cache hit rates, adjusting network settings, and tuning container resource allocation. Each change moves the numbers, and some improvements are significant.
Eventually, though, the gains get smaller.
When p99 variance tracks with time-of-day patterns that don't match the platform's own traffic, or when the tail widens during periods your logs show as quiet, the source is likely shared resource behavior at the physical host level, CPU and memory contention from co-tenant workloads that application-layer tuning can't reach.
At some point, you stop looking for another tuning win and start wondering whether the limitation is in the application or the environment it's running on.
Better to review these assumptions with a spreadsheet open than with a campaign live and win rates dropping.
Bradley Lewington , AdTech Infrastructure Account Executive
Bradley Lewington is an AdTech Infrastructure Account Executive at servers.com by Nexcess, where he works with performance-sensitive businesses to evaluate infrastructure strategies that support both technical and commercial goals.