Q Thoughts

Drinking From The Fire Hose

by Chris Back

Real-Time-Bidding (RTB) for ad impressions is tremendously powerful. But, to make it work in practice there are plenty of technology challenges that have to be addressed by a demand-side-platform (DSP). In particular, the number of ad opportunities that have to be examined to decide which ones are of interest is something akin to drinking from a fire hose: there is so much going by at any instant how do you get what you want?

Sources of ad inventory, such as Google, AppNexus and AdBrite among others, taken together, can require a DSP platform to process literally billions of transactions per day. For perspective, a billion transactions uniformly distributed throughout the day represent about 12,000 ad opportunities per second. Of course, the ad opportunities aren’t uniformly distributed. We see that the busy hour runs 38% above the mean so a billion ad opportunities really represents about 16,500 transactions per second at peak rate.

Keeping up with this data flow requires DSPs, and OwnerIQ’s MostIQ Ad Platform specifically, to deploy a lot of processing power to both decision on each impression opportunity and keep up with the resulting data exhaust (characteristics of bids won, inventory observed, etc.).

It has very real costs too. Although the cost of each marginal bid request is almost nothing, the cost of processing bid requests rises by the quantum of each computing instance. Each cloud-deployed computing instance costs money and there are technical resources required to keep it all running. Harder to quantify, but every bit as real, are the developer resources required to squeeze out every drop of performance as the transaction load rises.

Fortunately, the ad exchanges do offer some help with the problem. Most of them allow a DSP to set some filters on the traffic of interest, either by geography or by exclusively looking at traffic associated with a particular cookied population.

The exchanges will also reduce the “offered load” to a DSP if the DSP hasn’t scaled and is not meeting response time criteria, or if bid rates or win rates fall off. Depending on the exchange, the effect of this can be to artificially even out the impression opportunities seen throughout the day. The other real impact of this is to greatly diminish the ability of a DSP to bid on the desirable opportunities.

Agencies and advertisers often take it for granted that a DSP has a view of all inventory flowing from a source merely by virtue of being RTB-enabled with the source. Put another way, it’s taken for granted that all DSPs see the same inventory. Our experience is that this isn’t true. Pre-set filters, processing power, software code quality and even what other advertising (due to its impact on bid rates and win rates) is being pushed through the platform all impact exactly what inventory a DSP sees.

Categories:Posts from 2010


2 thoughts on “Drinking From The Fire Hose”

Leave a Reply

Your email address will not be published. Required fields are marked *