We recently launched a new reporting interface for our ad network customers to provide more actionable insights into downstream publisher sources. The early feedback from new adopters has been very positive. In this blog post, I will provide a brief overview of the new enhancements.

So, why did we decide to roll out a new reporting platform? As we are growing with our customers, we learned to better understand their most pressing business problems and how our products can help address those. Downstream publisher management and new publisher screening are critical to daily operations of ad networks. We were looking for new ways to tackle them in a more effective manner.

Based on insightful customer feedback we had received, we worked on the new reporting interface with the following design goals in mind:

• Present actionable insights into large volumes of click data to better highlight traffic quality from downstream publisher sources
• Easily handle daily click volumes of 10 million clicks and deliver sub-second interface response times
• Provide detailed background information about new publishers during the approval process before accepting traffic

Users of the new interface are greeted by the publisher dashboard that summarizes current network and publisher activity. It is the starting point for further investigation. At a glance, ad network staff find their top publishers sorted by different attributes, for example by volume and traffic. The “Movers and Shakers” sections highlight sources with significant traffic changes in certain dimensions. Here is a screenshot of the dashboard:

Throughout the interface, we added small sparklines to offer traffic information at a glance without having to leave the page. For example, an average click score is accompanied by a score histogram (see this previous post for more details) and a volume volatility graph. These cues help users to absorb the information more quickly and spot areas that warrant further investigation:

Another tab in the interface offers publisher screening capabilities. It supports ad networks in making better decisions about which publisher applications to approve before accepting any traffic. After entering a URL, a report will present detailed background information about the specific domain, combining reputation information from the Click Forensics community database and public sources like Alexa, Compete.com and whois.

Next to the highlighted features, we have added numerous other useful capabilities for our customers. The new reporting interface is now available to all of our ad network customers. If you would like to learn more please don’t hesitate to contact us.

Today we announced the pay-per-click (PPC) fraud figures for Q2 2009.  The data comes from the Click Fraud Index.  Traffic across more than 300 ad networks is also reflected in the data.

Key findings from data reported for Q2 2009 include:

-    The overall industry average click fraud rate was 12.7%. That’s down from 13.8% for Q1 2009 and from the 16.2% rate reported for Q2 2008.
-    Click fraud traffic from sophisticated sources and scripted programs rose again in Q2 2009. This included a rise in the incidents of publisher collusion fraud on ad networks.

The data in Q2 also showed that many of the new click fraud schemes identified last quarter continue to increase in number and sophistication. Publisher collusion fraud was one example. This scheme occurs when online publishers use rotating IP-addresses or botnets to click ads on their own sites in order to generate inflated commissions from unprotected ad networks. Ad networks have difficultly differentiating such attacks from valid clicks.

Sonny Corleone, in The Godfather, said “I want Sollozzo. If not, it’s all-out war: we go to the mattresses.”  While the origin of the “mattress” phrase is debatable, the meaning is clear: it’s time to get serious and face this threat head-on.  (Yes, this is the line Tom Hanks referenced in “You’ve Got Mail.”)

Why am I writing about gangster movies?  Because that’s the image that “collusion fraud” conjures up, appropriately.  Collusion fraud occurs when online publishers use rotating IP addresses, or botnets, or just armies of people, to click on paid search ads on their own sites.  Collusion fraud is hard to find because the fraudulent clicks are spread across dozens (or hundreds) of sites and the clicks are generated from dozens (or hundreds) of different IP addresses.  So everything looks kosher.  But it’s not.

Standard click fraud detection mechanisms employ various anomaly detection rules.  1,000 clicks from the same visitor in the span of a minute, or an hour, or week, is a traffic anomaly that’s easy to find.  Almost all search engines and ad networks would screen this out.  But the same number of clicks spread across hundreds of sites by hundreds of “visitors” could look totally normal.  Anomaly detection rules won’t help find it.

Click Forensics employs proprietary technology utilizing high-dimensional cluster analysis to find publisher collusion.  Ad networks rely on this technology to find fraud that they would miss with their standard in-house anomaly detection rules.

The first separate “product feature” that results from this advanced cluster analysis is the Block List that we announced earlier today.  Utilizing the block list feature, ad networks can eliminate this collusion fraud from online campaigns.  They won’t pay for fraudulent clicks, and they’ll deliver higher quality traffic to advertisers.

We’ll write a lot more about this technology in future posts.  In the meantime, it’s time for ad networks to go to the mattresses.

Over the past week four major players in the online media space have announced accreditation to the Interactive Advertising Bureau’s Click Measurement Guidelines.  This list includes Yahoo!, Google, Microsoft and Business.com.  I wanted to take a moment and explore why you should care about this development and what accreditation means for advertisers.

The IAB is a publisher-focused organization that has led the process to develop click measurement guidelines.  The task force is made up of thirty or so companies representing the online advertising community.  Click Forensics has been a member since day one and participated in every step of the process.

There are three main benefits for advertisers and conversely, three concerns advertisers need to keep in mind associated with the entire process.  First, the benefits;

IAB Accreditation Represents a Commitment
The process to become accredited to the IAB guidelines is time consuming and certainly not free.  At Click Forensics, we have first hand knowledge of this and can assure you that any company that takes time and spends the money to become accredited is committed to their customers.  The level of detail the auditors go into is amazing.  Our community is fortunate to have auditors that have demonstrated a deep commitment to both the development of the process and the implementation of the guidelines.

IAB Accreditation Demonstrates Leadership
The IAB established a gating period to allow member companies and others to become accredited to the guidelines.  The companies mentioned above were the first to announce compliance.  This is important because it represents a sense of urgency among these four that enhances the urgency for others.  As an advertiser, you should reward these leaders with business.  They were first out of the gate and in my book that demonstrates leadership.

IAB Accreditation Means Better Quality Traffic
The IAB Guidelines are a lengthy narrative of “best practices” and rules in delivering quality traffic to advertisers.  While it is not intended to be a complete list, it serves as a firm foundation and includes practical steps to help ensure advertisers get what they pay for.  By working with an accredited ad provider, advertisers will be assured that the clicks they are buying have met the guidelines established by the industry.  This is a good thing and an excellent first step.

While we applaud the efforts of the IAB, Media Rating Council and member companies who participated in this process, there are things advertisers need to keep in mind.  There was a great deal of discussion and debate during the nearly three years of meetings it took to develop these guidelines.  In that process, there were a lot of valuable and important items that fell to the floor.  This is a good start, not a perfect process.  Keep in mind the following;

IAB Accreditation is a “Moment in Time” Process
The process for an ad provider to become accredited is a long one.  The auditor is invited in for a pre-assessment then the actual audit begins.  At the end of the process accreditation is awarded.  The problem is there is no mechanism for ongoing compliance.  When we buy gas at the gas station there is a meter that is routinely calibrated to ensure that when we fill our tank with 20 gallons of gasoline, we get 20 gallons.  This approach is not taken nor addressed in the guidelines.  While an annual audit is suggested in the guidelines, it is still important for advertisers to be monitoring their campaigns and holding the ad providers feet to the fire for every click.

IAB Accreditation Does Not Cover Everything
The 27 page Guideline document is quite comprehensive.  Our task force worked hard to ensure that both the guidelines are made clear and that the standard for measurement is defined.  However, when you consider that the dominant constituency in this process was multibillion-dollar ad providers, you might imagine not everything met their liking.  A few examples of chaff that hit the threshing room floor included:

Click ID – Each click should have a unique identifier so investigations can be “apples to apples”
Persistent Cookie – It’s important that ad providers can identify unique visitors to ensure they are billed for only once.
Standards for Investigation – Advertisers deserve to feel confident that they get what they pay for.  By setting an investigation format and agreeing to a timeline, ad providers can build trust with customers.

IAB Accreditation is a Roadmap
There is a Japanese proverb that says, “Beginning is easy and continuing is hard”.  There is truth in this as it relates to the guidelines.  We have begun the process.  We have released guidelines that will make the world of online advertising a better place.  Now we should look to leadership to take the next step and continue what we have begun.  The current guidelines will serve as a roadmap to the future standards.  We need to examine the items removed, listen to the community and think of better ways to ensure advertisers get what they pay for in the future.  The roadmap has been built.  Now we need to move on.

In January of 2006 as Click Forensics was just beginning as a company, I wrote the following challenge to our industry:

“Define standards for what an unwanted click looks like. We believe that there are certain characteristics or attributes that are common to a large percentage of click fraud. We are working with publishers and advertisers to agree on common ground and work together to expose it. Once this is developed it should be published so that the entire community can benefit from it.”

Today, over three years later, we have the cooperation of community leaders, the foundation of technical standards and the desire to continue to improve on what we have built.  I invite you, to join us as we build a future of ongoing growth and improving effectiveness by enhancing the process of online advertising.  I can assure you that both the Click Quality Council and Click Forensics will continue to support the work of the IAB and other industry organizations to work together to make our community a better place.  Let’s not stop with the foundation.