Just when you thought the fraudsters couldn’t get any more sophisticated … they surprise you.  Click Forensics researchers have recently discovered one of the most advanced sources of click fraud we’ve seen.  We’ve named it the “Bahama botnet” because when first discovered it was redirecting traffic through 200,000 parked domain sites located in the Bahamas.  It has since been reprogrammed to redirect through other intermediate sites hosted in Amsterdam, the U.K., and even San Jose, CA, but the Bahama name stuck.

Interestingly, the Bahama botnet appears to be closely related to the recent spate of “scareware” attacks, such as the one perpetrated against The New York Times digital site just a few days ago, reported by ComputerWorld.  Visitors to the NYTimes.com site were greeted with a pop-up informing them their computer was infected and directed to an authentic-looking site where they could install a program called Personal Antivirus.  Users duped into purchasing this phony software were then infected with a Trojan that gave control of their computer to an unknown third party that we now know to be part of a gang in the Ukraine.

We believe the Bahama botnet is controlled by this same gang, or their neighbors down the street.  More info about the “Ukranian fan club” can be found in Dancho Danchev’s excellent security blog.  We’re pretty sure the Bahama botnet is related to the Ukranian fan club and the NYTimes.com scareware because they each phone back to a bogus “Windows protection” domain located on the same IP address.

These sources were originally identified by the Black Hat community, but we believe Click Forensics is the first to discover the breadth and depth of click fraud being perpetrated by the botnets it controls.  And the botnet is incredibly insidious.

As seen in this video of the botnet in action, caught on film and narrated by Click Forensic’s own Matt Graham, the infected machine will exhibit some really funky behavior.  Clicks on organic search results are redirected through a series of parked domains across a number of top-tier ad providers (search engines and ad networks), eventually arriving at an advertiser unrelated to the original query.  The user is momentarily confused, but likely just performs the search again, this time with easy success.

What makes the botnet so insidious is that it operates intermittently so that the user doesn’t really know that anything is wrong.  Additionally, it can operate independently of the user because the authors appear to be building a large database of authentically user-generated search queries.  And because the queries come from many different machines (IPs) across a broad segment of the Internet population, it is very difficult to find and identify these clicks as fraudulent.  But these auto-generated clicks were not able to disguise themselves well enough to escape Click Forensics anomaly detection algorithms.  Additionally, large amounts of non-converting clicks were spotted in the data we receive from advertisers.  From there, our team was able to hone in on the source of the Bahama botnet.

Wow!  Click fraud is real?  Click fraud costs online advertisers millions of dollars?  Click fraud can be uncovered and the perpetrators caught and punished?   Who knew?   Well, we did.

This week Microsoft filed the a complaint in U.S. District Court (Microsoft v. Lam, et. al., case number 09-cv-0815) seeking injunctive relief and damages from a group of people found to be perpetrating click fraud through the Microsoft adCenter platform.  This is only the second time (Google sued Auctions Expert International in 2004) that a search provider has ever caught and sued an individual (or a family, in this case) for click fraud.  We congratulate Microsoft for their efforts to root out this activity and encourage them in their pursuit of relief.  Online advertisers should appreciate knowing that click fraud does not always go undetected or unpunished.

For those not familiar with the case, it’s an example of what we call “competitor click fraud.”  The motivation of the perpetrators was simply to obtain higher-placed ad positions for lower bid amounts by depleting the daily budget of their competitors.  The verticals affected were auto insurance and the online role-playing game World of Warcraft.  Microsoft identified two brothers and their mother who controlled adCenter accounts that benefited from this fraud.  They believe that this scheme affected more than just adCenter advertisers, but also the advertisers on competitive search engines.

Microsoft’s complaint, now public information, is so well written it could be used as a tutorial on click fraud detection.  The most fascinating section describes the nearly year long game of cat-and-mouse played with the defendants.  Reading from the complaint: “When Microsoft took steps to mitigate these automated attacks, the perpetrators followed by implementing countermeasures to Microsoft’s actions.  A cycle of events ensued whereby the Defendants would update their attack methods to bypass the fixes implemented by Microsoft, and Microsoft would take additional steps to combat the new click fraud attacks.”

The lessons here are pretty clear:  Click fraud is still a problem and solving it requires constant vigilance.  The online advertising community needs to work together – search engines, ad networks, advertisers, and third-party auditors – to protect ourselves from this threat.

Today the Click Fraud Index for Q1 2009 was released, and the startling news would appear to be that the click fraud rate in Q1 dropped to 13.8% from over 17% a quarter ago.  But there are several plausible explanations for the sharp drop in click fraud.  The real news was in the types of fraud that were identified, and the likely targets of these new attacks.

So why the drop-off?  Several factors likely contributed.  First, the Q4 fraud rate was unusually high, the highest ever in the history of publishing the click fraud index.  Due to the huge amount of online spending during the holiday season and the eroding economic climate, there was a sort of “perfect storm” for fraudulent activity.  In Q1, by contrast, there was a drop-off in online ad spending — Google reported their first-ever quarterly revenue decline — and more importantly, a decrease in the average cost-per-click (the number of paid clicks still increased by 17%, even though total revenue was down).  Lower CPCs means less reward for fraudsters.

Maybe the most important contributor to the reduced click fraud rate, though, was the heightened awareness of bots, worms, and other forms of malware created by the Conficker reporting (anyone see “60 Minutes?”).  It would appear that the tier 1 ad providers and ad networks did a much better job of mitigating fraudulent activity in Q1 than in previous quarters.  We can only hope this is a trend that continues.

But the more interesting trend uncovered in the Q1 data were the new types of fraud and the new targets of click fraud.  The data showed several examples of malicious scripts (JavaScript) designed to perpetrate click fraud.  When a visitor lands on a site these scripts execute by opening a zero iframe or zero-pixel window and clicking on paid ads.  The site visitor never sees these hidden frames and never visits the advertiser’s site.  But the advertiser pays for the click, and the site owner reaps the commission for the paid click.  Because the clicks are generated by a real browser with a valid IP address on a real web site with no suspicious repeat clicking patterns, this type of fraud is very difficult to discover for the average advertiser or unsophisticated ad network.  (No, we can’t tell you how we find it, but we do.)

The bottom line is that the click fraud rate was down in Q1, but click fraud schemes continue to get more sophisticated.  As tier 1 ad networks and ad providers like Yahoo! and Google continue to increase their efforts and effectiveness, the fraudsters will migrate elsewhere.  Tier 2 ad networks must focus on traffic quality initiatives in order to protect their advertisers, and themselves.

This past week in New York during the Search Engine Strategies conference, we sponsored a VIP dinner for the Click Quality Council.  Attendance was robust.  Although the food and drinks were excellent (thank you, McCormick & Schmick’s) I think the enthusiastic response to the CQC invite was more about the guest list than the menu.

In attendance were a wide variety of senior executives from large ad networks, online publishers, and agencies, including at least three CEOs and a dozen VPs.  Tom Cuthbert, president of Click Forensics, served as host and MC for the evening.  Some of the special guests included Dr. Alex Tuzhilin from NYU, a recognized authority on click fraud and online traffic quality, and author of the famous Lane’s Gifts v. Google report, as well as Joe Lazlo, Director of Research for the Interactive Advertising Bureau (IAB).

Joe was an especially welcome guest in light of the imminent Click Measurement Guidelines that the IAB will be publishing very soon.  Both Click Forensics and the CQC have been active participants in the discussions and debate that have formed the proposed guidelines.  Joe updated the Council on the fact that the guidelines were currently open for public comment, and provided a brief overview of what the guidelines (and the IAB itself) were designed to accomplish: ensuring advertisers get what they pay for.  Since that’s a core tenet of the CQC, the message was well received.

We thank Joe for his participation and hope to see him again at future events, perhaps even the next CQC Dinner in San Francisco on April 21 during Ad:Tech?

The remainder of the evening was filled with lively discussion about click quality and plenty of war stories having nothing at all to do with click quality.  I hope everyone found the evening as productive and enjoyable as I did.  Kudos to Laura Wolf for organizing and executing yet another successful event!

Depending on whom you ask, search marketing is either in a world of hurt or faring pretty well. While the mainstream media is quick to highlight the decline in online advertising, their focus is usually on display (banner) advertising and the plummeting CPM rates that publishers and ad networks can command. By almost all accounts, 2009 will be a difficult year for display advertising. But search advertising (CPC) continues to be a bright spot that will continue to shine for the foreseeable future.

Prior to Google’s and Yahoo’s earnings reports, the estimates for Q4 search spending were all over the map. An Efficient Frontier study concluded that search advertising spending had dropped by 8% in the most recent quarter. Another SEM firm, SearchIgnite, reported that search spending by retailers was up 12%. And yet another search marketing solution, Clickable, reported that Q4 search advertising spending saw “marginal search … spending increases” in the same quarter. So based on data from three leading solution providers, search advertising spending in Q4 either decreased, increased, or stayed about the same. Thanks, guys.

We now have earnings results from Google, Yahoo, and Microsoft and the picture from these Tier 1 search providers is pretty clear.

Results from Microsoft’s online and Yahoo’s search businesses were relatively flat, which isn’t all that bad in an economy where a 10% decline is often viewed as “good news.”  But Google’s results showed an 18% growth in paid clicks and revenue. That’s just stellar.  Since Google controls the large majority of the search market and 98+% of Google’s revenue comes from paid search, this means two things. One, Google is increasing market share. No news there. Two, the market as a whole is still growing at double-digit rates. That’s the real surprise and the truly good news.

Sure, we all have fond memories of the days when the CPC market was growing at 100% and new ad networks and business models were sprouting every week. Heck, that was only last year! But it’s important to remember that in a world where the unemployment rate in California exceeds 9% and even Google is cutting costs and laying off employees, the CPC market is still healthy and growing.

This week we released the Click Fraud Index® numbers for Q4 2008 and they showed a surprising uptick in the click fraud rate last quarter. I say surprising because the overall Index had been trending slightly down for almost a year.  But in bad times, fraudsters become more active. I think it’s healthy for an industry to pause from time-to-time to focus on the downside of business, to make sure proper controls are in place, that people get what they pay for. That’s the point of the Click Fraud Index. After all, no one was focused on Bernie Madoff when the market was going up, right?