This tool turns YouTube into a phenomenal source of insights.

The fourth chapter of the Math Marketing paper describes the current landscape of potential Math Marketing partners.

If companies need external partnerships to stay at the forefront of the Math Marketing revolution, whom should they partner with and what should they look out for? The Math Marketing partner of the future needs to be strong across three dimensions.

Broad Aperture
Math marketers must be able to look at the marketing landscape in its entirety. This includes all media and all elements of the marketing mix. They should understand the impact of marketing on overall business results. They also need the senior-level client relationships that can give them the authority
to take, and act on, this broader view.

Specialty Skills
Math marketers must have the specialty skills necessary to master the modern Math Marketing toolkit. This includes econometric modeling, data mining, statistics, Web analytics, online ad serving competencies, quantitative market research techniques, dashboard and visualization technologies, and social media analytics. They must always stay abreast of the latest developments in the Math Marketing industry; departments must also have the robust technical infrastructure to handle the large volumes of data involved in modern Math Marketing.

Access to Data
Math marketers need access to data. Traditionally, most data was owned and stored by clients. While clients still internally house the bulk of transactional and customer-level data, the digital era has seen other islands of data being created outside of clients’ corporate data warehouses. Microsoft and Google got in the ad-serving game with their acquisitions of aQuantive and DoubleClick, respectively. They are rapidly gathering online ad-serving data that can be used to optimize online ad spend. Online media agencies are investing in data infrastructures that enable them to get feeds from ad servers and store them themselves. Search engines have the most powerful source of search data for needs-based targeting. But very few companies have integrated this digital data with their customer data warehouses. It will be interesting to see in the next couple of years who will own what data and who will be creating the next-generation, single-customer views that will combine media exposure and interaction with search, website behavior with transactional and demographics data, all in one place.

There are various players in the Math Marketing area:

Media companies have always used mathematics to make decisions about investing media budgets. Most of them have econometric capabilities and are investing in ad-serving, data-warehousing infrastructures, which will allow them to mine that data for optimization and give them a considerable competitive advantage. They are predominantly focused on media decisions and have historically had a more transactional and less strategic view. This was usually reflected in client relationships, where media agencies tend to have less board-level access. This is changing rapidly, however, with media companies aggressively building their strategic capabilities. If they can attract the talent and own the data, they will be in a great position to answer the ROI questions of the future. However, there will always be the perceived issue of the “fox guarding the henhouse.” Clients will always have a suspicion that media agencies will favor recommendations that might benefit media agencies first.

Integrated communications agencies have traditionally had strategic relationships with clients through brand-planning teams. Some of them have built strong strategic services departments over the years. Teams like these will be able to take a broad view of ROI, and look at the impact of all 360 Degree marketing communications on a brand holistically. Their analytics teams also tend to have the technical skills to analyze large volumes of data. However, with new channels being introduced continuously, the teams will need to keep investing in new specialists who understand the inner workings of these channels well enough to use the data to answer the ROI question. (This technical knowledge might come more naturally for media agencies who know the details for media execution purposes.) Ogilvy’s strategic services group in New York, for example, is comprised of a team of 150 planners and consultants specializing in brand planning, marketing and digital strategy, and analytics. The integrated communications agencies usually get access to data through clients and partnerships with media agencies. Very few communications agencies store their own data, which could put them at a disadvantage versus others in the category.

Management consultants have entered the marketing ROI arena as an extension of their broader strategic services. For them, marketing represents perhaps the last area of expansion. By the very nature of what they do, they tend to take a broad and holistic view and probably have more senior client relationships than any other player in the category. This can give them the authority to tackle the marketing ROI question holistically. They are also often seen as more impartial than media and communications agencies. One of their main challenges is that they are often too far removed from the day-to-day execution of marketing campaigns, which means they often lack the in-depth technical knowledge of today’s marketing channels required to analyze ROI. Some of them are ramping up their services fast, though, as demonstrated by Accenture’s acquisition of digital multivariate testing company Memetrics in 2007. Management consultants usually do not have direct data access and therefore rely on clients to share data with them.

As mentioned earlier, Google and Microsoft entered the marketing effectiveness area with their acquisitions of the major ad servers in the industry. With the digitization of all media, they will soon hold the majority of all marketing effectiveness data. They have the capability to handle the largest volumes of data, and have mathematicians and engineers who, if focused on marketing effectiveness, could have the ability to do just about anything. They are not positioned strategically and therefore lack the broad aperture required to become real ROI partners, but they have the size and the financial power to change this quickly.

These main players will be supported by specialized companies to help them solve certain pieces of the overall ROI puzzle:

Agencies with a strong CRM and technology focus, such as Epsilon, Acxiom, Targetbase, Unica and Dunnhumby, have a great ability to mine high volumes
of customer-level data. Some of them also have easier access to data, as they tend to manage clients’ data warehouses. They have mastered the art of optimizing one-to-one communications, but they are almost never responsible for the entire marketing mix, which usually prevents them from looking at marketing ROI holistically.

Marketing research agencies help quantify ROI through surveys and panels. Since they are primarily the providers of research data and insights, some of them have developed extended ROI capabilities in brand measurement and econometric modeling. Millward Brown’s Optimor is a great example of such an extended capability.

Finally, there are analytics vendors who specialize in just one area of analytics. This category includes web analytics vendors such as Omniture and WebTrends; marketing mix specialists such as Market Share Partners, MMA and Hudson River Group; Financial social media analytics vendors such as Nielsen BuzzMetrics, Visible Technologies and TNS Cymfony; and many more. These players usually have partnerships with some of the players mentioned above and, in some cases, have even been acquired by them.

The diagram below summarizes the current Math Marketing landscape, with the main players plotted on a grid based on their ability to have a broad aperture and on their specialty skills.

A true ROI partner will have a combination of both and would therefore appear in the top right corner. No one is there at the moment. However, it is clear that the main contenders are management consultants, integrated communications agencies, media agencies, and Google and Microsoft.

The use of mathematics in marketing seems to be one of the hot topics today.  But is it that new?  Not really.  Here’s a little piece I wrote recently on the history of mathematical marketing.  I split in into four main era’s.

First Era : The Early Days of Direct Response

It’s probably fair to assume that the 1st applications of mathematics in marketing started soon after the invention of the first direct response campaigns. The first mail-order catalog was invented by Aaron Montgomery Ward in 1872, and it was copied by Richard Sears and Alvah Roebuck in 1886.  While there is no real evidence of how the early catalog pioneers measured their success and optimized their operations, they had the ability to do so, and the fact that catalogs are both around today suggests they probably did a good job at it! 

Claude Hopkins’ Scientific Advertising (1923) was one of the first books on the topic.  It opens with the following words: “The time has come when advertising has in some hands reached the status of a science. It is based on fixed principles and is reasonably exact. The causes and effects have been analyzed until they are well understood. The correct methods of procedure have been proved and established. We know what is most effective, and we act on basic laws.”  Hopkins and, later, John Caples — with his Tested Advertising Methods (1932) — wrote mainly about mail-order and other direct response vehicles. 5 They measured what was easy to measure and thereby focused mainly on short term effects. Unfortunately, from a measurability POV, the primary focus of marketing efforts would soon be directed toward mass media, and hence new techniques would be required to maintain the same levels of marketing accountability.

Second Era : Mass Marketing Effectiveness

The first applications of more advanced mathematical techniques in marketing can be traced back to the 1950s, when operations research and management science models in production and manufacturing that had become popular during and just after WW II were being applied to marketing for the first time.  In those days, of course, marketing relied nearly exclusively on mass media such as print and radio, and later, TV. Data on effectiveness of marketing in these mass media was scarce, which meant that the application of scientific methods in marketing had its limitations.  Data was either gathered through tracking sales and investments over time or through polls, which had been around ever since Raymond Rubicam hired George Gallup in 1932.  Panels were another popular source of data. But econometric modeling became the technique of choice in this era.  It helped marketers better understand the impact of various elements of the marketing and media mix on outcomes such as brand awareness, consideration and, ultimately, sales and profit.  Early work from the likes of Timothy Joyce, Colin McDonald, Simon Broadbent in the UK and John Little in the US helped shape scientific marketing in this era.

Today, independent companies — such as Market Share Partners, MMA and the Hudson River Group — specialize in econometric modeling and still use pretty much the same techniques to make recommendations regarding the effectiveness of mass media.  This includes determining the impact of different marketing investment levels, the contribution of individual elements of the marketing mix, and the timing of the effects.  The insights lead to recommendations as to how much of any budget should be allocated to TV, radio, print and OOH, and what the timing and geographic dispersion of the investment, should be.

Econometric modeling has been around for a while now, and its power in helping marketers understand what works and what doesn’t has been demonstrated over time.  Today, however, its use is still relatively limited. To with, the UK’s IPA awards set the international gold standard for advertising case material, but focus, on proof of effectiveness, only 15% of the case studies submitted for the awards use econometric modeling to identify the effects of campaigns. It seems that after all these years, econometric modeling still hasn’t been adopted in day-to-day marketing decision making.

Third Era : 1990 CRM Effectiveness

The third era happened during the 1990s, when customer relationship management (CRM) became an obsession for many marketers.  During this period, the possibilities offered by new, powerful database solutions really transformed direct marketing — and scientific marketing with it.  The customer relationship management (CRM) revolution in the 1990s forced companies to think in a customer- centric way.   Frederick Reichheld published The Loyalty Effect in 1996, in which he demonstrated that a 5% improvement in customer retention rates usually yields a 25% to 100% increase in profit.  That same year Garth Hallberg’s All Consumers Are Not Created Equal appeared, in which he demonstrated that a small proportion of the average company’s customer base usually represents a disproportionate share of company revenue.

Companies became determined to get to know their most valuable customers and focused on keeping them by treating them differentially. Loyalty cards were introduced that allowed companies to capture transactional data, and they invested heavily in data warehousing technology that stored all customer information in one database.  These “single customer views” allowed companies to analyze their customers’ transactions, value, responses to communications and even demographics.  RFM models classified customers according to recency, frequency and monetary value of purchases.  Lifetime value models predicted what a customer would be worth over their entire lifetime.  Anti-attrition models were built to predict an individual’s likelihood to cease being an active consumer.  In this same period, a plethora of other analytical tools and frameworks were born that allows companies to better understand who their most valuable customers are, what their next move would be, and how they could be influenced through direct, one-to-one communications.

Many of the mathematical techniques behind these models were very old.  Statistical techniques such as logistic regressions and discriminant analysis became powerful tools, once applied to customer level data.  These more traditional techniques were supplemented by new data-mining techniques mad possible by ever-increasing computing power that collected vast quantities of data, as well as by developments in machine learning and artificial intelligence.  Data mining gurus such as Michael J. A. Berry and Gordon S. Linoff made new techniques such as neural networks, genetic algorithms and decision trees popular and added them to the mathematical toolkit.  To this day, companies such as Dunnhumby, Epsilon and Acxiom still thrive in what is now a mature, very scientific, data-rich CRM industry.  The CRM revolution expanded marketing effectiveness tools and techniques considerably, and the toolkit’s ability to analyze vast quantities of data was soon tested on customer-centric data derived from digital media.

Fourth Era : Digital Effectiveness

One of the main promises of the digital communications era is that everything is measurable.  In digital, everything generates data — and the volumes are enormous.  Google’s digital database is probably the largest, capturing almost 10 billion searches per month.  These huge quantities of data can give companies unprecedented visibility into how our customers engage with brands and how that engagement ultimately leads to revenue. 

E-commerce environments provide us with a closed-loop system, which in marketing effectiveness terms gets us close to nirvana.  Digital media data can show us exactly which media individuals have been exposed to.  Website data can show us where individuals came from (or, in the case of search, what terms they typed in to arrive at a site).  We can then observe these individuals’ entire shopping behavior, all the way to their actual conversion to a sale.  With more and more media becoming digital, we could easily imagine a scenario where most, if not all, media exposures can be traced to an individual sale.

Digital data is also available in real time.  We no longer have to wait weeks or months before we can observe the impact of our marketing activities.  We can get a read almost instantaneously, allowing for real-time optimization.

This abundance of data, the promise of a closed loop and the speed with which we can react to insights have given birth to a wide range of analytical services in digital. 

Web analytics vendors such as Omniture, Coremetrics and Google Analytics specialize in gathering the vast amounts of data generated by websites and transforming this data into insights as to how many people come to a site and how they behave.  This very powerful information can help streamline online processes such as online registrations, downloads and purchases, and it  can play a vital role in  site redesign and optimization.

Ad servers such as Google’s DoubleClick can provide data on online media exposures and click-throughs (and beyond the click data) that enable us to optimize real-time frequency of exposure and automatically drive creative rotation decisions.  Companies such as Memetrics (now part of Accenture), Offermatica (now part of Omniture) and Tumri have automated multivariate testing.

Tacoda (now part of AOL) and Audience Science (former Revenue Science), among others, are applying the mathematical targeting techniques first pioneered in the CRM era to digital data in a way that has made behavioral targeting almost a commodity.

Vendors such as TNS Cymfony, Nielsen BuzzMetrics and Radian6 specialize in analyzing what people write on blogs and message boards and in forums.  Other vendors such as 33Across are popping up who will analyze the connections between people on social networks to optimize social media communications.

Soon all media will be digital.  Today, Google TV enables you to use TV set-top-box data to analyze advertising tune-out rates, allowing us to optimize their TV commercials by using a number of the digital optimization techniques mentioned above.

It seems that, almost every day, new companies find ways to apply mathematics to the vast amounts of digital data currently available in order to optimize marketing efforts.  Digital really has put the math revolution in marketing on steroids.

Here’s a very cool little app that allows you to search what is being mentioned on TV and trend the data over time.  It is provided by SnapStream whose servers gather closed captioning data of TV content.  Below you can see an example of how often “Obama” is mentioned over time. 

Not surprisingly, the top 4 mentioned topics today on TV are all related to Michael Jackson’s death : “michael jackson”, “the hospital”, “the ucla medical center” and “cardiac arrest” … .

You can also download the data in excel which could make it a great data source for capturing the impact of PR in marketing mix models.

Last week Microsoft launched Bing, it’s new search engine.  Does it outperform Google? 

You be the judge with this cool little Google vs Bing comparison tool. 

On Monday, British scientist Stephen Wolfram is probably going to go live with his Wolfram Alpha (or “Alphram”, as it’s called affectionately) – a giant electronic brain that has been touted as the next Google Killer.  If you want to get a good understanding of how it works have a look at this webcast.  Though was pretty impressed when I first saw this, it’s clear that Alphram is quite different from Google or any other search engine.

It doesn’t search the web –Rather than searching the entire web, Alphram sits on top of a huge database that has been combed over by scientists on Wolfram’s team.  This database currently holds mainly academic data and facts.  Wolfram claims that “probably 90% of the shelves in a reference library we can say we’ve got a reasonable start on”.  According to some test users it is light on popular culture.

It’s computational knowledge engine, not a search engine – The system computes answer to questions such as “What is the average temperature in Chicago?” or “What is the average wage of a high school teacher”.  It does this by mapping the questions to data components in the underlying database which are then combined to compute the answer.  This is fundamentally different from how search engines work.  If you ask them a question, they will retrieve answers that have already been formulated somewhere on the web by someone else.  That, of course, already covers a lot of ground so it remains to be seen whether computing answers to new questions is actually going to be perceived as a real advantage by the users.

It returns more than just a list of answers – The display and organization fo the information is where the Alphram adds a lot more value than search engines.  Search engines have put all their energy in becoming really good at finding information but in doing so, one could argue they have neglected the actual organization and display of the information they return. 

It is in this area that Google has really been trying to spoil Alphram’s coming out party.  As mentioned in a previous post, Google bought Gapminder but they haven’t really done anything with it other than create the Public Data feature they came out with recently.  More promising is the beta version of Google Squared they will be launching later this month.  It will return search data as a matrix.  The video below shows a demo.  The quality is pretty bad but you do get a good view on how it works.

One potential downside of Alphram is that it only likes “sensible” questions.  In the official screencast mentioned above, every question gets answered properly.  However, more objective reviews, like the one from CNET below, show that about 2/3^rd of the searches did not return the appropriate results.  This is because Alphram only seems to work well with structured questions, which suggests users will need to learn how to use the tool properly.

So Alphram is no replacement for Google.  But it feels like a new and very exciting cross between Google and Wikipedia.  And if people have the patience to learn how to use it, it may challenge search engines to become more innovative in the way they display search results.  As for the launch on Monday, it’ll be hard to have one that is worst than Cuil’s!

My earlier post talked briefly about the power of testing and the emergence of a different breed of creatives who embrace the feedback of results they can get from the web and make it part of the way they work.  But can the role of data in the creative process be taken too far?

According to Doug Bowman it can.  2 weeks ago he resigned as Visual Design Lead at Google and joined Twitter as their Creative Director.  Doug posted the following on his blog   :

“I’m thankful for the opportunity I had to work at Google. I learned more than I thought I would. I’ll miss the free food. I’ll miss the occasional massage. I’ll miss the authors, politicians, and celebrities that come to speak or perform. I’ll miss early chances to play with cool toys before they’re released to the public. Most of all, I’ll miss working with the incredibly smart and talented people I got to know there. But I won’t miss a design philosophy that lives or dies strictly by the sword of data.”

Kevin Fox, who worked with Doug as at Google as Senior User Experience Design Lead, replied to this on his blog.  Though doesn’t agree that Google’s design philosophy “lives or dies strictly by the sword of data” he does have a word of caution for the Google engineers :

“Data-driven design is a vital tool for hill-climbing iteration of a site, but you should take great care not to use it as an appeals process whenever you and your designer reach an impasse. It sidelines the designer into being no more than a brainstormer, devoid of design ownership. I realize this is not the usual case, so just treat it as a cautionary tale. Also please keep in mind that everyone has opinions on design, and that your UX professional has devoted years of their life to learning to separate their subjective opinions from their objective understanding about how the larger audience will interpret an interface. It’s not as demonstrable as code that passes unit-tests, but trust in it anyhow.”

I agree testing (or any form of data driven design) can be counterproductive if you don’t learn from the past.  Obsessive testing can slow down the creative process.  As mentioned in a previous post, there are design and communications principles that have been around for a long time.  They don’t necessarily need to be tested over and over again.  Experts familiar with them should be given the authority to overrule over-eager data analysts.  A knowledge and test management system can help here. 

Test management systems usually include the following basic components :

Testing History : findings from previous tests should be documented consistently so that they can be stored in an insight repository.  At the very minimum this can be a simple spreadsheet that lists all tests that have run in the past, their objective, hypothesis and the outcome.

Testing Pipeline : the testing pipeline keeps track of all the tests that are being planned.  Depending on the number of tests you run and the complexity of your organization this can be anything from a simple spreadsheet that lists all planned tests with a short description, timeline and status to a system that is managed through sophisticated campaign management platforms.

Testing Briefing : Every test should have a briefing document that outlines the testing hypothesis, the test design, the timings, the anticipated benefit, cost and ROI and the owner.  This detailed briefing document will standardize the inputs required to build the testing history and pipeline.

Guidelines : Storing the results of past tests in an insights repository is not enough.  The findings need to be aggregated and written up in guidelines that are communicated throughout the organization.  Google’s Ten principles that contribute to a Googley user experience is a great example of how to do this.  Again, tried and tested principles do not need to be tested over and over again.

Testing Prioritization : New tests can be prioritized based on what we know already (through testing history and guidelines), what we have in the testing pipeline and what the potential ROI could be from the rollout.  Prioritization of tests will prevent excessive testing.

Now Google is not your average company of course.  I don’t know many companies who test too much.  The opposite is usually the case.  Most companies’ creative decisions don’t live or die by the sword of data, they are based on often very subjective expert opinions or even worse, on what web analytics author Avinash Kaushik calls the Hippo – the Highest Paid Person’s Opinion … . 

This means companies tend to leave millions of dollars on the table.  They don’t test new ideas and fail to build up a body of knowledge on what works and what doesn’t.  A testing management system will help these companies as well.  It can identify knowledge gaps and areas where mandatory tests are required.  And it can institutionalize testing by making it an integral part of the creative process. 

Last week Rory Sutherland explained on his blog “Why creatives should wear ties occasionally”.  He talked about the relevance of old tried and tested techniques in modern advertising and used the iPhone single page ad as an example of a “perfectly sensible kind of advertising” that is so rare these days.  He described how the ad “contains a large shot of the product so you know what it looks like, carries a headline which discloses a consumer benefit and, ‘…’ contains a series of little captions (old direct marketing hands will know them as “call-outs”) wherein a number of words are arranged in some grammatical order to convey to the reader useful information about what the product does.”

Many of these techniques were invented a long time ago by early direct response pioneers like John Caples and Claude Hopkins, whose book Scientific Advertising I mentioned in my previous post on marketing accountability in 1923.  Over the past few years we have seen a new generation of direct response practitioners emerge.  They combine Hopkins’ and Caples’ relentless focus on results with a thorough understanding of modern web analytics and testing tools. 

Bryan and Jeffrey Eisenberg are great examples of this new breed of marketing practitioners.  They have written extensively on digital consumer persuasion techniques and their blog grokdotcom is full of practical tips on how to improve online conversions through making small changes to your site. 

Bryan Eisenberg recently teamed up with Google to write Always Be Testing, a book on how to get the maximum out of Google Site Optimizer – a free testing software package by Google.  GSO allows the user to test multiple versions of their website automatically and thereby makes testing extremely easy, fast and cheap.  Always Be Testing is packed with tips and suggestions that will get you addicted to testing.  It will change the way you work.