As you can imagine, there was much to entertain data watchers at this year’s TED conference.

Here’ s a quick roundup of the talks most relevant to what we’re talking about on this blog.

There was a live telecast with David Cameron, who in a few months time is almost certainly going to become the next Prime Minister of the UK.  He claimed that the big political question of the next few years was “How do we make things better without spending more money?”  (On the very sensible grounds that there isn’t going to be much.)  His answer is that which we judge quality of life more broadly by including measures like happiness, health, job satisfaction and so on.  (The British conservatives seem to be taking “choice architecture” as their big idea, expressed by Thaler and Sunstein in Nudge, and other behavioral economists. )  He went on to argue that a new era of transparency in government was coming which would make this much easier.  His examples included crime maps like the one below, which, he argued, prompt and liberate people to make their own decisions.

Nicholas Christakis, a Harvard physician and sociologist and the author of Connected: The Surprising Power of Social Networks and How They Change Our Lives.  He used social networking mapping to show, among other things, how hanging around with fat people is likely to make you fat.

John Underkoffler invented the media interface made famous by Minority Report and is now imminently being supplied commercially through his company Oblong.  Check out this amazing demonstration.

My favorite piece of data visualization came from Carter Emmart, the director of “Astrovisualization” at the American Museum of Natural History.  He demonstrated some of the digital universe atlas, a project that plots every satellite, moon, planet, star and galaxy precisely according to the best information and brings it alive through virtual graphics.  Please, please watch it here.

Probably the most talked about new company ad TED was Blippy.  This audacious start-up can be thought of as a Twitter linked to your credit card.  The idea is similar to Facebook’s  disastrous Beacon feature, where purchases members made were automatically posted.

If this sounds like an invasion of privacy, it is – gloriously so.  To join you simply register your credit card with the site and then all of your purchases are automatically posted.  Blippy already has agreements with three online retailers (iTunes, Amazon and Zappos) which allow to cross reference other purchases.  If you want to purchase something privately, you have to use another credit card.

Founder Philip Kaplan (who you may remember as founder of FuckedCompany ), describes this as “passive sharing”, which is a provactive idea.  It reminded me of my favorite quote on privacy, from Scott McNealy of Sun Micosystems:  “You have zero privacy anyway, get over it”.

Blippy  sits at the intersection of commerce and social networking and is an attempt to exploit Jeff Zuckerberg’s insight that friends are the ultimate source of product recommendation and somewhere, somehow, there must be a way of commercializing that.  (Mustn’t there?)

Because it targets some of our most private information, Blippy points to a world of full transparency. It actually provides a much more revealing record of someone’s doings that Twitter.   If you track Kaplan’s (Pud’s) own purchases, you see that he is a “loyal” Shell customer, but only a “frequent” purchaser at Slicehost.  You can see when and where he saw Crazy Heart and what his friends thought of that movie.

Perhaps more importantly, you can also see what people paid for different purchases.  This may inhibit differential pricing strategies, like the one Amazon pursued a few years ago.

And of course, it’s the aggregated data that Blippy collects that is presumably the real money maker.  With a big enough base, you could look at spending patterns by region, retailer or any other number of cuts.



There is an increasing amount of empirical evidence that suggests TV advertising is getting increasingly effective.  Perhaps the most often quoted analysis is the one done by Les Binet and Peter Field in 2008.  They mined the IPA databank which holds more than 1000 marketing effectiveness case studies from around the world gathered as part of the IPA Marketing Effectiveness Awards.  They found that campaigns that used TV have been growing in effectiveness over the past 3 decades.  The graph below shows the average % increase in market share growth for campaigns that include TV for the 1980’s, 1990’s and 2000’s.

In June 2009 Joel Rubinson of the ARF performed one of the most extensive reviews of quantitative evidence available in an attempt to validate the hypothesis that TV is losing its effectiveness.  He combined 7 different databases of quantitative evidence of the impact of TV campaigns on sales.  They included data from the IRI panel, econometric modeling companies like PM Group, Dratfield and Marketing Evolution and research data from Pointlogic and Milward Brown.  This combined database contained more than 300 cases across 74 different product categories.  Rubinson was not able to validate the hypothesis that TV is becoming less effective.  On the contrary – 6 out of the 7 databases showed that TV advertising has become more effective over time.  The Pointlogic data for example showed that among 25 different types of touchpoints measured in their survey between 2004 and 2007, TV moved up from 7^th to 4^th in terms of people impacted per $1,000 spent.  In an interview with Brandweek Rubinson said that his study concluded that “units sold numbers increased as a result of increased TV impressions. [When you observe it] across 388 case histories, I think you’ve got to believe it.

These findings are also confirmed by marketing science company MMA.  “We haven’t seen a significant trend in the erosion of effectiveness of TV,” said Douglas Brooks, senior VP of MMA in an interview with Advertising Age in Feb 2009. In fact, MMA, which reports to clients each year on its findings regarding aggregate TV effectiveness, has seen a slight uptick in effectiveness in recent years.

As usual, yesterday’s Superbowl was accompanied by the Ad Bowl the annual contest for best TV commercial during the most expensive airtime of the year.  This seems to be the only time of the year where marketers are still allowed to get excited about TV advertising.  For the most part, TV has become a dirty word in today’s marketing world.  Most marketers can’t wait to move as many of their dollars away from it.  They claim is the medium of the past – for obvious reasons.  Most people, especially younger generations, seem to be spending all their time online. They can’t possibly have time to watch TV.  And even if they are watching it, they are probably skipping the ads on their DVR’s.  Also, with the fragmentation of TV channels, you couldn’t even reach them if you wanted to.  It wouldn’t be cost efficient.  Putting your money in TV is a bad idea.  The majority of the US marketers feel TV is no longer working (The Association of National Advertisers did a survey in February 2008 that showed that 62% of US marketers believe that TV advertising had become less effective in the last couple of years).  They have to be right. 

But are they really?  What do the facts tell us?  That’s what I will explore in the next couple of posts.  In this one we will take a closer look at TV consumption in general and TV advertising more specifically. 

 The first video is an interview with the fascinating Dr Carl Marci – the founder of neuroscience company Innerscope.  I wrote about him in a previous post but nothing beats listening to the man himself!

This is a bit of a geeky post so apologies in advance.  But I have been meaning to write about how fantastic I think Google Analytics Intelligence is.  If you are not sure what I am talking about have a look at the video below.

Just to prove that the graphic presentation is new, take a look at this selection of images from  BibliOdyssesy, a blog devoted to “amazing archival images from the internet”.  (There’s a book too.)  The image above is titled, “A timetable indicating the differences in time between the principle cities of the world”, with their air line distances from Washington.  It was published in 1883, in Philadelphia by WM Bradley.

These images come from the David Rumsey map collection.  This is the life’s work of a map collector who has created a virtual museum.  (This must surely be the future of curation.  Tom Campbell, the new Director of the Metropolitan Museum of Art, plans to put the Met’s entire collection online.)

My first reaction looking at these is “where’s the interactivity, man?”  We’re so used to things popping away in flash, that these look rather flat.   But when you look at them closely, there’s an intensity to the way every calculation is made, then drawn, by a human.

What’s interesting about them is that they have the same mixture of amateurism and fascination with new technology (in this case, printing technology) that has energized today’s data visualization revolution.

People have been buying and selling data about consumers for a long time.  Companies like Axciom have been doing this for years in the direct marketing business.  But recently a new breed of companies has been popping up who are acquiring and selling data about consumers in the online advertising space.  Companies like BlueKai and Datran are the modern day, digital equivalent of the Axcioms. The good old direct marketing techniques from the 80’s and 90’s are now also being used for targeting online advertising ads.  Therefore data collected about consumers can now be used for smarter targeting across all direct and digital channels.  This broader playing field will dramatically grow the size of the data reselling business in the next few years. 

So with so many companies buying and selling data about consumers, what really determines the value (and therefore the price) of a data point?  I think there are 3 drivers.

1. Predictive Power

The 1^st driver is Predictive Power of the data point.  Let’s say for example that I am a manufacturer of drills and that I am trying to purchase data points that will help me identify whether a consumer is interested in buying a drill.  And let’s assume that I can choose between the following 2 sets of data points.

Most people would agree that the data points in set 1 are more valuable for a drill manufacturer than those in set 2.  This is because of their natural correlation with someone’s likelihood to purchase drills.  This example is very straightforward.  If you had to determine the Predictive Power of a 100 different data points however, you would have to build statistical models that predict the likelihood of someone buying a drill based on all 100 data points.  Those that enter the model have a high Predictive Power which can be quantified by the lift they generate in the models.  Whether you build statistical models or not, the principle is that data points with a high Predictive Power will improve our prediction of whether a consumer will be interested in buying a drill and, as a drill manufacturer, I am prepared to pay a higher price for them.

2. Recency

The 2^nd driver is Recency.  This is really a special case of Predictive Power but I want to call it out separately as it has become an increasingly important driver.  In a digital world people often reveal real time what their intentions are.  Knowing whether a person has searched for drills on Google, whether they have clicked on a banner for drills or whether they have seen a drill related video online can be very powerful.  These data points generally outperform the more traditional data points that are listed in the example above because they are direct indications of a consumer’s interests and needs at a certain point in time.  For these self disclosed data points, Recency is very important.  When someone searches for a drill on Google then that is very valuable information if I can target that person immediately.  However, if I know that someone searched for a drill 3 months ago then that single observation in itself is a lot less valuable.  The predictive power of self disclosed data points starts to decline minutes after the observed event.  Because of the disproportionately high value of very recent data we anticipate most of the future innovation to focus on capturing multiple events real time and shortening the cycles between observed events and the ability to use that knowledge for targeting.  This is already happening on advertising exchanges through the introduction of Real Time Buying. 

3. Exclusivity

The final driver is Exclusivity.  Let’s use the same example and let’s assume that I can only buy the data points in set 1.  Let’s also assume that I have built a statistical model and have determined that the general predictive power of the number of hammers a consumer owns is far more predictive than the other 2 data points.  I would be prepared to pay a relatively high price for data on hammer ownership.  Now consider an alternative scenario where one additional data point is available: the number of nails a person uses per year.  Let’s assume that the general predictive power of nails consumption is almost as high as that of hammer ownership.  The availability of nails consumption will have an effect on the price I am prepared to pay for hammer ownership.  It’s the basic laws of supply and demand.

In the next few years the buying and selling of data will undoubtedly become a lot more streamlined.  When that happens, the market drivers described above will increasingly determine the price companies are willing to pay for information about their consumers.  Consumers on the other hand will get a much more transparent view of the value they are generating by allowing companies to collect their data.  Who knows, maybe they’ll even be able to claim their share of the pie.

Take a look at this superb interactive tool developed by The New York Times:  A Peek into the Netflix Queue.

The principle is simple enough.  The Times has taken a database of most rented movies from Netflix and overlapped it on ZIP codes on a Google map to create a geodemographic look at tastes and the hyper local level.  New York is shown here, but they’ve featured many cities.

The magic is in the way you can look at the same data in many different aspects.  You can rollover the different ZIPs to see their top ten picks.  You can sort by movie, using a slider to highlight ZIPs in which that movie was the most or least viewed.  You can file through the movies differently:  alphabetically, by most rented, or by “Metascore” ( a score that averages scores given by the nation’s most influential critics, compiled by the very useful Metracritc.com.)

You could call this a prizm presentation.  There’s no dominant point of view.  You have to look at it from many different angles to fully understanding it.

As you might imagine, there are polarizing titles with distinct patterns, such as Mad Men, Obsessed and Last Chance Harvey.

Like Google Earth, this is, of course, most interesting when you’re looking at your own backyard.  My own ZIP (10001 or Chelsea) doesn’t disappoint.  Milk was the most popular movie.  In fact, its pattern looks like most of lower Manhattan.  (I’m proud to say how well I conform to type, having rented every title on the top ten list.)   My friend Charlie McKittrick points out that “Brooklyn is much more eclectic and long-tail in their taste than more main-stream, follow-the-leader zip type ZIP codes like Tribecca/Chelsea and Bergen County NJ, which seem to more consistently be renting off the top of the best seller list”.  Feisty!

So what makes this compelling is the splicing of geodemographic data with some believable measure of something super soft like taste.

It seems to me that this is not only the future of data presentation, but perhaps also the future of newspaper reporting.  I found this article only mildly interesting in the print edition.  It only really works online.  Could all stories (from Madoff to Britney) be told in this multidimensional way?  And would people pay for that?

On Sunday I wrote a small post about how UK company Prozone captures all English Premier League soccer matches on video and then uses that footage to create a database of every shot, pass, tackle and assist made by every player (according to a comment on that post Opta Sports are powering the Guardian Chalkboard – sorry I got that wrong).  That’s a lot of data.  However, as I found out in the New York Times this morning, this is nothing compared to the volume of footage captured by the US Airforce spy drones.  Apparently it would take one person 24 years to watch all the surveillance footage captured in Afghanistan in 2009.  According to the article, the Army has a group of young soldiers watching every second of this footage live in order to detect suspicious activities. 

The Army expects the volume of footage to grow exponentially in the near future as new drones are being deployed that can film at up to 65 different angles.  Processing all that footage will soon become unmanageable.  The Airforce top intelligence official LT. Gen. A. Deptula said the army could soon be “swimming in sensors and drowning in data”.  So they are looking for new analytical techniques that can help reduce the volume of footage that needs to be analyzed by the human eye.  Now where do you think they started their quest for this technology?  You got it – the sports networks. Apparently Army officials have been shadowing the ESPN broadcast trucks outside football stadiums to learn how the TV network is tagging and retrieving all the highlights in their NFL footage.  It sounds like Opta Sports and Prozone have just found a second market.

One Army official called out the limitations of automated recognition technologies for the Army.  He said that when it comes to the use of force “You need somebody who’s trained and is accountable in recognizing that that is a woman, that is a child and that is someone who’s carrying a weapon.  And the best tools for that are still the eyeball and the human brain.”  But even if technology can identify the 10% of footage that needs to be analyzed by humans it would still make a big difference.