Take a look at this superb interactive tool developed by The New York Times:  A Peek into the Netflix Queue.

The principle is simple enough.  The Times has taken a database of most rented movies from Netflix and overlapped it on ZIP codes on a Google map to create a geodemographic look at tastes and the hyper local level.  New York is shown here, but they’ve featured many cities.

The magic is in the way you can look at the same data in many different aspects.  You can rollover the different ZIPs to see their top ten picks.  You can sort by movie, using a slider to highlight ZIPs in which that movie was the most or least viewed.  You can file through the movies differently:  alphabetically, by most rented, or by “Metascore” ( a score that averages scores given by the nation’s most influential critics, compiled by the very useful Metracritc.com.)

You could call this a prizm presentation.  There’s no dominant point of view.  You have to look at it from many different angles to fully understanding it.

As you might imagine, there are polarizing titles with distinct patterns, such as Mad Men, Obsessed and Last Chance Harvey.

Like Google Earth, this is, of course, most interesting when you’re looking at your own backyard.  My own ZIP (10001 or Chelsea) doesn’t disappoint.  Milk was the most popular movie.  In fact, its pattern looks like most of lower Manhattan.  (I’m proud to say how well I conform to type, having rented every title on the top ten list.)   My friend Charlie McKittrick points out that “Brooklyn is much more eclectic and long-tail in their taste than more main-stream, follow-the-leader zip type ZIP codes like Tribecca/Chelsea and Bergen County NJ, which seem to more consistently be renting off the top of the best seller list”.  Feisty!

So what makes this compelling is the splicing of geodemographic data with some believable measure of something super soft like taste.

It seems to me that this is not only the future of data presentation, but perhaps also the future of newspaper reporting.  I found this article only mildly interesting in the print edition.  It only really works online.  Could all stories (from Madoff to Britney) be told in this multidimensional way?  And would people pay for that?

On Sunday I wrote a small post about how UK company Prozone captures all English Premier League soccer matches on video and then uses that footage to create a database of every shot, pass, tackle and assist made by every player (according to a comment on that post Opta Sports are powering the Guardian Chalkboard – sorry I got that wrong).  That’s a lot of data.  However, as I found out in the New York Times this morning, this is nothing compared to the volume of footage captured by the US Airforce spy drones.  Apparently it would take one person 24 years to watch all the surveillance footage captured in Afghanistan in 2009.  According to the article, the Army has a group of young soldiers watching every second of this footage live in order to detect suspicious activities. 

The Army expects the volume of footage to grow exponentially in the near future as new drones are being deployed that can film at up to 65 different angles.  Processing all that footage will soon become unmanageable.  The Airforce top intelligence official LT. Gen. A. Deptula said the army could soon be “swimming in sensors and drowning in data”.  So they are looking for new analytical techniques that can help reduce the volume of footage that needs to be analyzed by the human eye.  Now where do you think they started their quest for this technology?  You got it – the sports networks. Apparently Army officials have been shadowing the ESPN broadcast trucks outside football stadiums to learn how the TV network is tagging and retrieving all the highlights in their NFL footage.  It sounds like Opta Sports and Prozone have just found a second market.

One Army official called out the limitations of automated recognition technologies for the Army.  He said that when it comes to the use of force “You need somebody who’s trained and is accountable in recognizing that that is a woman, that is a child and that is someone who’s carrying a weapon.  And the best tools for that are still the eyeball and the human brain.”  But even if technology can identify the 10% of footage that needs to be analyzed by humans it would still make a big difference.

According to an article in the New York Times today, UK supermarket chain Tesco is taking on the most daunting of statistical endeavors – predicting the British weather.  “Rapidly changing weather can be a real challenge,” Jonathan Church, a Tesco spokesman, said in the article, describing the system Tesco has developed to predict the weather and the impact it has on demand. “The system successfully predicted temperature drops during July that led to a major increase in demand for soup, winter vegetables and cold-weather puddings.” 

Winter veggies in July … it almost makes me miss the summers in London.  Crap weather for weeks but that one day when the sun does come out makes it all worthwhile.  It‘ll have everyone running to the park in ecstasy to get their pale white skins roasted.  And literally everyone will have a BBQ.  According to Tesco a temperature increase of 18 degrees generally triples sales of barbecue meat and increases demand for lettuce by 50 percent.

If there’s one organization able to predict the unpredictable it will be Tesco.  They are the most famous example of a company that turns the data about their customers into a strategic asset.  The insights they get from their data help them manage approximately four thousand stores across 5 different formats in 13 countries.  Their ability to turn these insights into sales means they estimate the value of their database at $3.2bn.  God knows what it will be worth if it can also predict the weather!

A couple of weeks ago I wrote about the new York Times’ award winning visualization of movie ticket sales.  This is a similar graph displaying words that are included in Tweets about American Idol leading up to yesterday’s finale.  The Neoformix folks put a nice little Twitter Streamgraph app online so you can make these graphs yourself about anything you want.  Here’s one for the word doublethink. 

Colin’s post on 200 million faces showed a great data visualization example from the New York Times.  We have gotten used to seeing great visualization work from the newspaper’s visualization team which is at the leading edge of this rapidly developing discipline.  Two weeks ago this NYT visualization of box office receipts won best of show at the Malofiej awards, the equivalent of the Oscar for data visualizations.

One of the most interesting collaborations in this space is that of the NYT and IBM’s Many Eyes .  Many Eyes is a project that was born out of IBM’s Watson Research centre.  Its idea was to “democratize” data visualization and to enable a new social kind of data analysis.  The Many Eyes site gives anyone the ability to create visualizations using the site’s tools.  Here’s an example of one of the many visualizations people have done of Barack Obama’s inauguration speech.

You can either build visualizations using your own data or you can go to the NYT visualization lab website and play around with some of the data they have made available.  Last month, in this spirit of distributed journalism, the NYT opened up data from their entire archive to the public through API’s.  Shortly after that the Guardian in the UK did exactly the same.  This will make masses of data available to anyone who wants to play with it.  The combination of this democratization of data with open platforms like Many Eyes is bound to produce very exciting work.