The first video is an interview with the fascinating Dr Carl Marci – the founder of neuroscience company Innerscope.  I wrote about him in a previous post but nothing beats listening to the man himself!

Take a look at this superb interactive tool developed by The New York Times:  A Peek into the Netflix Queue.

The principle is simple enough.  The Times has taken a database of most rented movies from Netflix and overlapped it on ZIP codes on a Google map to create a geodemographic look at tastes and the hyper local level.  New York is shown here, but they’ve featured many cities.

The magic is in the way you can look at the same data in many different aspects.  You can rollover the different ZIPs to see their top ten picks.  You can sort by movie, using a slider to highlight ZIPs in which that movie was the most or least viewed.  You can file through the movies differently:  alphabetically, by most rented, or by “Metascore” ( a score that averages scores given by the nation’s most influential critics, compiled by the very useful Metracritc.com.)

You could call this a prizm presentation.  There’s no dominant point of view.  You have to look at it from many different angles to fully understanding it.

As you might imagine, there are polarizing titles with distinct patterns, such as Mad Men, Obsessed and Last Chance Harvey.

Like Google Earth, this is, of course, most interesting when you’re looking at your own backyard.  My own ZIP (10001 or Chelsea) doesn’t disappoint.  Milk was the most popular movie.  In fact, its pattern looks like most of lower Manhattan.  (I’m proud to say how well I conform to type, having rented every title on the top ten list.)   My friend Charlie McKittrick points out that “Brooklyn is much more eclectic and long-tail in their taste than more main-stream, follow-the-leader zip type ZIP codes like Tribecca/Chelsea and Bergen County NJ, which seem to more consistently be renting off the top of the best seller list”.  Feisty!

So what makes this compelling is the splicing of geodemographic data with some believable measure of something super soft like taste.

It seems to me that this is not only the future of data presentation, but perhaps also the future of newspaper reporting.  I found this article only mildly interesting in the print edition.  It only really works online.  Could all stories (from Madoff to Britney) be told in this multidimensional way?  And would people pay for that?

On Sunday I wrote a small post about how UK company Prozone captures all English Premier League soccer matches on video and then uses that footage to create a database of every shot, pass, tackle and assist made by every player (according to a comment on that post Opta Sports are powering the Guardian Chalkboard – sorry I got that wrong).  That’s a lot of data.  However, as I found out in the New York Times this morning, this is nothing compared to the volume of footage captured by the US Airforce spy drones.  Apparently it would take one person 24 years to watch all the surveillance footage captured in Afghanistan in 2009.  According to the article, the Army has a group of young soldiers watching every second of this footage live in order to detect suspicious activities. 

The Army expects the volume of footage to grow exponentially in the near future as new drones are being deployed that can film at up to 65 different angles.  Processing all that footage will soon become unmanageable.  The Airforce top intelligence official LT. Gen. A. Deptula said the army could soon be “swimming in sensors and drowning in data”.  So they are looking for new analytical techniques that can help reduce the volume of footage that needs to be analyzed by the human eye.  Now where do you think they started their quest for this technology?  You got it – the sports networks. Apparently Army officials have been shadowing the ESPN broadcast trucks outside football stadiums to learn how the TV network is tagging and retrieving all the highlights in their NFL footage.  It sounds like Opta Sports and Prozone have just found a second market.

One Army official called out the limitations of automated recognition technologies for the Army.  He said that when it comes to the use of force “You need somebody who’s trained and is accountable in recognizing that that is a woman, that is a child and that is someone who’s carrying a weapon.  And the best tools for that are still the eyeball and the human brain.”  But even if technology can identify the 10% of footage that needs to be analyzed by humans it would still make a big difference.

Yesterday Landon Donovan tore up the right flank of the pitch at the Emirates Stadium and helped his new club Everton secure a draw against my beloved Arsenal.  This morning one of the soccer blogs I follow showed this great visualization of Donovan’s impact on the game. 

 by Guardian Chalkboards

The graph shows Donovan’s completed passes in blue.  If you roll across the dots you can see when he passed to which team mate.

The graph is built with the Guardian’s interactive chalkboard which has apparently been around for almost a year.  I have to say that I am ashamed that I only found out about this fantastic little tool today.  OK, I have been living in the US now for 6 years and I have started to refer to the beautiful game as soccer (mainly because I discovered the other beautiful game of American football).  And coverage of soccer in the US media is spotty at best.  But that should still not be an excuse for not knowing about this amazing interactive visualization tool.

It uses data collected by UK company Prozone who record all Premier League matches and use that footage to code all data on passes, shots, tackles etc for every player on the pitch.  The guardian then build this very easy to use application that allows every aspiring Arsene Wenger (who was one of the early Prozone adopters) to analyze the patterns of the game.  Here are a couple of great examples.

What is probably most impressive about this little application is the ease with which one can share chalkboards with others.  This makes it a potentially powerful social media tool.  And with most of the coverage of soccer in the US happening on social media, it could be a great tool to educate the US audience on the tactical aspects of the game.  Nothing gets America more excited about sports than stats and analysis!

CNN were the 1st network to use the Perceptive Pixel touchscreens to visualize data for the elections last year.  I just saw they have a new article on data visualization on their site.  They call it “the new way of looking at the world”.  The article discusses the new tools that are making it easier to see and analyze complex data.  It claims that cheaper technology and the availability of digital data are the main drivers of innovation in the space.  It also mentions the power of data visualization on mobile devices.  The article has the following examples, many of which we have discussed on thedoublethink :

Ben Fry 

Everyblock 

IBM’s Many Eyes 

Yelp’s Monocle

Citysense

Flowing Data

Data visualization is a fascinating and now apparently even newsworthy field.  It can give people access to data in an intuitive and user friendly way.  Some data visualizations can also be incredibly beautiful (like the visualization of the port traffic I described in a previous post).  It can also be extremely helpful when one needs to communicate insights derived from data to a non technical audience. 

One promise however I haven’t really seen it fulfill is the discovery of insights that could not have been found with traditional multivariate statistical techniques.  Though I agree it can help statisticians communicate their insights, I dont think it really helps them discover them.  On the other hand it does provide a much broader audience than just statisticians with data analytics tools.  This is probably the biggest potential for data visualization.  Having people with vastly different backgrounds mine data will almost definitely produce more and richer insights.

I just came across this beautiful visualization of vessel movements on sea on visualcomplexity.com today.  Not only is it a spectacular visualization that uses millions of datapoints, it is also very close to my heart as it shows shipping movement around the port of Antwerp, my hometown.

The graph below shows shipping movements around the nearby port of Rotterdam.  You can clearly see the shipping lanes that show traffic on sea is not an open playground as the authors say. 

The illustration below however shows shipping movements during bad weather.  This clearly makes it a lot harder for ships to stay on course.

This last picture gives the overview of the movements around Antwerp, Rotterdam and Amsterdam.  Another beauty showing the connectivity between the Netherlands and Belgium.  I might frame this one!

I had a very interesting conversation with David Huffaker last week.  David just finished his PhD in Media, Technology and Society at Northwestern University, which makes him an expert in social media analytics, an exciting relatively new field. 

David has analyzed behavior on social networks such as Google groups to get a better understanding of how influencers behave.  He defines influencers based on 3 criteria :

1. Does the content they create trigger replies?

2. Do these replies generate conversations?

3. Do the followers adopt the influencer’s language in these conversations.

The insights were not surprising but interesting nevertheless :

- Leaders are active – they are more likely to post messages, reply to others, and have a longer tenure in the community.

- Leaders are social – rely more on replying to others than merely on broadcasting.

- Leaders are passionate – they will demonstrate higher frequencies of talkativeness, affective language (use words such as good, bad, love, hate, …), assertiveness (use words such as always and never) and linguistic diversity.

He also analyzed the characteristics of the group.  He analyzed groups based on

- The size of the group

- The density of the group – how well connected are individuals in the group?

- The stability of the group – what is the group turnover?

- The participation equality – do group members participate equally or are there only a handful of members who account for the bulk of the activity?

The picture above is one of his group visualizations which show how dense the group is, where the influencers can be found (size of the bubble indicates degree of influence).  On the outside of the graph you can see how the group reaches out to new members in the outside world. 

His analysis showed that more nodes and connectivity results in more individual influence.  At the same time equality in participation and turnover don’t matter.

I asked David whether influencers were really that important.  One of my previous posts showed an interesting analysis on Facebook data that seemed to support Duncan Watts theory on the importance of group dynamics vs influencers.  David said that the jury was still out for him.  He said that much more empirical evidence was required to really validate or refute any of the theories on how social networks work.

That’s exactly what he will be doing next.  He is working on a post doc where he will analyze Second Life to get a better understanding of how groups behave (part of his research is funded by the military and the government).  According to David, Second Life is an interesting lab for studying group interactions because of the similarities with real life.  Second Life unlike other social networks has the notion of geography and “physical” spaces which could make the findings more applicable to real life.  I am looking forward to his findings.

MIT’s media lab created this interesting application that builds your persona based on data it finds about you online. 

It’s a very cool idea.  Problem is that I ran my name twice and got very different results.