Saturday, 2 February 2008

REVOLUTION: SEARCH, PAGERANK AND THE SOCIAL GRAPH

We are at the beginning of the next revolution in digital - so called web 2.0 is moving away - and the next big thing is starting to take shape. Don't get me wrong, web 2.0 was fun, but something far more serious is starting to take place. Two key forces are at play: advertising funded utility-based computing, e.g. software as a service; and the dovetailing of search and the social graph, resulting in a semantic web. I say this in the week that Microsoft bid for Yahoo to counter the growing dominance of Google.

To get to the heart of what's going on, its worth looking at the algorithm at the heart of Google's strength: PageRank. You'll find a detailed definition on Wikipedia for PageRank. The diagram below - from Wikipedia - shows a network of sites - using a mathamatical scale of 100 - are used to calculate a PageRank. It's essentially a measure of populairty within a network of sites, measuring the likelihood that a person randomly clicking on links will arrive at any particular page. Put simply B,C and E in the diagram have the highest PageRanks because the link flow points to them. And specifically, C is important because an important site - B - points to it. They are more important in the network. This is the wisdom of crowds at work.

If you have a Google Toolbar you'll see the PageRank score as a whole number between 0 and 10. The most popular websites have a PageRank of 10. The actual page on Wikipedia that defines PageRank scores 7/10. yahoo.com and facebook.com score 8/10, and google.com 10/10

However, we all know the most popular person in a commuinity is not always the most 'right' person in a community. I might be connected to a popular friend for very different reasons that you are connected to that friend, but we are both boosting his link popularity, and this friend will get more and more popular as more people connect to us and him - this is network effects. Now this is where the social graph comes in. When you connect with someone on Facebook you say what the nature of the relationship is. In other words you give meaning and context to that link. Search could become more accurate if we added context to links and categorisation to the content like we do on Facebook.

This could be the next step in search. Obviously, there will be other innovations, but when you add context to link popularity, and meaning to the content, then not only do search spiders such as Googlebot, find content more effectively, but the whole web becomes machine read/writable. That is machines will no longer dumbly search and find content without knowing what the content is about, but will instead be able to read it and automatically carry out tasks for you with that content.

Although there is more than a hint of hype behind the phrase social graph, it does name something new, and powerful, even the venerable Sir Tim Berners-Lee on his blog blessed the term social graph, and talked about the web evolving into a Giant Global Graph. This means we stop thinking about searching a web, and instead we search a graph. What will this graph finally look like? What will it allow us to do? I'd like to share the same answer to a tough question about this Revolution, that Zhou Enlai - Chinese Premier from 1949 until his death in 1976 - gave when asked for his assessment of the 1789 French Revolution: it's too early to say!