Blogs And Google

I'm knee-deep in preparation for a slide talk tomorrow, but I can't resist posting a link to a pretty mind-numbing open letter to Dave Winer from a certain Daniel Brandt.

The gist of Brandt's letter is to ask Dave Winer to take down his blogroll, which (in combination with a zillion other blogrolls) is skewing Google's rankings to favor A-list bloggers. The reasoning (which is correct) is that Google's ranking algorithm tends to favor sites that are heavily linked to, and the net result of a bunch of widely-read bloggers linking to each other through a blogroll is to exaggerate their relevance in comparison with other sources of information.

I agree with this, and I even agree with many of Brandt's other criticisms of Google (on the Google Watch homepage), specifically the danger of having one centralized search engine collecting so much information about its users, or the risk that some data will become invisible to the Web because Google stops listing it, whether for political, technical, or other reasons.

But Brandt phrases his criticism in a curious way: "This practice [blogrolling] among the A-list bloggers is seriously skewing the Web ".

Where did he get that conclusion? If PageRank is doing a bad job at, well, ranking pages, that sounds like a problem for Google. The Web isn't being skewed - those blogroll links are part of the Web. It's like saying language is getting skewed because teenagers are making up their own slang.

The barrier to entry for building a Web search engine is not that high, and anyone who can build a better Google will quickly attract a lot of user traffic. I've spread my share of Google paranoia, but I've never claimed that they could get away with being a bad search engine.

Blogrolls are a natural phenomenon that has slowly evolved as a feature of many weblogs, and they carry a lot of information. They're also pretty easy to detect programatically (look for a really long list of links, most of them to top-level domains), and I suspect that Google will soon be giving them special treatment to reduce the 'blogger effect'. If they don't, someone else might see that as an opportunity.

But the idea that user behavior needs to change, rather than the algorithms that analyze it, is exactly wrong. Weblogs are pretty much the only Web-native format we've got, and they've developed all kinds of conventions and structure through natural selection, without any central authority. Blogrolls are a great example - I keep a blogroll to point my readers to sites I really enjoy. Other people keep blogrolls as a convenience to themselves. No one makes us do it. But collectively, blogrolls end up carrying a lot of information about the network as a whole. Hell, who's to say that at this very minute, someone isn't at work on a project to do something cool with that data?

The cult of Google exists because they found a way to use another emergent phenomenon - link authority - to make a great search engine. So something tells me not to worry so much about blogrolls.