« Umberto Eco on ComputersAmong The Biogeeks Of San Diego »

Subscribing to the LazyWeb search engine

Richard Soderberg came up with a great idea today for the LazyWeb search engine - why not offer an RSS feed for each search, so that subscribers can monitor the results over time?

Why not indeed? It was a bit of genius that turned out to be easy to implement. Now every search query on LazyWeb gives you a little orange XML button that you can click on to get an RSS feed. Just paste the link into your aggregator, and when the search results change, you'll find out about it.

That's one reason vector-model search engines are good: you can actually do things like offer an RSS feed without making your server catch fire. Vector engines keep all their search data in memory, so there's no need to read every entry from a disk every time you run a search. You only have to retrieve the documents that are in the result set, and those can even be served from a different machine.

Being able to subscribe to individual queries is cool. Now what we need is the ability to combine many different queries, to different places, into one integrated list. I should be able to subscribe to the same search query on LazyWeb, and Mark Pilgrim's blog, and Bruce Schneier's security newsletter, and get the results back in a meaningful ranked list that updates whenever the results change. That's actually not hard to do on the search side, but there's no room in the RSS protocol for the kind of handshake I need. And of course aggregators don't support distributed search (yet).

If anyone who knows about designing protocols can spare the brainpower, I could use some instruction.

« Umberto Eco on ComputersAmong The Biogeeks Of San Diego »

Greatest Hits

The Alameda-Weehawken Burrito Tunnel
The story of America's most awesome infrastructure project.

Argentina on Two Steaks A Day
Eating the happiest cows in the world

Scott and Scurvy
Why did 19th century explorers forget the simple cure for scurvy?

No Evidence of Disease
A cancer story with an unfortunate complication.

Controlled Tango Into Terrain
Trying to learn how to dance in Argentina

Dabblers and Blowhards
Calling out Paul Graham for a silly essay about painting

Attacked By Thugs
Warsaw police hijinks

Dating Without Kundera
Practical alternatives to the Slavic Dave Matthews

A Rocket To Nowhere
A Space Shuttle rant

Best Practices For Time Travelers
The story of John Titor, visitor from the future

100 Years Of Turbulence
The Wright Brothers and the harmful effects of patent law

Every Damn Thing

2020 Mar Apr Jun Aug Sep Oct
2019 May Jun Jul Aug Dec
2018 Oct Nov Dec
2017 Feb Sep
2016 May Oct
2015 May Jul Nov
2014 Jul Aug
2013 Feb Dec
2012 Feb Sep Nov Dec
2011 Aug
2010 Mar May Jun Jul
2009 Jan Feb Mar Apr May Jun Jul Aug Sep
2008 Jan Apr May Aug Nov
2007 Jan Mar Apr May Jul Dec
2006 Feb Mar Apr May Jun Jul Aug Sep Oct Nov
2005 Jan Feb Mar Apr Jul Aug Sep Oct Nov Dec
2004 Jan Feb Mar Apr May Jun Jul Aug Oct Nov Dec
2003 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2002 May Jun Jul Aug Sep Oct Nov Dec

Your Host

Maciej Cegłowski


Please ask permission before reprinting full-text posts or I will crush you.