Anil on the Microcontent Client

Anil has a interesting article in his Magazine about a notional piece of software he calls a 'microcontent client'. It sounds something like a personal search engine I have been thinking about -- it also has some overlap with what opencola is doing. But I can't help but feel a little frustration when I read pieces like his. Let's say, for the sake of argument, that you were a handsome young devil of a man who had been taught an elegant way to make homebrew search engines, in Perl, that returned four times as many relevant results as a traditional keyword search engine, returned results faster, returned relevant results that didn't even contain the keyword, and gave you features that have not been seen before on the public Internet. Features like being able to paste in an entire article as your query, or do real similarity searches, or say 'find me documents that are like this set of six documents I've saved'. Furthermore, let's say you created a demo of the technology to get bloggers excited, shopped it around to whomever you could think of, and got your superiors to agree to release it as open source code. You built it, and nobody came.

I'm sitting on a frigging microcontent client. I don't have to ask anyone to mark up their data, I don't need no stinkin' semantic web, I don't need keywords, metadata, standards, controlled vocabularies, summaries, Dublin Core, attributes, descriptors, or magic pixie dust. Just the raw text, thank you, and from that text I can generate a very good search engine, with high tolerance for synonyms, hyponyms, all kinds of nyms. And I need some people who will actually collaborate on making it happen. A little Napster of search engines ( Steven Johnson came up with that ), where everyone has a bunch of files they make available for searching, and you can send out queries over the Internet, hit a thousand little search engines, and get ranked, ready results back. A Google on your desktop, searching your own documents, as well as those of anyone else who cares to share. Is there not one search engine Perl geek among you?