« Piotrek Sus is a HoaxGuar Gum »
06.17.2003

Cormac Twomey

Cormac Twomey is a nice guy. I met him at this year's Emerging Technology conference, and we geeked out on search engines several times. He and his company are big believers in RDF, they have a nice demo search engine of medical data, they don't seem to be evil. Cormac is a compact and friendly guy, you wouldn't think twice about introducing him to the parents.

Yet today I find Cormac pilloried at Mark Pilgrim's website, for the sin of ignoring a robots.txt file and repeatedly downloading image files. Having recently written my first crawler, I'm shaking in my boots a bit at the idea of this kind of vigilante justice. It's very easy to make programming mistakes, or lazy errors in judgement, when testing out a crawler. I was mortified to find that my own crawler had been re-indexing sites that gave an error, over and over again, but fortunately none of those sites was as trigger-happy as Dive Into Mark, and I was able to fix it without scandal.

Mark's response seems disproportionate. Like shooting at the kids who try and climb into his plum tree.

But Cormac - a warning. Try swiping bandwidth from Idle Words, and I'm getting Orrin Hatch to destroy your computer.

« Piotrek Sus is a HoaxGuar Gum »

Greatest Hits

The Alameda-Weehawken Burrito Tunnel
The story of America's most awesome infrastructure project.

Argentina on Two Steaks A Day
Eating the happiest cows in the world

Scott and Scurvy
Why did 19th century explorers forget the simple cure for scurvy?

No Evidence of Disease
A cancer story with an unfortunate complication.

Controlled Tango Into Terrain
Trying to learn how to dance in Argentina

Dabblers and Blowhards
Calling out Paul Graham for a silly essay about painting

Attacked By Thugs
Warsaw police hijinks

Dating Without Kundera
Practical alternatives to the Slavic Dave Matthews

A Rocket To Nowhere
A Space Shuttle rant

Best Practices For Time Travelers
The story of John Titor, visitor from the future

100 Years Of Turbulence
The Wright Brothers and the harmful effects of patent law

Every Damn Thing

2020 Mar Apr Jun Aug Sep Oct
2019 May Jun Jul Aug Dec
2018 Oct Nov Dec
2017 Feb Sep
2016 May Oct
2015 May Jul Nov
2014 Jul Aug
2013 Feb Dec
2012 Feb Sep Nov Dec
2011 Aug
2010 Mar May Jun Jul
2009 Jan Feb Mar Apr May Jun Jul Aug Sep
2008 Jan Apr May Aug Nov
2007 Jan Mar Apr May Jul Dec
2006 Feb Mar Apr May Jun Jul Aug Sep Oct Nov
2005 Jan Feb Mar Apr Jul Aug Sep Oct Nov Dec
2004 Jan Feb Mar Apr May Jun Jul Aug Oct Nov Dec
2003 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2002 May Jun Jul Aug Sep Oct Nov Dec

Your Host

Maciej Cegłowski


Threat

Please ask permission before reprinting full-text posts or I will crush you.