Thursday - January 30th, 2003

RSS Aggregator Referer Problem

I just want to say, right on! The referer field is being abused by just about every RSS aggregator. Instead of Referer, they should be using the User-Agent request-header field to identify themselves. My guess is that they also want to show up in referrer logs for marketing reasons...because they're more likely to be read by site authors. I block them specifically from my referrer reports because they're meaningless. (Has anyone compiled a comprehensive list?) But I shouldn't have to.

posted at 2:34 PM

Comments

Hey pb, I posted a followup to Jason's post here: http://www.jayallen.org/journey/2003/01/index#using_and_abusing_the_referrer_field

I also sent an email with both links to Brent Simmon's (at Ranchero, maker of NetNewsWire) to see what he thought.

We'll see what the next version holds...

posted by Jay Allen on 1/31/2003 at 12:21 AM

Thanks Jay! (Though your URL is messing with my design. ;)

posted by pb on 1/31/2003 at 12:29 PM [PGP]

I like what Straw (an aggregator for Linux) does - you can set a URL to be included in the referer, which basically lets people know "hey, I'm reading your feed". It is still referer-abuse, but it's at least a bit more informative and useful to the site owner.

posted by Bill Stilwell on 1/31/2003 at 12:59 PM

Oops. Sorry pb. Guess I should make shorter titles. ;-)

posted by Jay Allen on 2/3/2003 at 1:53 AM

hey! progress:

http://inessential.com/?comments=1&postid=2357

posted by pb on 2/4/2003 at 10:57 AM

Commenting Is Closed

This post is closed to new comments, thanks contributors!