Traffic boost

18 Jul 2004

Well, sort of.  My RSS feed attracted a fairly sizeable number of hits in the last 24 hours, compared to the rest of my website ;).  Hardly unique, of course, but still.  It’s been live for all of a day, and has attracted about 1/5th of the traffic IN A DAY that the rest of my site has accrued in a month.  That’s hits speaking, not bandwidth, but still!  Mildly insane.

I can now see why larger sites with RSS feeds request you keep your updates to an hourly timing cycle or whatever!  Not that I want anyone to do that… I don’t really care, and I’d imagine that, at 3KB a pageload, Dale’s connection doesn’t really, either.

So, when I was looking at my logs, trying to figure out what kind of traffic impact the RSS had had, I was looking at the clients people were using to access RSS.  Awstats doesn’t identify more obscure User-Agent strings (despite reputedly having 97 browsers which it supports), so I looked under “Unknown”.  Which was cool.  A viewer called “Soup” came out on top, then KNewsTicker (I run that one in my panel, and updates are set to a meagre 5 minutes!  Should probably set that value a little higher…), then FeedValidator.

Which is all well and good.  But then I see another one in there.  “Mediapartners-Google/2.1” – a few searches later, it’s revealed that this bot is used to spider websites using AdSense…  gee, thanks guys.  Nice to know that you think my measly 200 unique impressions a month is worth advertising to, really…

Okay, so now I’m really curious.  Why am I being spidered for a service I’m not even advertising with?!  Seems kind of stupid to me.  Seeing this hosting isn’t costing me anything, I’ve got no reason to try and recoup costs on it… and really, that’d be all I could ever hope to achieve with Google Advertising, unless I saw a few hundred-fold increase in traffic.  Which, of course, would probably mean I’d need to start paying for hosting somewhere with a nice amount of bandwidth and resources (got to love working for a webhost, at-cost hosting is a nice perk)… that said, it’d make more sense to purchase a TLD first.  No, a frame-based redirect from some pacific island doesn’t count, as cool as Tokelau must be.  At any rate, that kind of traffic seems to be a few months away yet. (Yeah, web expansion is the one area I’m an optimist… it WILL be months, not years ;) )

Has anyone else who is a bit pedantic about their log-checking noticed anything like this?  I don’t care if you’re using AdWords on your site – that’s justifiable – but if you’re NOT, and still getting this weird spidering… yeah, if you’ve got any ideas, let me know.