Googlebot and phpiCalendar

I installed php iCalendar a few weeks ago, it was really nice being able to publish some of my iCal files from Sunbird on the web, and students could see whan I was free without me having to print out a paper calendar every week and stick it on my door.

I got a message today though, for some reason the system network usage had shot up, and my web server was consuming something like 6.4MB/s.
It was traced to Googlebot indexing my calendar file, as there are so many links, and technically the calendar can keep going forever, the bot had got a little out of control.

I have now put a robots.txt file in the correct place, but I’m still not sure that it’s going to solve the problem – it’s still generating a request every second or so, and eating bandwidth.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.