In mid December 2009, I upgraded my site to TikiWiki 3.3, resulting in an explosion of bandwidth consumption, see attached chart.
What I tried doing afterwards:
- Set "revisit-after: 60 days" in robots.txt
- Delete an image gallery with about 20 high resolution images
- Always update to the newest
Nothing has really helped to reduce the massive traffic waste. Any other ideas what I can do?
In February, traffic consumption was the following:
Viewed traffic: 858.46 MB
Not viewed traffic: 4.59 GB (!!!)
The overall traffic consumption by bots was the following:
- 55.53% Yahoo! Slurp
- 11.94% MJ12bot
- 11.57% NHN bot
- 2.45% Googlebot
Why are these four bots wasting my badwidth limit, and what can I do about it?
My site is located here: http://list.vic-fontaine.com/
Test user and password: smarty