Waking up finding your site taken down for too many hits and too much bandwidth for weeks brought a mean introduction to Munax, Munax high performance. Munax the wannabee search engine whose manners mirror black hat attitude in a white hat world. Munax is a search engine that steals your bandwidth in huge chunks. Bypassing all protocols it will go right into the most private documents – all it needs is a link somewhere, in some email, blog, friends post no robots.txt will save it.
Check your logs for a user, that is right, a user with the IP numbers from 18.104.22.168 – 22.214.171.124 and you will find this user probably took several thousand gig of bandwidth trying to suck your site in. They disguise their robots as a web browser from a normal human being to get past the robots.txt and any exclusions you may put in there.
Why do they come in as users instead of robots.txt? Let me first put it in their own words:
“The reasons for this are: (a) Today, web servers are intelligent enough to react on the type of user agent. If our crawlers had a name, say MunaxRob or something like that, many web servers would not know about it and would return junk or maybe nothing at all. (b) We want the web server to return a page to us where the page looks as close as possible to a page that can be viewed with a standard web browser.”“It is true that many of today’s search engines are doing well with a name set to ”Robot” or something like that, but those search engines are well known and site owners have given the crawlers of those search engines a chance to retrieve the best possible information. We want to be given the same chance.”
They want that chance at your cost and with or without permission. They will go to severe black hat to get there!
To rub salt in the wound deeper these pirates have the nerve to post an outrageous demand that you send them a verified email from the domain, presumably to spam or sell to spammers as certified email address, telling them which urls you want removed. If you are like me with over 30k urls per site you do not want to hear these wads tell you how they are going to circumvent all the policies of the Internet to get the data it took Google, Yahoo and Microsoft 10 years to get at your cost.
In the bottom line we, the websites have to pay the bill for this terrible companies policies. We hope they rot in hell and have a download you can take with a sample .htaccess showing our list of bad apples.
Spread the word of warning on the black hat bandwidth hogs of the year MUNAX.
Here is a file that has all the deny statements you need to get rid of the vermin. After the IP”s for Munass you will find the ones for China, Korea, and Russia that may also add to your security and keep your site from getting spam.
These hogs have been going wild for quite a while