How To Combat Bad Bots Print
Written by Administrator   
Wednesday, 04 July 2012 23:18

Today is Sunday, October 25, 2026

IP Addresses are the key to tracking down spammers and bad bots to your website.

Every computer on the internet is represented by a numerical address called an "IP Address". The IP address consits of four sets of numbers, each separated by a "dot". Here is an example: "201.10.5.101". The example happens to belong to a company in Brazil. Each set of numbers can range from 0 - 255, giving 256 combinations for each set, with a total of almost 4.3 billion combinations for all four groups. A number of possible IP Addresses are reserved for local use on your own computer, so actually not all of the combinations are used on the internet.

It is possible to find out information for the company to which an IP address is assigned. You do this by querying a "Regional Internet Registry whois database".

Because Internet addresses must be unique and because address space on the Internet is becoming limited, there was a need for some organization to control and allocate address number blocks. IP address management was formerly a responsibility of the Internet Assigned Numbers Authority (IANA), which contracted with Network Solutions Inc. for the actual services.

In December 1997, IANA turned this responsibility over to ARIN, which, along with Reseaux IP Europeens (RIPE), Asia Pacific Network Information Center (APNIC), Latin American and Caribbean IP Address Regional Registry (LACNIC) and African Network Information Center (AfriNIC), now manages the world's Internet address assignments and allocations. The management of domain names is still the separate responsibility of Network Solutions and a number of other registrars accredited by the Internet Corporation for Assigned Names and Numbers (ICANN).

We see now that the information about any IP or IP Range is stored in the Regional Internet Registry. As mentioned previously, the 5 main registries are :

AFRINIC: Africa
APNIC: Asia and the western Pacific (India , China , Australia)
ARIN: North America and parts of the Caribbean (USA and Canada)
LACNIC: Latin America and parts of the Caribbean (Mexico and South America continent)
RIPE: Europe, Parts of Asia and the Middle East (Greenland , Russia , Europe and middle east)

That basically covers the whole world. For a visual representation check :



The first question you may have is how do I find out the IP address of any bots visiting my website?
This can be done by viewing your website access logs.
You can also add visitor IP detection to your web page.
Subscribers to Schembrionics can download a free script that will track visitor IP addresses and also monitor how may times these visitors or bots visit your site.

An other question you might have is "how do I know which Internet Registry to query?" That's a good question, considering that if you don't know where an IP address is located in the first place, then you don't know which continent/registry it will be listed under. The good news is that if you query the wrong database, you will be told which database to query.

 




How To Query An IP Address

Subscribers to Schembrionics can go to the "IP/Location/Speed/Weather" menu and click on the "Find ISP Provider For Any IP Address" link to query any IP address.

To find the allocation information for a specific IP address, the query is done first using the ARIN whois server. If the IP address is allocated through any of the other registries this information will be reported by ARIN. Another lookup will then be required to locate the actual data.

It is important to remember that although a bad bot might have a single IP address, this may be only one in range of IPs.  This means that although you may block the IP address, the bad bot can use other IPs to access your site.

Let us look at 180.76.0.12

What we do first is go to the Find ISP link, enter the IP address and Submit to get the following:





We see that the registrar for 180.76.0.12 is APNIC.

We thus need to query the APNIC database to get more information by clicking on the "Click Here For More Data" link to get the following(relevant information shown only):

 

We now know that 180.76.0.12 is actually part of the allocated range 180.76.0.0 to 180.75.255.255

This corresponds to a CIDR of 180.76.0.0/16

We also see that the IP range belongs to Beijing Baidu Netcom Science and Technology Co., Ltd., located in China.

We are therefore looking at the Baiduspider bot.

 

Let us now look at IP address 208.115.113.80

Using our Find ISP link we get:

 

We see from above that 208.115.113.80 is part of ARIN.

The ISP is dotnetdotcom.org located in the US.

The associated IP range is 208.115.113.80 - 208.115.113.95

The corresponding CIDR is 208.115.113.80/28

If you Google the IP address you will see that it is the Ezooms bot.

 

Lastly let' look at IP address 77.75.76.115

Using our Find ISP link we get:

 

 

 

We see that the registrar is RIPE.

However, in this case we are also given more data on 77.75.76.115

We see that the owner is Seznam located in the Czech Republic

The allocated IP range is 77.75.76.0 - 77.75.76.255

The corresponding CIDR is 77.75.76.0/24

We are therefore looking at the SeznamBot.

 

 


 

How To Block Bad Bots or Any IP Address

 

Now we know who our bad bots are, where they are from and their asooicated IP ranges.

We now need to deny them access to our website.

 

One way to ban IPs to our website is by using the .htaccess file located in your root directory.

The .htaccess file is a hidden file and so the only way you can get access to it is to FTP to your website.

 

Most websites have the .htaccess file.

If you do not, then Google it to learn more.

 

To ban the three bots that we found, we need to add the following text to the .htaccess file:

 

 

<Limit GET POST HEAD>

Order Allow,Deny

Allow from all

deny from 180.76.0.0/16

deny from 208.115.113.80/28

deny from 77.75.76.0/24

</Limit>

 

You will notice that we specified the CIDR value.

If we did not do this, we would have to deny each IP in the IP range.

You can learn more about CIDR by clicking on the "Understanding CIDR" link in the same menu box.

 

 

On a final note, always analyze your results before you ban any IP address. Sometimes you may be being attacked by a Zombie Bot.

 

In this case, the address appears to be one that’s used for normal DSL customers, but we can guess that the machine that sent us the spam was an open proxy, an innocent home user’s computer that was subverted into a “spam pump” through the use of malware. We can hope that the ISP provider will soon contact its customer to get the situation cleared up.

 

 

Last Updated on Thursday, 05 July 2012 08:37