Blocking unsupported requests on nginx

Recently I have been observing a lot of traffic of bots, crawlers and malicious users on a project I am working on. It is a rails application.

Lot of crawlers trying to access pages like /phpmyadmin/manager.php etc. All of them ending in .php. Similarly lot of requests ends in .cgi, .xml, .jsp.

My application does not support such request types. nginx transfers these requests to the rails router, and since they do not exist, it returns back an wroung route exception. I do not want to catch and monitor these exceptions. In fact I do not want to these requests to even reach the application server. I think unsupported request types should be handled at the level of web server i.e nginx.

And there is a way to do it. Make this entry in nginx.conf file, under the server block as:

location ~ \.(aspx|asp|php|jsp|cgi|xml)$ {
    return 410; 
}

It matches the unsupported the request types and returns a 410 status code.
410 basically means that the page you are looking for has been moved permanently to some other location and has no forwarding address.

The difference between 404 and 410 is that 404 means that page does not exist or has been moved temporarily/permanently. And 410 means that page has been moved permanently to a new location or has been removed permanently. And the main use case of 410 is when you want to inform the search engines that it might be the time to reindex as this routes no more exists. And also I could 410 for blocking unsupported request types as shown above.

If you want to read about more options available for nginx, read it here.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s