Referer spam
From Wikipedia, the free encyclopedia
Referer spam is a kind of spamdexing (spamming aimed at search engines). The technique involves making repeated web site requests using a fake referer url pointing to a spam-advertised site. Sites that publicize their access logs, including referer statistics, will then end up linking to the spammer's site.
This benefits the spammer because of the free link, and also gives the spammer's site improved search engine link placement due to link-counting algorithms that search engines use.
Contents |
[edit] Technical solutions
As with e-mail spam, web site operators who receive unwanted referer spam may respond using filtering and blocking. Some web sites receive so many referer spam hits that they amount to a denial of service attack on the server because there are not enough resources left on the server to handle legitimate traffic.
[edit] Word-based filtering
An example configuration fragment for filtering using the Apache server is as follows:
# Filter rules # The regexp can be refined to reduce false positives... # As many SetEnvIfNoCase directives can be used... SetEnvIfNoCase Referer "(hold-?em|poker|casino|hotel|loan|mortgage|payday|credit)" refspam SetEnvIfNoCase Referer "(viagra|cialis|penis|diet|porn)" refspam # Whitelists can be used, too... note the !refspam vs. refspam SetEnvIfNoCase Referer "white-listed_site\.com" !refspam # Deny access to refspam... Deny from env=refspam # For cleaniness, we'll separate the logs CustomLog /var/log/apache/access.log combined env=!refspam CustomLog /var/log/apache/access_refspam.log combined env=refspam
The "fake" web site hits will go to access_refspam.log, whereas normal traffic goes to access.log. The "SetEnvIfNoCase" lines contain Regular expressions (more specifically, Perl regular expressions) that can be used to match any undesirable traffic.
[edit] IP-based filtering
If most of the spam is coming from a few IP addresses, or is requesting a certain page (that may no longer exist on the server) the Apache server may also be configured to deny access via the configuration file, (often named httpd.conf), based on either IP address or the name of the requested file by adding lines like the following:
# Deny access based on the filename or path of the requested file <Location /links> Deny from all </Location> # Deny access based on the IP address or host name of the offending site <Directory /usr/local/etc/httpd/htdocs> Deny from 72.36.244.166 Deny from big-bad-spammer-blah-blah.com </Directory>
A good statistics analysis program will allow you to target the worst offenders.
[edit] Advanced filtering using mod_security
A third solution for Apache is to install ModSecurity, which allows you to deny requests based on any variable from the server environment, such as referer, request, IP, host, etc.
[edit] See also
[edit] External links
- A script for filtering referer spam with Apache web server
- ReferrerCop - A searchable database of known referrer spam with tools for filtering referrer spam from Apache log files and AWStats data files.
- A proposal on addressing referrer spam - technical overview and ideas for combatting the practice
- ModSecurity (mod_security) homepage
| This article is part of the Spamming series. | |
|---|---|
| E-mail spam | DNSBL | Spamhaus | Stopping e-mail abuse | Spambot Address munging | E-mail authentication | Directory Harvest Attack |
| Spamdexing |
Google bomb | Keyword stuffing | Cloaking | Link farm | Web ring Referer spam | Blog spam | Spam blogs | Sping | Scraper site |
| Telemarketing | Autodialer | Mobile phone spam | VoIP spam |
| Scams | Phishing | Advance fee fraud | Lottery scam | Make money fast | Pump and dump |
| Misc. | Messaging spam | Newsgroup spam | Flyposting History of spamming |
