Detect search engine robot on your website
For anyone who want to know what kind of visitor visiting your website, here is the code. Visitor is not just a human, many search engine use a robot, a special programmed dedicated computer used to browse information on webs. This this robot machine (or “bot” for short) have a duty to index or collecting information around the internet, and this robot also visiting your website!.
Each search engine robot has a name, we can find it with special PHP Variable $_SERVER['HTTP_USER_AGENT']. This variable keep information on visitor’s browser. Try creating simple php script and open it using different browsers such as Firefox, Internet Explorer, Safari, etc.
<?php echo $_SERVER['HTTP_USER_AGENT'] ; ?>
When you open it on your browser it will output something like this :
Mozilla/4.5 [en] (X11; U; Linux 2.2.9 i586)
or
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20
or
Opera/9.23 (Windows NT 5.1; U; en)
or
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16
The result is depend on what browser do you use. For a search engine bot there are many possibilities such as googlebot/Google for Google Search engine, msnbot for MSN / live.com, and many more. With this information we can detect which type of visitor visiting our website. so, here’s the code inside a function, so we can easily call in another part of script.
Detect robot name
<?php
/** spider robot detector return spider name, or else return false **/
function crawlerDetect($USER_AGENT) {
$crawlers = array(
array('Google', 'Google'),
array('msnbot', 'MSN'),
array('Rambler', 'Rambler'),
array('Yahoo', 'Yahoo'),
array('AbachoBOT', 'AbachoBOT'),
array('accoona', 'Accoona'),
array('AcoiRobot', 'AcoiRobot'),
array('ASPSeek', 'ASPSeek'),
array('CrocCrawler', 'CrocCrawler'),
array('Dumbot', 'Dumbot'),
array('FAST-WebCrawler', 'FAST-WebCrawler'),
array('GeonaBot', 'GeonaBot'),
array('Gigabot', 'Gigabot'),
array('Lycos', 'Lycos spider'),
array('MSRBOT', 'MSRBOT'),
array('Scooter', 'Altavista robot'),
array('AltaVista', 'Altavista robot'),
array('IDBot', 'ID-Search Bot'),
array('eStyle', 'eStyle Bot'),
array('Scrubby', 'Scrubby robot')
);
foreach ($crawlers as $c) {
if (stristr($USER_AGENT, $c[0])) {
return($c[1]);
}
}
return false;
}
$whatCrawler = crawlerDetect($_SERVER['HTTP_USER_AGENT']);
if ($whatCrawler) {
echo "Robot name : ".$whatCrawler;
}else{
echo "Sorry, not listed as robot.";
}
?>
That’s it, with this function you can include and call this function in any part of your code.
If you know more search engine name not listed here, you can add on the array variables, and please let me know.
Related posts:
- Creating user online program with php
- Why Search Engine Optimization (SEO) Matters
- A Blog and a Website: This is How We Do It
This post was extremely interesting, especially since I was searching for thoughts on this subject last week.
[Reply]