What is user agent
A
user agent (ua) is the
client application used with a network protocol (example HTTP),
in biggest part of cases the HTTP_USER_AGENT phrase is linked with World Wide Web.
Web user agents range from web browsers and e-mail clients to search engine crawlers ("spiders"),
as well as mobile phones, screen readers and braille browsers used by people with disabilities, and scripts that sending some manual ua data.
When Internet users visit a web site, a text string is generally sent to identify the user agent to the server.
Don't forget that sent
user agent data can be not real, because client application can send that it want, and
nobody control it.
This forms part of the HTTP request, prefixed with User-Agent: (case does not matter) and typically includes information such as the application name, version, host operating system, and language.
Bots, such as web crawlers, often also include a URL and/or e-mail address so that the webmaster can contact the operator of the bot.
Lots of Web Browsers support replacing of ua string, for example Konqueror can send ua string of Google Bot.
Different web browsers (Internet Explorer, Firefox, Opera, Safari,etc.) would therefore identify themselves with different user agent strings.
Search engines often using
ua for identify self, for example -
Google [Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)],
Yahoo [Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)].
This is how web site can know is visitor human or bot.
User agents consist of 6 parts
- Application name and version
- Browser type[2]
- The operation system[3]
- Any extensions installed with browser/system[4]
- "compatible" shows that this browser works correctly and conmptable with [2],[3],[4]
- Some installed soft and OS version.
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322) SeoTool/1.0023
Mozilla/4.0 ( | compatible; | MSIE 6.0; | Windows NT 5.1; | .NET CLR 1.1.4322 | ) SeoTool/1.0023 |
app name/version | "compatible" | browser type | OS | extensions | installed soft OS version |
User agent string - it's string that sending browser/application during request (as usualy called "user agent").
The user-agent string is one of the criteria by which crawlers can be excluded from certain pages or parts of a website using the "Robots Exclusion Standard" (robots.txt).
This allows webmasters who feel that certain parts of their website should not be included in the data gathered by a particular crawler,
or that a particular crawler is using up too much bandwidth, to request that crawler not to visit those pages.