I've written a custom web advertisement server which
currently serves a couple million ad impressions per day. It's important that
this application only show certain kinds of ads to users who can support them
(for instance, Flash ads to users who have installed Flash), and that our
reporting only include non-spider user agents. These are fairly simple
requirements, but I also needed these calculations to be accurate, reliable,
and extremely fast in order to avoid degrading my site's performance.
Sample Usage - Detect Spiders
One of my primary requirements for this component was that
it quickly and accurately detects spiders (or crawlers) so that I could filter
these requests from my statistics. While I certainly could have written my own
user agent parsing engine to try and capture every known spider, new ones are
being written every day (usually stupid ones that don't even know about
robots.txt, but that's another issue), and I didn't really want to be saddled
with learning about every new spider agent that came along. I was pleased to
find that I could implement spider detection in my application with just two
lines of code:
Listing 1: Detect Spiders and Crawlers - C#
cyScape.BrowserHawk.BrowserObj browObj =
cyScape.BrowserHawk.BrowserObj.GetBrowser();
bool isCrawler = browObj.Crawler;
At this point, it's a simple matter to limit activity
logging to only non-crawler user agents, or to terminate the response
immediately if the resource in question is not something a crawler should be
looking at (for example, the stupid ones that ignore your robots.txt file).
But… is it fast? In short, yes. I did not run it through
any vigorous stress tests using thousands of different user agent strings, but
I did observe it using ASP.NET's Trace feature as well as run it in production
for several months serving upwards of 30 requests per second at times. I never
ran into any performance issues as a result of BrowserHawk.
(I can say the same for cyScape's CountryHawk,
but that is a review for another day.)
For anybody wondering why I didn't just use the built-in
ASP.NET Request.Browser.Crawler property, the answer is… I did. However, since
the list of browsers this uses is by default rather antiquated and no automatic
update service comes with it, I felt it would be best for my users if I
invested in a professional and more accurate solution, which BrowserHawk
provided. And indeed, there are a lot of user agents caught by BrowserHawk
that the default ASP.NET object did not flag as crawlers, so BrowserHawk is in
fact more effective.
Sample Usage - Detect Flash
This is covered in the documentation as well. There are two
properties in the ExtendedBrowserObj object that relate
to Flash: Plugin_Flash and Plugin_FlashVerEx.
Normally you'll simply make use of the former. Note that the ExtendedBrowserObj
must be instantiated early in the page life cycle, before anything has been
sent to the client browser (e.g. above your <head> and any Response.Write
statements -- typically at the top of Page_Load in ASP.NET apps).
Listing 2: Detect Flash - C#
<%
ExtendedOptions options = new ExtendedOptions();
options.AddProperties("Plugin_Flash");
ExtendedBrowserObj extBrow = BrowserObj.GetExtendedBrowser(options);
%>
<html>
Plugin_Flash: <% Response.Write(extBrow.Plugin_Flash); %>
</html>Note that the ExtendedBrowserObj is
only available in the Professional or Enterprise edition of BrowserHawk.
That's It?
For my production needs, the two properties above pretty
much exceeded my expectations. I had some issues due to my server
configuration that were unique to my setup, which caused me some headaches
implementing licensing, but cyScape's support was first-class, and they even
sent me an updated build within a day or so of my contacting them that
corrected the problem for my special case. Now, since this is a review, I
don't want to leave you thinking that BrowserHawk only has two useful features.
On the contrary -- I actually have quite a TODO list of features I would like
to implement with the help of BrowserHawk that will make my sites more user
friendly. If you'd like to see what BrowserHawk can tell you about your web
visitors, go to their Browser
Analysis Page. From there, you'll see everything from your current monitor
display settings to your browser plugins to your Internet connection speed.
