If you haven’t been careful when coding your sites, chances
are you are suffering from one (or more) of the above SEO problems.
Addressing these issues will improve your search engine relevancy ranking and
drive more traffic to your site.
The “good news” is that fixing the above 4 issues is really
easy using the URL Rewrite Extension. This is a completely free
Microsoft extension available for IIS 7.x (on Windows Server 2008, Windows
Server 2008 R2, Windows 7 and Windows Vista). The great thing about using
the IIS Rewrite extension is that it allows you to fix the above problems
*without* having to change any code within your applications.
You can easily install the URL
Rewrite Extension in under 3 minutes using the Microsoft Web Platform Installer (a free tool we ship that
automates setting up web servers and development machines). Just click
the green “Install Now” button on the URL
Rewrite Spotlight page to install it on your Windows Server 2008, Windows 7
or Windows Vista machine:
Once installed you’ll find that a new “URL Rewrite” icon is
available within the IIS 7 Admin Tool:
Double-clicking the icon will open up the URL Rewrite admin
panel – which will display the list of URL Rewrite rules configured for a
particular application or site:
Notice that our rewrite rule list above is currently empty
(which is the default when you first install the extension). We can click
the “Add Rule…” link button in the top-right of the panel to add and enable new
URL Rewriting logic for our site.
Scenario 1: Handling Default Document Scenarios
One of the SEO problems I discussed earlier in this post was
the scenario where the “default document” feature of IIS causes you to
inadvertently expose two URLs for the same content on your site. For
example:
http://scottgu.com/
http://scottgu.com/default.aspx
We can fix this by adding a new IIS Rewrite rule that
automatically redirects anyone who navigates to the second URL to instead go to
the first one. We will setup the HTTP redirect to be a “permanent
redirect” – which will indicate to search engines that they should follow the
redirect and use the new URL they are redirected to as the identifier of the
content they retrieve.
Let’s look at how we can create such a rule. We’ll
begin by clicking the “Add Rule” link in the screenshot above. This will
cause the below dialog to display:
We’ll select the “Blank Rule” template within the “Inbound
rules” section to create a new custom URL Rewriting rule. This will
display an empty pane like below:
Don’t worry – setting up the above rule is easy. The
following 4 steps explain how to do so:
Step 1: Name the Rule
Our first step will be to name the rule we are
creating. Naming it with a descriptive name will make it easier to find
and understand later. Let’s name this rule our “Default Document URL
Rewrite” rule:
Step 2: Setup the Regular Expression that Matches this
Rule
Our second step will be to specify a regular expression
filter that will cause this rule to execute when an incoming URL matches the
regex pattern. Don’t worry if you aren’t good with regular
expressions - I suck at them too. The trick is to know someone who is good at
them or copy/paste them from a web-site.
Below we are going to specify the following regular
expression as our pattern rule:
(.*?)/?Default\.aspx$
This pattern will match any URL string that ends with
Default.aspx. The "(.*?)" matches any preceding character zero or
more times. The "/?" part says to match the slash symbol zero or one
times. The "$" symbol at the end will ensure that the pattern will
only match strings that end with Default.aspx.
Combining all these regex elements allows this rule to work
not only for the root of your web site (e.g. http://scottgu.com/default.aspx) but
also for any application or subdirectory within the site (e.g. http://scottgu.com/photos/default.aspx.
Because the “ignore case” checkbox is selected it will match both
“Default.aspx” as well as “default.aspx” within the URL.
One nice feature built-into the rule editor is
a “Test pattern” button that you can click to bring up a dialog that allows you
to test out a few URLs with the rule you are configuring:
Above I've added a “products/default.aspx” URL
and clicked the “Test” button. This will give me immediate feedback on
whether the rule will execute for it.
Step 3: Setup a Permanent Redirect Action
We’ll then setup an action to occur when our
regular expression pattern matches the incoming URL:
In the dialog above I’ve changed the “Action Type” drop down
to be a “Redirect” action. The “Redirect Type” will be a HTTP 301
Permanent redirect – which means search engines will follow it.
I’ve also set the “Redirect URL” property to be:
{R:1}/
This indicates that we want to redirect the web client
requesting the original URL to a new URL that has the originally requested URL
path - minus the "Default.aspx" in it. For example, requests
for http://scottgu.com/default.aspx
will be redirected to http://scottgu.com/,
and requests for http://scottgu.com/photos/default.aspx
will be redirected to http://scottgu.com/photos/
The "{R:N}" regex construct, where N >= 0, is
called a regular expression back-reference and N is the back-reference index.
In the case of our pattern "(.*?)/?Default\.aspx$", if the input URL
is "products/Default.aspx" then {R:0} will contain
"products/Default.aspx" and {R:1} will contain
"products". We are going to use this {R:1}/ value to be the URL
we redirect users to.
Step 4: Apply and Save the Rule
Our final step is to click the “Apply” button in the top
right hand of the IIS admin tool – which will cause the tool to persist the URL
Rewrite rule into our application’s root web.config file (under a
<system.webServer/rewrite> configuration section):
<configuration>
<system.webServer>
<rewrite>
<rules>
<rule name="Default Document" stopProcessing="true">
<match url="(.*?)/?Default\.aspx$" />
<action type="Redirect" url="{R:1}/" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>Because IIS 7.x and ASP.NET share the same web.config files,
you can actually just copy/paste the above code into your web.config files
using Visual Studio and skip the need to run the admin tool entirely.
This also makes adding/deploying URL Rewrite rules with your ASP.NET
applications really easy.
Step 5: Try the Rule Out
Now that we’ve saved the rule, let’s try it out on our
site. Try the following two URLs on my site:
http://scottgu.com/
http://scottgu.com/default.aspx
Notice that the second URL automatically redirects to the
first one. Because it is a permanent redirect, search engines will follow
the URL and should update the page ranking of http://scottgu.com
to include links to http://scottgu.com/default.aspx
as well.
Scenario 2: Different URL Casing
Another common SEO problem I discussed earlier in this post
is that URLs are case sensitive to search engines on the web. This means
that search engines will treat the following links as two completely different
URLs:
http://scottgu.com/Albums.aspx
http://scottgu.com/albums.aspx
We can fix this by adding a new IIS Rewrite rule that
automatically redirects anyone who navigates to the first URL to instead go to
the second (all lower-case) one. Like before, we will setup the HTTP
redirect to be a “permanent redirect” – which will indicate to search engines
that they should follow the redirect and use the new URL they are redirected to
as the identifier of the content they retrieve.
To create such a rule we’ll click the “Add Rule” link in the
URL Rewrite admin tool again. This will cause the “Add Rule” dialog to
appear again:
Unlike the previous scenario (where we created a “Blank
Rule”), with this scenario we can take advantage of a built-in “Enforce
lowercase URLs” rule template. When we click the “ok” button we’ll see
the following dialog which asks us if we want to create a rule that enforces
the use of lowercase letters in URLs:
When we click the “Yes” button we’ll get a pre-written rule
that automatically performs a permanent redirect if an incoming URL has
upper-case characters in it – and automatically send users to a lower-case
version of the URL:
We can click the “Apply” button to use this rule “as-is” and
have it apply to all incoming URLs to our site.
Because my www.scottgu.com
site uses ASP.NET Web Forms, I’m going to make one small change to the rule we
generated above – which is to add a condition that will ensure that URLs to
ASP.NET’s built-in “WebResource.axd” handler are excluded from our
case-sensitivity URL Rewrite logic. URLs to the WebResource.axd handler
will only come from server-controls emitted from my pages – and will never be
linked to from external sites. While my site will continue to function
fine if we redirect these URLs to automatically be lower-case – doing so isn’t
necessary and will add an extra HTTP redirect to many of my pages.
The good news is that adding a condition that prevents my
URL Rewriting rule from happening with certain URLs is easy. We simply
need to expand the “Conditions” section of the form above
We can then click the “Add” button to add a condition
clause. This will bring up the “Add Condition” dialog:
Above I’ve entered {URL} as the Condition input – and said
that this rule should only execute if the URL does not match a regex pattern
which contains the string “WebResource.axd”. This will ensure that
WebResource.axd URLs to my site will be allowed to execute just fine without
having the URL be re-written to be all lower-case.
Note: If you have static resources (like references to .jpg,
.css, and .js files) within your site that currently use upper-case characters
you’ll probably want to add additional condition filter clauses so that URLs to
them also don’t get redirected to be lower-case (just add rules for patterns
like .jpg, .gif, .js, etc). Your site will continue to work fine if these
URLs get redirected to be lower case (meaning the site won’t break) – but it
will cause an extra HTTP redirect to happen on your site for URLs that don’t
need to be redirected for SEO reasons. So setting up a condition clause makes
sense to add.
When I click the “ok” button above and apply our lower-case
rewriting rule the admin tool will save the following additional rule to our
web.config file:
<configuration>
<system.webServer>
<rewrite>
<rules>
<rule name="Default Document" stopProcessing="true">
<match url="(.*?)/?Default\.aspx$" />
<action type="Redirect" url="{R:1}/" />
</rule>
<rule name="Lower Case URLs" stopProcessing="true">
<match url="[A-Z]" ignoreCase="false" />
<conditions logicalGrouping="MatchAll"
trackAllCaptures="false">
<add input="{URL}" pattern="WebResource.axd" negate="true" />
</conditions>
<action type="Redirect" url="{ToLower:{URL}}" />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration> 
