SortFix is basically a value added search provider who wraps Google search results. Their approach is to analyze other keywords which appear frequently in your search results. You can then drag these other high frequency keywords to either the “Add to search” box, or the “Remove” box. There’s also a “Dictionary” box which defines keywords you may not know.

For example, if you search for “RS6“, you’ll get “Power words” like “v10″, “performance”, “audi”, “carlos”, 2010″, “2003″, “juan”, etc… By adding or removing those keywords, you can tune your search for either the 2003 edition of the RS6, or the new 2010 one, or you can check out the king of Spain, Juan Carlos’, ride.

I can see this being useful for people who don’t have super good Google-Fu. I don’t see myself using it, but I can see it being useful for many other folks. Another point against it is that currently it’s a Flash based interface, and generally I avoid Flash as much as possible. Apparently they are working on a non-Flash version, which would be a nice improvement IMHO.

I really like the idea the idea of offering up high frequency additional keywords to people who are searching for things to help them refine their search. I can see this being very useful for onsite eCommerce searching, helping narrow down products based on common attributes, etc…

Share on Facebook

Other than using lynx to test your site, there are several other tools I would recommend utilizing in your quest for better SEO. One great tool is the SEO Analysis Tool from SEOWorkers.com. This tool quickly analyzes the page, and provides excellent reports on the size and relevancy of your meta tags. It also provides data on the frequency of keywords, singly and in groups of two and three. Here is a part of the a report for sparkred.com:

I’ve already mentioned utilizing Google Analytics to monitor what keywords are bringing you good traffic versus bad. I’d also recommend another Google tool, Google Webmaster Tools. Google provides a TON of useful data here. You can see things like the crawl frequency, page response times during the crawl, and page rank of your pages (including a by month breakdown). You can also use quick links to see related pages, pages which link in to your site, all the indexed pages of your site, and more. It also gives you diagnostics into any problems with the crawl or your meta tags.

How to Promote Your Site

In order to drive traffic to your site and to increase your ranking in search engines, such as your Google Page Rank, you want high quality links pointing to your site. By high quality links I don’t mean spam links, link farms, or the like. I mean links on relevant sites pointing your site, driving high quality traffic.

There are two halves to this. The first is to provide really good quality content on your site. This means having a great site, it means updating your site frequently, and it means providing non-core but helpful related content. An example of that might be having a blog that provides related content. For instance for MyShoeStore.com you might have a blog with posts about the latest fashions and how to match shoes to each look, the best sales you’re currently running, the 10 most extreme shoes of all time, photos of the shoes worn by celebrities that week, etc… You want something that will have people returning to visit your site more frequently, and something that will make other people link to your site in their blogs, tweets, Facebook pages, IMs to friends, and so forth. If you have a blog post about the shoes on the red carpet at the People’s Choice awards, people will link to that from tons of forums discussing fashion. Those a great links! Relevant high quality links bringing in high quality traffic. And you just need a blogger. A college fashionista who wants to get into fashion journalism will do it for cheap.

The second is to promote your site on other sites. Find forums and communities that are filled with the people you want to attract to your site, signup, and contribute to the community conversation. Note I didn’t say spam the community. Post useful stuff. Even stuff that links to sites that aren’t yours. Be a real contributing helpful member of the site. In your site profile, have a link to your site, and when it’s relevant feel free to link to your site, your latest sale, whatever, as long as it’s relevant. Obviously if you’re an ATG developer or a VP of IT, it’s probably not you doing all this, but have someone do it.

Those two things are how you get traffic and a high page ranking in the search engines. It’s all about having good content and being an upstanding citizen of the relevant online communities.

Share on Facebook

By making your URLs expressive and relevant to the content and structure of the site, you help not only your search engine ranking but also your users, since they can easily tell what a given link will take them to.

This is a bad URL:

http://myshoestore.com/app/cat/browse.jsp?id=245345&cat=234523

This is a good URL:

http://myshoestore.com/shop/mens-shoes/fluevog/size-12

It is chock full of descriptive words. The page allows you to “shop” for “mens shoes”, more specifically “fluevogs” in “size 12″. This makes it much easier for search engines to know the purpose of the page and also for users to know what a link will take them to.

In order to accomplish this, you should name your directories and pages as accurately and descriptively as possible. You should also structure your site’s content and URLs in a logical hierarchical fashion.

Now your site may have a single actual JSP that handles displaying a category, and another one that handles displaying a product, any product. So you need to map the URL http://myshoestore.com/shop/mens-shoes/fluevog/size-12 to actually serve up the content from http://myshoestore.com/app/cat/browse.jsp?id=245345&cat=234523.

Depending on your technology there are different ways to do this.

If you’re using JBoss Seam it’s very easy to use rewrite patterns in the pages.xml mapping file. This not only handles mapping the incoming requests for pretty URLs to the actual resources on the backend, but also handles generating the pretty URLs within the site automatically, which is a huge time saver.

If you’re using Apache you can use mod_rewrite to translate the pretty requested URLs to the ugly actual URLs. Of course in that case you need to ensure you’re generating the correct pretty URLs on the pages of your site.

If you’re using ATG you should read the chapter of the ATG Programmers Guide titled Search Engine Optimization (chapter 10 for ATG 2006.3). This covers the ATG support for URL Templates and the Jump Servlet. A few of the downsides to be aware of, are that it’s not super simple to set up, and that it only displays the pretty URLs to search engines, not to all users. I really prefer solutions that give users the benefits of readable URLs as well. The out of the box ATG system has too much of a performance impact to use for all situations.

We’ll be releasing a high performance open source solution for URL re-writing in ATG eCommerce applications as part of the Open Source Foundation ATG eCommerce Framework in the near future.

Know Your Limits

Search engine spiders, like the GoogleBot, have limits as to what they’ll parse and consider. For instance the GoogleBot will only read in the first ~101kb of your page’s HTML. Anything after that is ignored. So you need to ensure that your pages are smaller than 101kb. This is also a best practice with regards to performance: keep your HTML as small as possible.

Search engines will often display a small chunk of text with the search results, usually this is taken from the page’s description meta tag. Most will only show the first 160 characters of the description, so you want to be sure that your description content is less than 160 characters and makes sense for a human to read.

Many search engines will ignore, or penalize you, for having more than 100 links on a given page. Keep the number of links on a single page to a reasonable level. If your primary navigation must have more than 100 links, you can load in the second, third, etc… level navigation via AJAX/Javascript.. This will let your users have access to the full navigation structure from any page, but keeps things more reasonable for the search engine crawler. You’ll want to be sure that the crawler will be able to traverse through the complete site structure using the more limited navigational options it can see, the non-AJAX navigation.

Share on Facebook

Use the right tags for the job. For high importance headings use <h1>, for lists use <li>, and so on. Don’t JUST use CSS classes for styling, make sure you’re also using the appropriate HTML markup for your content. By using semantic markup you can identify the importance and structure of the content and data on the page in a way that not only assists search engine crawlers, like GoogleBot, understand your content better, but also supports alternate browsers/web clients such as screen readers, other accessibility tools, and future applications.

Accessibility

By designing your site to be easily accessible to search engines, you also end up making it accessible to people with disabilities such as the blind or vision limited. Or you can look at it the other way: by designing your site to be accessible to people with disabilities and to be WAI or 508 compliant, you end up with an excellent site for search ranking.

It can often be hard to justify the cost and effort to make your site 508 compliant, even if it’s a legal requirement, however if you view it as a SEO effort, it’s much easier to assert a strong ROI on the project, and kill two birds with one stone.

You can read more about accessibility here:

• Andrea Hill’s Blog
• WAI Homepage
• Web Accessibility entry on Wikipedia

Standards

Be sure you’re following standards and best practices around your markup. This includes having nicely structured, valid, HTML/XHTML markup, and also being sure you have accurate helpful alt and title tags on your links, images, and other dom elements.

See what GoogleBot sees

First you have to be able to see the page the way GoogleBot sees it. The easiest way is to use a text based web browser like links or lynx. GoogleBot is actually a customized version of lynx, and supposedly has been/is being improved to be able to cull some data out of Flash, and possibly even some Javascript driven content, however it’s best to assume that what lynx shows you is what GoogleBot sees.

You’ll want to design your page structure and dom tree to be logical in order, structure, and semantic. Here is a view of CNN.com in lynx:

The CNN.com homepage viewed using the lynx browser

It’s extremely usable. You can see the main navigation first thing, and if you scroll down you see the main headline stories, latest news, etc… This is what you want your site to look like to GoogleBot.

In contrast, here is what a large theatre’s website looks like in lynx:?

The primary navigation on the real site doesn’t even appear here. The images don’t have titles or alt tags, and the structure is lacking and confusing. This is NOT what you want your site to look like to GoogleBot.

Share on Facebook

No matter how good your website or web application is, if people can’t find it, then it doesn’t do anyone any good.

The Secret to SEO

So how do you game Google? How do you trick search engines to put you at the top?

You don’t.

The “secret” to search engines is that they are built to return the most useful relevant results. All you have to do is to build your site in accordance with standards, common sense, and have it be helpful or useful. The upside to taking these elements into consideration is that not only will it help people find your site, but often it will improve the experience on the site as well.

I’m going to cover aspects of your site such as head tags, semantic tags, standards and accessibility, URL structures, limitations to be aware of, how to test what GoogleBot sees, tools to check your site with, and how to boost your Google ranking using means beyond just fixing up your site.

These pointers will apply to any website, but I will make some specific recommendations and provide some sample code for ATG sites, which are often not very SEO friendly, despite best intentions.

Head Meta Tags – Title, Description, and Keywords

The first important things that the search engine will see in your HTML are the tags in the head section of the HTML. There are three that we care about, Title, Description, and Keywords.

Title

The Title meta tag should uniquely identify the content of the page. Every page should have a different title, and the title should describe the content of the page accurately and succinctly. My preference is to put the site name after the unique portion of the title, like “Men’s Shoes – MyShoeStore.com” or “Men’s Fluevog Size 12 – MyShoeStore.com”. The content of the title should be relevant to the content on the page, words from the title should be found prominently within the page content.

When you’re using a template or including a shared header file, as is common in many dynamic sites, especially ATG sites, you need to pass in the unique title as a param.

Within your header page you use code like this:

<title><dsp:valueof param="title">Shoes</dsp:valueof> - MyShoeStore.com</title>

And from within the calling page you pass in a title like this:

<dsp:include page="fragments/page-header.jsp">
     <dsp:param name="title" value="Contact Us" />
</dsp:include>

Or with catalog data, like this:

<dsp:include page="fragments/page-header.jsp">
     <dsp:param name="title" param="category.displayName" />
</dsp:include>

The other thing to keep in mind is that the Title is the default name of the bookmark if the user bookmarks the page. It needs to be clear, concise and helpful, and should make it easy for the user to find among their other bookmarks.

Description

The description meta tag should contain a longer description of the page content. It should be no longer than 160 characters, and should use terms that are highly relevant for the page content, and are repeated within the page.

For dynamic sites, including ATG, I recommend the same approach as above for the title tag.

Keywords

The keywords meta tag should contain close to eight keywords which best represent the content of the page, and terms that people might use to search for similar content. Those keywords should feature prominently within the page content itself.

For dynamic sites, including ATG, I recommend the same approach as above for the title tag.

Tuning Your Head Tags

It’s important to tune your description and keywords to get the right users to the right content. This is the great part: by using the best relevant terms, by providing helpful accurate terms and not trying to scam or spam traffic your way, you end up helping searchers to find what they are looking for, and you end up helping yourself to get more and higher quality traffic..

So in order to be more efficient, you need to figure out what people are searching for when they end up at your site. There are a couple ways to do this. The most basic is to use an analytics package, such as Google Analytics, and look at the keywords people are using that direct them to your site. Google Analytics makes this easy by also showing you the average time on the site, and number of pages visited by users divided by the keywords they searched for. This makes it easy to tell which keywords generate traffic of users who WANT to be on your site, and find it useful (and hence will come back, make purchases, etc…), and which keywords brought in the wrong people, people who were looking for something else.

Here is the Google Analytics keyword report for my site:?

You can see that people who come to my site when they’re searching for “create web service seam” only look at the first page they see, and have 100% bounce rate. That means one of two things: A) this group of people was really looking for something else and my site didn’t help (in which case I should probably change my keywords and content so they don’t waste their time on my site) or B) the first page answered all of their questions. In this case the page on my site is probably pretty helpful for people searching for those words, so I’m going to keep the keywords, but you understand the point.

These types of reports should allow you to tune your keywords to get the best quality traffic and the happiest users.

To be more advanced you can track the keywords that didn’t just lead people to your site, but led people who made purchases/other goals to your site. You can do this with Google Analytics by setting up Goals, or with other analytics packages.

There will be three more posts on SEO coming in the next few days!

———
ATG SEO – Accessibility and GoogleBot

ATG SEO – URL Formats and Crawler Limits

ATG SEO – Tools and Traffic

Share on Facebook

First, what are those jsessionid params, and why are they there?

For a web application to have state, i.e. remember things from one page request to the next (such as that you’re logged in, who you are, what is in your shopping cart, etc…), most web applications have something called a session. The session starts when you hit the website at first, sticks with you while you are on the site, and expires after you have either logged out or have been idle (i.e. not clicked on anything) for a set period of time (perhaps 30 minutes).

In general the actual session data is held on the server, things like your shopping cart, your user profile, all of that. However, in order to associate requests from your web browser with the correct session, your browser needs to pass something for the web application to recognize which session is yours. This is traditionally done in two ways:

firstly and primarily using a session-life browser cookie (or two) which hold a session identifier and optionally some additional security token(s). The browser receives this cookie from the web application, and then sends the cookie back to the web application with each page request. The web application looks at the cookie, and figures out which session is yours, and handles your page request appropriately.

secondly, and usually only as a fall-back for browsers which do not support cookies or whose cookie support has been turned off, is to rewrite every link in the web application which points to another page in the same web application with a special session id added to the URI of the link. This is usually done as a path parameter (following a ‘;’), but sometimes is also done as a query parameter (following a ‘?’).

Since on the first request to a web application, the browser is not sending a session cookie, the web application has no way of knowing if the browser actually supports cookies or not. So for the first page, the web application will usually send back the session cookie AND rewrite all of the links on the page with the jsessionid just in case the cookie is not returned.

So what’s the problem?

Search engine spiders, like Google’s GoogleBot, usually do not support cookies. This means that they see the site with the jsessionid parameter in every link and every requested URL. So this leads to three related problems. First, the links that show up in a Google search include an ugly ‘jsessionid=xxxxxx’ which looks ugly. Second, Google doesn’t recognize that the jsessionid parameter doesn’t change the page content, and as such each time the GoogleBot hits the site, and gets a different jsessionid, it indexes all of the pages again. This leads to getting multiple result listings for the same page in search results. For instance you might see the same page listed 7 times in a row. Third, by having multiple instances of the same page with the same content, the Google PageRank of the actual page is severely diluted and perhaps even penalized due to the multiple presentations.

Because of these problems, we do not want the GoogleBot to see the jsessionid URI parameters.

In my earlier post, linked to above, I used Apache mod_rewrite to look for requests from GoogleBot, and send a redirect back to GoogleBot, redirecting it to the same URI it had initially requested, just stripped of the jsessionid parameter.

This time I’m going to use a Servlet Filter to prevent the jsessionid parameter from being inserted into the URL links on the page for GoogleBot requests. This is more elegant since there are no redirects.

First, I want to link to the web page which provided the starting point for the solution I used: JSESSIONID considered harmful

I took that approach and modified the filter code to only do this for GoogleBot requests, which will allow users who don’t support or allow cookies to still use the site.

I have one Java class: DisableUrlSessionFilter.java

package com.digitalsanctuary.util;

import java.io.IOException;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpServletResponseWrapper;

/**
 * Servlet filter which disables URL-encoded session identifiers.
 *
 *
 * Copyright (c) 2006, Craig Condit. All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 * * Redistributions of source code must retain the above copyright notice,
 * this list of conditions and the following disclaimer.
 * * Redistributions in binary form must reproduce the above copyright notice,
 * this list of conditions and the following disclaimer in the documentation
 * and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 *
 * Modified by Devon Hillard (devon@digitalsanctuary.com) to only filter for GoogleBot,
 * not for users without cookies enabled.
 *
 */
@SuppressWarnings("deprecation")
public class DisableUrlSessionFilter implements Filter {

    /**
     * The string to look for in the User-Agent header to identify the GoogleBot.
     */
    private static final String GOOGLEBOT_AGENT_STRING = "googlebot";

    /**
     * The request header with the User-Agent information in it.
     */
    private static final String USER_AGENT_HEADER_NAME = "User-Agent";

    /**
     * Filters requests to disable URL-based session identifiers.
     *
     * @param pRequest
     *                the request
     * @param pResponse
     *                the response
     * @param pChain
     *                the chain
     *
     * @throws IOException
     *                 Signals that an I/O exception has occurred.
     * @throws ServletException
     *                 the servlet exception
     */
    public void doFilter(final ServletRequest pRequest, final ServletResponse pResponse, final FilterChain pChain)
	    throws IOException, ServletException {
	// skip non-http requests
	if (!(pRequest instanceof HttpServletRequest)) {
	    pChain.doFilter(pRequest, pResponse);
	    return;
	}

	HttpServletRequest httpRequest = (HttpServletRequest) pRequest;
	HttpServletResponse httpResponse = (HttpServletResponse) pResponse;

	boolean isGoogleBot = false;

	if (httpRequest != null) {
	    String userAgent = httpRequest.getHeader(USER_AGENT_HEADER_NAME);
	    if (StringUtils.isNotBlank(userAgent)) {
		if (userAgent.toLowerCase().indexOf(GOOGLEBOT_AGENT_STRING) > -1) {
		    isGoogleBot = true;
		}
	    }
	}

	if (isGoogleBot) {
	    // wrap response to remove URL encoding
	    HttpServletResponseWrapper wrappedResponse = new HttpServletResponseWrapper(httpResponse) {
		@Override
		public String encodeRedirectUrl(final String url) {
		    return url;
		}

		@Override
		public String encodeRedirectURL(final String url) {
		    return url;
		}

		@Override
		public String encodeUrl(final String url) {
		    return url;
		}

		@Override
		public String encodeURL(final String url) {
		    return url;
		}
	    };

	    // process next request in chain
	    pChain.doFilter(pRequest, wrappedResponse);
	} else {
	    pChain.doFilter(pRequest, pResponse);
	}
    }

    /**
     * Unused.
     *
     * @param pConfig
     *                the config
     *
     * @throws ServletException
     *                 the servlet exception
     */
    public void init(final FilterConfig pConfig) throws ServletException {
    }

    /**
     * Unused.
     */
    public void destroy() {
    }
}

and the servlet filter configuration in my web.xml file:

	<filter>
		<filter-name>DisableUrlSessionFilter</filter-name>
		<filter-class>
			com.digitalsantuary.util.DisableUrlSessionFilter
		</filter-class>
	</filter>

....

	<filter-mapping>
		<filter-name>DisableUrlSessionFilter</filter-name>
		<url-pattern>/*</url-pattern>
	</filter-mapping>

So far, it seems to be working beautifully. It only impacts the GoogleBot, and it successfully strips the jsessionid parameter from the links on the site.

Enjoy!

Share on Facebook

The best thing you can do is to have your site be real, be useful, be focused on a specific topic or niche, and make sure links to it show up on other relevant sites.

A simple way to get those links is to make your site’s URL part of your signature, and post HELPFUL posts on relevant forums, blogs, etc… Be engaged in other communities who serve a demographic similar to your site’s ideal demographic. Become a contributing and trusted member. Your posts will drive the best type of traffic to your site.

As someone who uses Google to find content 1,000 times more than I use Google to get traffic to my site, I am firmly against gaming the system, link farming, spam, google bombs, or anything that drives down the signal-to-noise of search results. But there are steps you should take to ensure your site shows up as good signal where it should.

1. Provide free information. Even if you’re an e-commerce site, provide free product information, reviews, specifications, research, forums, etc… People tend to research online before they buy. If they do their research at your site, they’re more likely to buy from you because by providing helpful, ideally unbiased, information free of charge, your site will be more trusted. You will also end up with more external sites linking to your helpful free information, than would have linked to your Buy Me Now page. Useful information is valuable, and is referenced far more often than a commerce only page.

2. Ensure the information is visible to the Search Engines. Content in images, Flash, Ajax loaded data, sometimes iframes, etc… is not visible to search engines. If you must present information that way, ensure the same information is available via plain html. You can use javascript to load Flash over html for instance. Test your site using a text-based web browsers such as lynx or links. Make sure your site is structured well, has all your content available, and “works”.

3. The Title tag: should be the first tag within the head tag. Should be about 9 words. Keywords should be toward the beginning. Do not repeat words. It should be readable to a person and make sense. It should be unique for each page.

4. The Description tag: should be the second tag within the head tag. Should be about 16-20 words. It should be made up of complete sentences, with keywords toward the beginning. It should be readable to a person and make sense. It should be unique for each page.

5. The Keywords tag: should be the third tag within the head tag. Should be 30-50 words (assuming they’re all relevant). Should be in short phrase form, separated by commas. Capitalize all words. It should be unique for each page.

6. Heading content ( tags): should contain keyword phrases, but should not repeat identical phrases. Should be 2-4 words. Should structure your document just like an outline for a paper or presentation.

7. Image alt tags: every image should have an alt tag. The alt tag should be no more than 12 words long. It must be descriptive to a person and describe the image or it’s purpose in the page clearly. Again, test with a text-based web browser to see the alt tags in context.

8. Links: use keywords in the URL (more on this later). The link text should contain keywords. The surrounding text should also be keyword rich, using keywords which are relevant for the target page.

9. Main Content: Minimum of 300 words. Make the first 150-200 words keyword rich. Think of it like an executive summary. Words used in your title, description, and keyword head tags should appear at least twice each within the main text. CSS and JS should be externalized so as to not obscure the focus on the text.

10. URLs should be made up of useful words, separated by hyphens. Bad: “blog2″ Good: “tech-blog”. Bad: “/prod.jsp?id=1342342″ Good: “/mp3-players/apple/ipod/ipod-nano.jsp”. Many blogging tools and e-commerce tools will create category based faux directory structures for your pages.

Try to keep these guidelines in mind while building your site and templates. In general most of these tips not only help search engines understand your site best, but also help with accessibility, graceful degradation of browsers, and general readability for your users.

I’d welcome any other tips in the comments. Or any ideas around specific products (ATG, Seam, etc…).

Share on Facebook