Search engines like Google will often index content with params like JSESSIONID and other session or conversation scope params. This causes two problems: first the links returned in the Google search results can have these parameters in them, resulting in “session not found” or other incompatible session state issues. Secondly it can cause a single page of content, to be indexed multiple times (with differing parameters) this diluting your page’s rank.

I’ve posted two solutions to this issue in the past: Using Apache to ReWrite URLs to remove JSESSIONID and a more advanced solution of using a Servlet Filter to avoid adding JSESSIONID for GoogleBot Requests.

Now there’s an even better way to handle this. Google has added an amazing new feature to their Webmaster Tools which allows you to specify how the GoogleBot indexer should handle various parameters. You can ignore certain parameters such as JSESSIONID, cid, and others, and also specifically not ignore other parameters such as productId, skuId, etc…

Log into your Google Webmaster Tools, and select the site you wish to work with. Under “Site Configuration” -> “Settings” there is a new section at the bottom called “Parameter handling”. Click on “adjust parameter settings” to expand the parameter handling configuration for your site. Sometimes Google will suggest various parameters it has discovered while crawling your site, and other times you just enter the parameters you want Google to ignore or pay attention to.

Google Webmaster Tools Parameter Handling Interface

This is a much more elegant solution to the JSESSIONID problem, and also allows you to easily handle other parameters your site may use for either session state or dynamic content generation correctly. The only downside is that this only impacts Google, whereas with the correct configuration my older two solutions can handle any Search Engine Bot. Maybe other search providers will or do provide a similar feature.

Share on Facebook

Having control over the HTTP response headers allows you to set cache related headers in responses for various content you’d like cached on the browser (or an intermediary proxy). I created the ATG Cache Control DAS pipeline Servlet a year ago, but when you’re using JBoss you need another solution.

Since the DAF pipeline is only executed for JSPs the pipeline Servlet it doesn’t allow you to set headers for the static media items you’re more likely to want to cache. I created a Servlet Filter which allows you to set cache headers in JBoss based on URI patterns. It doesn’t allow the same fine grained control that the old pipeline Servlet does, but it should work for most situations.

Servlet Filters are very similar to ATG pipeline Servlets in that they are executed within the lifecycle of a request and can read the request and modify the response. The filter I created gets configured from the web.xml and sets the response headers relating to caching. You can configure different instances of the filter for each cache time you need, an hour, a day, or a week, and map different URL patterns to the appropriate instances.

I’ve added the filter into Foundation, the open source ATG e-commerce framework project, which is hosted at the Spark::red Development Community.

The Servlet filter code looks like this:

package org.foundation.servlet.filter;

import java.io.IOException;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.Locale;
import java.util.TimeZone;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletResponse;

/**
 * The Class CacheHeaderFilter.
 *
 * @author Devon Hillard
 */
public class CacheHeaderFilter implements Filter {

    /**
     * The Constant MILLISECONDS_IN_SECOND.
     */
    private static final int MILLISECONDS_IN_SECOND = 1000;

    /** The Constant POST_CHECK_VALUE. */
    private static final String POST_CHECK_VALUE = "post-check=";

    /** The Constant PRE_CHECK_VALUE. */
    private static final String PRE_CHECK_VALUE = "pre-check=";

    /** The Constant MAX_AGE_VALUE. */
    private static final String MAX_AGE_VALUE = "max-age=";

    /** The Constant ZERO_STRING_VALUE. */
    private static final String ZERO_STRING_VALUE = "0";

    /** The Constant NO_STORE_VALUE. */
    private static final String NO_STORE_VALUE = "no-store";

    /** The Constant NO_CACHE_VALUE. */
    private static final String NO_CACHE_VALUE = "no-cache";

    /** The Constant PRAGMA_HEADER. */
    private static final String PRAGMA_HEADER = "Pragma";

    /** The Constant CACHE_CONTROL_HEADER. */
    private static final String CACHE_CONTROL_HEADER = "Cache-Control";

    /** The Constant EXPIRES_HEADER. */
    private static final String EXPIRES_HEADER = "Expires";

    /** The Constant LAST_MODIFIED_HEADER. */
    private static final String LAST_MODIFIED_HEADER = "Last-Modified";

    /** The Constant CACHE_TIME_PARAM_NAME. */
    private static final String CACHE_TIME_PARAM_NAME = "CacheTime";

    /** The Static HTTP_DATE_FORMAT object. */
    private static final DateFormat HTTP_DATE_FORMAT = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss z", Locale.US);

static {
HTTP_DATE_FORMAT.setTimeZone(TimeZone.getTimeZone("GMT"));
}

    /** The reply headers. */
    private String[][] mReplyHeaders = { {} };

    /** The cache time in seconds. */
    private Long mCacheTime = 0L;

    /**
     * Initializes the Servlet filter with the cache time and sets up the unchanging headers.
     *
     * @param pConfig the config
     *
     * @see javax.servlet.Filter#init(javax.servlet.FilterConfig)
     */
    public void init(final FilterConfig pConfig) {
        final ArrayList<String[]> newReplyHeaders = new ArrayList<String[]>();
        this.mCacheTime = Long.parseLong(pConfig.getInitParameter(CACHE_TIME_PARAM_NAME));
        if (this.mCacheTime > 0L) {
            newReplyHeaders.add(new String[] { CACHE_CONTROL_HEADER, MAX_AGE_VALUE + this.mCacheTime.longValue() });
            newReplyHeaders.add(new String[] { CACHE_CONTROL_HEADER, PRE_CHECK_VALUE + this.mCacheTime.longValue() });
            newReplyHeaders.add(new String[] { CACHE_CONTROL_HEADER, POST_CHECK_VALUE + this.mCacheTime.longValue() });
        } else {
            newReplyHeaders.add(new String[] { PRAGMA_HEADER, NO_CACHE_VALUE });
            newReplyHeaders.add(new String[] { EXPIRES_HEADER, ZERO_STRING_VALUE });
            newReplyHeaders.add(new String[] { CACHE_CONTROL_HEADER, NO_CACHE_VALUE });
            newReplyHeaders.add(new String[] { CACHE_CONTROL_HEADER, NO_STORE_VALUE });
        }
        this.mReplyHeaders = new String[newReplyHeaders.size()][2];
        newReplyHeaders.toArray(this.mReplyHeaders);
    }

    /**
     * Do filter.
     *
     * @param pRequest the request
     * @param pResponse the response
     * @param pChain the chain
     *
     * @throws IOException Signals that an I/O exception has occurred.
     * @throws ServletException the servlet exception
     *
     * @see javax.servlet.Filter#doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse,
     *      javax.servlet.FilterChain)
     */
    public void doFilter(final ServletRequest pRequest, final ServletResponse pResponse, final FilterChain pChain)
            throws IOException, ServletException {
        // Apply the headers
        final HttpServletResponse httpResponse = (HttpServletResponse) pResponse;
        for (final String[] replyHeader : this.mReplyHeaders) {
            final String name = replyHeader[0];
            final String value = replyHeader[1];
            httpResponse.addHeader(name, value);
        }
        if (this.mCacheTime > 0L) {
            final long now = System.currentTimeMillis();
             final DateFormat httpDateFormat = (DateFormat) HTTP_DATE_FORMAT.clone();
            httpResponse.addHeader(LAST_MODIFIED_HEADER, httpDateFormat.format(new Date(now)));
            httpResponse.addHeader(EXPIRES_HEADER, httpDateFormat.format(new Date(now
                    + (this.mCacheTime.longValue() * MILLISECONDS_IN_SECOND))));
        }
        pChain.doFilter(pRequest, pResponse);
    }

    /**
     * Destroy all humans!
     *
     * @see javax.servlet.Filter#destroy()
     */
    public void destroy() {
    }

}

And the web.xml configuration looks like this:

	<filter>
		<filter-name>CacheFilterOneWeek</filter-name>
		<filter-class>org.foundation.servlet.filter.CacheHeaderFilter</filter-class>
		<init-param>
			<param-name>CacheTime</param-name>
			<param-value>604800</param-value>
		</init-param>
	</filter>

	<filter>
		<filter-name>CacheFilterOneDay</filter-name>
		<filter-class>org.foundation.servlet.filter.CacheHeaderFilter</filter-class>
		<init-param>
			<param-name>CacheTime</param-name>
			<param-value>86400</param-value>
		</init-param>
	</filter>

	<filter>
		<filter-name>CacheFilterOneHour</filter-name>
		<filter-class>org.foundation.servlet.filter.CacheHeaderFilter</filter-class>
		<init-param>
			<param-name>CacheTime</param-name>
			<param-value>3600</param-value>
		</init-param>
	</filter>

.......

<filter-mapping>
		<filter-name>CacheFilterOneDay</filter-name>
		<url-pattern>*.js</url-pattern>
	</filter-mapping>
	<filter-mapping>
		<filter-name>CacheFilterOneDay</filter-name>
		<url-pattern>*.css</url-pattern>
	</filter-mapping>
	<filter-mapping>
		<filter-name>CacheFilterOneWeek</filter-name>
		<url-pattern>*.jpg</url-pattern>
	</filter-mapping>
	<filter-mapping>
		<filter-name>CacheFilterOneWeek</filter-name>
		<url-pattern>*.gif</url-pattern>
	</filter-mapping>

Share on Facebook

Some ATG product licenses are bound to a specific list of IP addresses. However, it may validate that in a somewhat counter-intuitive manner, at least under Linux.

If, for example, you are running ATG within JBoss on a server with multiple IP addresses (or multiple NICs), you might expect that if you bind JBoss to a specific IP address, that will be the IP that is used in the license check, since clearly that is the real IP the server is listening on.

Not so. It appears that instead of checking the IP address(es) of the J2EE container, it looks at the machine’s hostname. So if your hostname is configured to a name that is mapped to a different IP in your /etc/hosts file, your license will fail to validate.

The fix is relatively simple, just change the hostname of the server to match the name mapped to the licensed IP address in your /etc/hosts file, and start up JBoss again.

However, since this essentially alters the “identity” of your server it can have other repercussions on your server. So be warned. It may make more sense to get the licenses issued with IPs matching your hostname, not the actual IP you intend to run JBoss/ATG on.

I think this is a bit of a bug, in my opinion.

Share on Facebook

Instead of just using the Apache mod_rewrite rules from my post on “Hiding jsessionid parameters from Google“, which uses redirects, wouldn’t it be better to simply not output the jsessionid parameter into the URLs?

First, what are those jsessionid params, and why are they there?

For a web application to have state, i.e. remember things from one page request to the next (such as that you’re logged in, who you are, what is in your shopping cart, etc…), most web applications have something called a session. The session starts when you hit the website at first, sticks with you while you are on the site, and expires after you have either logged out or have been idle (i.e. not clicked on anything) for a set period of time (perhaps 30 minutes).

In general the actual session data is held on the server, things like your shopping cart, your user profile, all of that. However, in order to associate requests from your web browser with the correct session, your browser needs to pass something for the web application to recognize which session is yours. This is traditionally done in two ways:

firstly and primarily using a session-life browser cookie (or two) which hold a session identifier and optionally some additional security token(s). The browser receives this cookie from the web application, and then sends the cookie back to the web application with each page request. The web application looks at the cookie, and figures out which session is yours, and handles your page request appropriately.

secondly, and usually only as a fall-back for browsers which do not support cookies or whose cookie support has been turned off, is to rewrite every link in the web application which points to another page in the same web application with a special session id added to the URI of the link. This is usually done as a path parameter (following a ‘;’), but sometimes is also done as a query parameter (following a ‘?’).

Since on the first request to a web application, the browser is not sending a session cookie, the web application has no way of knowing if the browser actually supports cookies or not. So for the first page, the web application will usually send back the session cookie AND rewrite all of the links on the page with the jsessionid just in case the cookie is not returned.

So what’s the problem?

Search engine spiders, like Google’s GoogleBot, usually do not support cookies. This means that they see the site with the jsessionid parameter in every link and every requested URL. So this leads to three related problems. First, the links that show up in a Google search include an ugly ‘jsessionid=xxxxxx’ which looks ugly. Second, Google doesn’t recognize that the jsessionid parameter doesn’t change the page content, and as such each time the GoogleBot hits the site, and gets a different jsessionid, it indexes all of the pages again. This leads to getting multiple result listings for the same page in search results. For instance you might see the same page listed 7 times in a row. Third, by having multiple instances of the same page with the same content, the Google PageRank of the actual page is severely diluted and perhaps even penalized due to the multiple presentations.

Because of these problems, we do not want the GoogleBot to see the jsessionid URI parameters.

In my earlier post, linked to above, I used Apache mod_rewrite to look for requests from GoogleBot, and send a redirect back to GoogleBot, redirecting it to the same URI it had initially requested, just stripped of the jsessionid parameter.

This time I’m going to use a Servlet Filter to prevent the jsessionid parameter from being inserted into the URL links on the page for GoogleBot requests. This is more elegant since there are no redirects.

First, I want to link to the web page which provided the starting point for the solution I used: JSESSIONID considered harmful

I took that approach and modified the filter code to only do this for GoogleBot requests, which will allow users who don’t support or allow cookies to still use the site.

I have one Java class: DisableUrlSessionFilter.java

package com.digitalsanctuary.util;

import java.io.IOException;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import javax.servlet.http.HttpServletResponseWrapper;

/**
 * Servlet filter which disables URL-encoded session identifiers.
 *
 *
 * Copyright (c) 2006, Craig Condit. All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 * * Redistributions of source code must retain the above copyright notice,
 * this list of conditions and the following disclaimer.
 * * Redistributions in binary form must reproduce the above copyright notice,
 * this list of conditions and the following disclaimer in the documentation
 * and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 *
 * Modified by Devon Hillard (devon@digitalsanctuary.com) to only filter for GoogleBot,
 * not for users without cookies enabled.
 *
 */
@SuppressWarnings("deprecation")
public class DisableUrlSessionFilter implements Filter {

    /**
     * The string to look for in the User-Agent header to identify the GoogleBot.
     */
    private static final String GOOGLEBOT_AGENT_STRING = "googlebot";

    /**
     * The request header with the User-Agent information in it.
     */
    private static final String USER_AGENT_HEADER_NAME = "User-Agent";

    /**
     * Filters requests to disable URL-based session identifiers.
     *
     * @param pRequest
     *                the request
     * @param pResponse
     *                the response
     * @param pChain
     *                the chain
     *
     * @throws IOException
     *                 Signals that an I/O exception has occurred.
     * @throws ServletException
     *                 the servlet exception
     */
    public void doFilter(final ServletRequest pRequest, final ServletResponse pResponse, final FilterChain pChain)
	    throws IOException, ServletException {
	// skip non-http requests
	if (!(pRequest instanceof HttpServletRequest)) {
	    pChain.doFilter(pRequest, pResponse);
	    return;
	}

	HttpServletRequest httpRequest = (HttpServletRequest) pRequest;
	HttpServletResponse httpResponse = (HttpServletResponse) pResponse;

	boolean isGoogleBot = false;

	if (httpRequest != null) {
	    String userAgent = httpRequest.getHeader(USER_AGENT_HEADER_NAME);
	    if (StringUtils.isNotBlank(userAgent)) {
		if (userAgent.toLowerCase().indexOf(GOOGLEBOT_AGENT_STRING) > -1) {
		    isGoogleBot = true;
		}
	    }
	}

	if (isGoogleBot) {
	    // wrap response to remove URL encoding
	    HttpServletResponseWrapper wrappedResponse = new HttpServletResponseWrapper(httpResponse) {
		@Override
		public String encodeRedirectUrl(final String url) {
		    return url;
		}

		@Override
		public String encodeRedirectURL(final String url) {
		    return url;
		}

		@Override
		public String encodeUrl(final String url) {
		    return url;
		}

		@Override
		public String encodeURL(final String url) {
		    return url;
		}
	    };

	    // process next request in chain
	    pChain.doFilter(pRequest, wrappedResponse);
	} else {
	    pChain.doFilter(pRequest, pResponse);
	}
    }

    /**
     * Unused.
     *
     * @param pConfig
     *                the config
     *
     * @throws ServletException
     *                 the servlet exception
     */
    public void init(final FilterConfig pConfig) throws ServletException {
    }

    /**
     * Unused.
     */
    public void destroy() {
    }
}

and the servlet filter configuration in my web.xml file:

	<filter>
		<filter-name>DisableUrlSessionFilter</filter-name>
		<filter-class>
			com.digitalsantuary.util.DisableUrlSessionFilter
		</filter-class>
	</filter>

....

	<filter-mapping>
		<filter-name>DisableUrlSessionFilter</filter-name>
		<url-pattern>/*</url-pattern>
	</filter-mapping>

So far, it seems to be working beautifully. It only impacts the GoogleBot, and it successfully strips the jsessionid parameter from the links on the site.

Enjoy!

Share on Facebook

If you’re running a website on JBoss you may discover that Google has indexed your pages with a jsessionid query parameter in the links.

The Google crawl bot does not support cookies, therefore JBoss uses the jsessionid query parameter in order to maintain a session state without cookies. These query parameters can impact your Google rank and indexing efficiency as the same page can be indexed multiple times with different session ids, and dilute your ranking. Also, it leads to ugly links.

If you want to still be able to support non-cookie using users, but would like Google to see cleaner links, you can use Apache’s mod_rewrite to modify the links for the Google bot only, leaving the normal functionality available to the rest of your users.

Assuming you have mod_rewrite enabled in your Apache instance, use this configuration in your apache config:

	# This should strip out jsessionids from google
	RewriteCond %{HTTP_USER_AGENT} (googlebot) [NC]
	ReWriteRule ^(.*);jsessionid=[A-Za-z0-9]+(.*)$ $1$2 [L,R=301]

This rule says for request where the user agent contains “googlebot” (with case insensitive matching), rewrite the URL without the jsessionid. It seems to work nicely.

Share on Facebook