ATG CSC UI Customization

note: this is post three in my ATG CSC and why I hate it series:

UI Modification

Mostly customers using ATG CSC will want to make at least minor UI customizations. This could include providing information to CSRs about alternate payment types (PayPal, BillMeLater, Gift Cards), displaying customer loyalty program status, or more involved changes to support non-standard actions or flows.

Commerce Assist UI

When modifying the UI of the old Commerce Assist product, you basically ended up loading the DCS-CSR WAR into your own project (SVN/Eclipse), editing it’s JSPs directly, and deploying it out at the ATG install as part of your build process. While a little hacky, it was a very straight forward and simple process. Most of the JSPs were simple and used a small number of basic included files to support common fragments. Looking at a page in the Commerce Assist tool, and then tracking down which JSP you’d need to modify to change the page was very easy. The whole UI was driving using a single WAR filled with straightforward JSP files. With an Eclipse builder setup to call an ANT task that copies out the JSPs, you can actually see your UI changes in the Commerce Assist application in real time; no building, no bouncing. Simple.

ATG CSC UI

The new CSC tool’s UI is a much more complicated beast. The user interface is built dynamically using a mis-mash of JSPs and JSP fragments coming from two completely different WARs. The overall Service interface mostly comes from the agent.war, which is part of the Service module, where as commerce specific elements (such as order presentation and manipulation, catalog browsing, etc…) are pulled from the CSC’s DCS-CSR.war. There are many more JSP fragments and includes used, and the organization of the file is much more complex and confusing than the old Commerce Assist.

UI Framework Repository Data

The way the interface is built, how tabs and panels are presented, is managed via repository items in the database. These UI Framework items are the reason that you have to run a CA instance within CSC. This CA instance is used to deploy out the data used to build the UI. Until you setup CA and run a Full Deployment, you can’t even hit the CSC at all, since none of this data is available. Perhaps there is a good reason that these UI data elements need to be versioned and deployed from a workflow, independent of code/JSP development that makes sense in ATG Service, but within the context of CSC I doesn’t make any sense to me.

UI Components

Presumably in order to make UI customization easier, the JSPs (both from agent.war and DCS-CSR.war) include many fragments, and many of these fragments aren’t loaded via simple includes, but instead have includes referencing properties on Nucleus components. There are a number of undocumented components which each point to a JSP URI and a context root. For instance, a JSP representing a panel within the UI might include three sub-fragments by referencing three different ATG components, and doing each include based on properties of that component.

The premise is that instead of having to load the ATG WAR(s) into your project, and modify JSP files within the WARs, you can create a new custom WAR (myapp-csc.war), create new JSPs in that WAR for whatever overrides you need to make, and using the config layer override the URL and context root on the appropriate UI component. Then the parent JSP will include your custom JSP fragment from your WAR instead of the default JSP fragment.

This sounds great! Unfortunately it all breaks down about 5 minutes after you start actually looking at the JSPs.

The first issue is that many of the fragments you’d like to modify use a JSP (not DSP) include of a top.jspf file. You can’t just copy and paste the original JSP to your new JSP and modify it in place, because that include will fail and break most of the original functionality of the fragment. Because it’s a JSP include you can’t point it back at the original WAR context root. You have to copy the top.jspf into your war. However, agent.war and DCS-CSR.war have two different top.jspf files (in the same relative path), for instance one loads in the dsp taglib with a prefix of “dsp” while the other uses a prefix of “dspel” as well as many other differences. So assuming you want to override JSPs from both WARs, which is pretty likely, you have to copy in both, rename them, and reference the correct one for each of your custom JSPs. As far as I could tell, none of this was documented. Then you go ahead and setup your config layer overrides, build, and deploy.

The second issue is that there’s a great deal of logic and markup that is NOT in JSP fragments loaded in this way. As soon as you hit this scenario (which you will), you’re back to loading in the ATG WARs into your project, modifying the JSPs directly, and building out your customized versions of the WARs to the ATG install for assembly into the EAR. Only now you’re doing it on two WARs, not one.

As soon as you hit the second issue, you have to question the logic of using the configuration system at all, since it just adds more complexity, and you’re probably better off just working with the ATG WARs you have in your project.

Extensive markup and logic is hidden in tags

Then you’ll find some section of the UI that you need to change, only you can’t find the JSP that’s creating the markup. This is because there are JSP tags which include hundreds of lines of HTML output and JSP logic (DCS-CSR.war/WEb-INF/tags/displayOrderSummary.tag is a great example of this). So now you’re modifying tags as well as JSPs.

AJAX and DOJO

Another element that increases the complexity is that much of the interface is now AJAX driven, so pages are built and rebuilt via Javascript and AJAX calls, not just straight forward JSP. Figuring out how to make modifications to an AJAX driven UI is complex enough when its your own application and code, it’s MUCH harder when it’s two (two WARs) black boxes of DOJO magic with no documentation.

To summarize

In order to make relatively minor modifications to the CSC’s UI, you have to deal with:

  1. UI Framework Data in the DB and deployed via CA
  2. A custom WAR with JSP overrides and config files to override UI component properties
  3. JSPs in DCS-CSR.war, which you’ve had to add to your project and edit directly
  4. JSPs in agent.war, which you’ve had to add to your project and edit directly
  5. Tags in both wars
  6. Undocumented DOJO AJAX logic

It’s a real pain. I’m sure many of the changes were well intentioned, but in my experience when the rubber meets the road the new UI system makes customization and debugging much more difficult.

ATG CSC Search Issues

note: this is post two in my ATG CSC and why I hate it series:

For some unknown reason ATG has replaced the standard Repository based search used for looking up profiles and orders with ATG Search engine based indexing and lookups. First off, I’ve never had any issues with the old Repository based search. It always seemed fast and worked fine. So I’m not sure why the change was warranted.

Search Issue 1: Supported Environments and Configuration

ATG Search is a 3rd party product ATG acquired so it’s not very “ATG-esque”. It’s also based on native binaries, which means it doesn’t run on Mac OS X, which myself and many others use for development. Configuring and tuning it is a real pain, and in my experience with hosting ATG Search using sites, the binary Search engines will occasionally die or have port conflicts, or otherwise foul up the works. The ATG Search used within CSC to index and provide searching on Profiles and Orders is sort of a sub-Search system. It’s not exactly full blown ATG Search, but it uses the same binaries, configs, and needs you to run Search patches, etc… In fixing a problem, which as far as I can tell didn’t exist, the new ATG CSC 9.1 has dramatically increased it’s install and runtime complexity by requiring ATG Search binary engines to be running.

Search Issue 2: Bulk Indexing Means Downtime

Another issue is if you are upgrading an existing environment, you have to run Bulk indexing on all of your existing Profiles and Orders in your CORE schema during the cut-over. What this means is that after you cut over to the new 9.1 site, using CSC 9.1 your CSR agents will be UNABLE TO HELP CUSTOMERS until the bulk indexing is completed. I’ve seen bulk indexing take ~6 hours per million items. Running profile and order bulk indexing at the same time does work, but also slows down the process a bit. So if you have 2 million orders and 2 million profiles, expect 12-18 hours of CSC downtime while the bulk indexing completes.

Search Issue 3: Incremental Indexing Falls Behind Production Activity

Newly created or updated Order and Profiles make their way into the CSC’s Search indexes by way of incremental indexing jobs, which run every 5 seconds. In testing, this works great. However in production it seems like it’s very easy to have the incoming item index events exceed the incremental indexing rate of processing. By which I mean the incremental indexing queue (SRCH_UPDATE_QUEUE) grows and grows, and your CSC Profile and Order indexes fall further and further behind your live data. Since most CSR calls are about orders placed in the last 24 hours, this becomes a serious problem very quickly as the CSR reps are unable to lookup orders or profiles created in the last X hours, where X continues to grow each day.

Part of this is due to badly planned default configurations, but part of it also seems intrinsic to the product. I am testing some post-patch 2 Search hotfixes, and we’ll see if they help or not. Another complaint I have is that no one told us about the hotfixes until after we’d gone live, and have this issue as a critical problem in our production environment. So if you are going live to CSC 9.1, make sure that ATG gives you any and all hotfixes they may have for CSC and Search BEFORE you go live.

Search Issue 4: Searching is More Limited

Because you can now only search on indexed properties your search options are more limited. For instance you can’t search for all orders in a date range (i.e. orders placed in the last 1 hour or 1 day).

Search Summary

Overall CSC Search is now more complex, harder to upgrade to, more limited, and has some significant production issues. Not a fan.

ATG CSC and why I hate it

I recently worked on upgrading a client from ATG 2006.3 to ATG 9.1. The upgrade included upgrading Commerce Assist 2006.3 to CSC 9.1.

Commerce Assist 2006.3 is the ATG Customer Service application for ATG 2006.3 which allows CSRs to work with user profiles and orders. Commerce Assist 2006.3 is a relatively simple straight forward application. It’s essentially a standalone module. It can run against the existing CORE schema. The UI is simple but generally effective.

In stark contrast the new CSC is dependent on ATG Service and the front end is actually presented within the ATG Service UI. The UI is amazingly complex, which I’ll go into shortly. It also requires a new separate Service schema. It also has it’s own CA instance to deploy UI data… to itself…. It also requires it’s own Search engines.

For what it’s worth, before I met CSC 9.1, CA and Search were my two least favorite ATG products. Now CSC not only takes the cake as my #1 most disliked ATG product, but also requires #2 and #3.

There’s actually enough about the new CSC that I dislike that I’m going to break this out into a few posts:

Installation and Schema

CSC now requires it’s own schema. It also requires a bunch of schema changes to be made to CORE. Unfortunately the database installation scripts aren’t clear about where they should be run. service_production_all.sql throws a ton of errors regardless of where it’s run. The install docs are lacking, and point you to Service docs, Search docs, CA docs, etc… You also can’t really upgrade from Commerce Assist, so you lose all your old CSR related data.

Also see:

Guide for Large Scale ATG Applications

Kelly Goetsch has written a very impressive 64 page guide on the development and deployment of very large scalable ATG e-Commerce applications. There’s a ton of great information (a little bit came from me!) and it’s a worth-while read, even if your application isn’t running on 200 production servers. Unfortunately you have to have an atg.com account AND be associated with an active support contract in order to read it.

If you do, check it out: https://community.atg.com/docs/DOC-1871

Obligitory Spark::red ATG Hosting plug: The three founders all worked on AT&T’s massive ATG e-Commerce cluster, so we understand how to scale ATG to hundreds of nodes, and how to squeeze out maximal performance and stability out of clusters of any size.

ATG Newsletter Went Out

Our first Spark::red ATG Newsletter was sent on Tuesday morning! We’re pleased and proud to have delivered the first of many monthly ATG Newsletters.

In this newsletter we talk about the importance of improving your site performance, especially now that Google is using site performance as a search result ranking factor. We talk about Why, and provide several helpful links to help with How. I’d be remiss if I didn’t plug the fantastic ATG Hosting that Spark::red can provide, including extensive performance tuning at every level, web, app and database, using knowledge gained from 12 years of ATG experience.

We also reveal our PCI Compliant ATG Encryption Module which is the first PCI complaint credit card encryption option for the ATG eCommerce Platform. It handles strong encryption, key management, importing plain text and encrypted data, periodic re-keying/re-encryption, and more. It’s the fastest and most affordable path to being able to pass a PCI audit for your ATG eCommerce application. Contact us for more info: sales@sparkred.com.

You can see the whole newsletter here: Spark::red ATG Newsletter #1, and if you haven’t already, I recommend signing up so you don’t miss the next one: Sign Up for the Sparkred ATG Newsletter!