note: this is post two in my ATG CSC and why I hate it series:

For some unknown reason ATG has replaced the standard Repository based search used for looking up profiles and orders with ATG Search engine based indexing and lookups. First off, I’ve never had any issues with the old Repository based search. It always seemed fast and worked fine. So I’m not sure why the change was warranted.

Search Issue 1: Supported Environments and Configuration

ATG Search is a 3rd party product ATG acquired so it’s not very “ATG-esque”. It’s also based on native binaries, which means it doesn’t run on Mac OS X, which myself and many others use for development. Configuring and tuning it is a real pain, and in my experience with hosting ATG Search using sites, the binary Search engines will occasionally die or have port conflicts, or otherwise foul up the works. The ATG Search used within CSC to index and provide searching on Profiles and Orders is sort of a sub-Search system. It’s not exactly full blown ATG Search, but it uses the same binaries, configs, and needs you to run Search patches, etc… In fixing a problem, which as far as I can tell didn’t exist, the new ATG CSC 9.1 has dramatically increased it’s install and runtime complexity by requiring ATG Search binary engines to be running.

Search Issue 2: Bulk Indexing Means Downtime

Another issue is if you are upgrading an existing environment, you have to run Bulk indexing on all of your existing Profiles and Orders in your CORE schema during the cut-over. What this means is that after you cut over to the new 9.1 site, using CSC 9.1 your CSR agents will be UNABLE TO HELP CUSTOMERS until the bulk indexing is completed. I’ve seen bulk indexing take ~6 hours per million items. Running profile and order bulk indexing at the same time does work, but also slows down the process a bit. So if you have 2 million orders and 2 million profiles, expect 12-18 hours of CSC downtime while the bulk indexing completes.

Search Issue 3: Incremental Indexing Falls Behind Production Activity

Newly created or updated Order and Profiles make their way into the CSC’s Search indexes by way of incremental indexing jobs, which run every 5 seconds. In testing, this works great. However in production it seems like it’s very easy to have the incoming item index events exceed the incremental indexing rate of processing. By which I mean the incremental indexing queue (SRCH_UPDATE_QUEUE) grows and grows, and your CSC Profile and Order indexes fall further and further behind your live data. Since most CSR calls are about orders placed in the last 24 hours, this becomes a serious problem very quickly as the CSR reps are unable to lookup orders or profiles created in the last X hours, where X continues to grow each day.

Part of this is due to badly planned default configurations, but part of it also seems intrinsic to the product. I am testing some post-patch 2 Search hotfixes, and we’ll see if they help or not. Another complaint I have is that no one told us about the hotfixes until after we’d gone live, and have this issue as a critical problem in our production environment. So if you are going live to CSC 9.1, make sure that ATG gives you any and all hotfixes they may have for CSC and Search BEFORE you go live.

Search Issue 4: Searching is More Limited

Because you can now only search on indexed properties your search options are more limited. For instance you can’t search for all orders in a date range (i.e. orders placed in the last 1 hour or 1 day).

Search Summary

Overall CSC Search is now more complex, harder to upgrade to, more limited, and has some significant production issues. Not a fan.

Share on Facebook

I recently worked on upgrading a client from ATG 2006.3 to ATG 9.1. The upgrade included upgrading Commerce Assist 2006.3 to CSC 9.1.

Commerce Assist 2006.3 is the ATG Customer Service application for ATG 2006.3 which allows CSRs to work with user profiles and orders. Commerce Assist 2006.3 is a relatively simple straight forward application. It’s essentially a standalone module. It can run against the existing CORE schema. The UI is simple but generally effective.

In stark contrast the new CSC is dependent on ATG Service and the front end is actually presented within the ATG Service UI. The UI is amazingly complex, which I’ll go into shortly. It also requires a new separate Service schema. It also has it’s own CA instance to deploy UI data… to itself…. It also requires it’s own Search engines.

For what it’s worth, before I met CSC 9.1, CA and Search were my two least favorite ATG products. Now CSC not only takes the cake as my #1 most disliked ATG product, but also requires #2 and #3.

There’s actually enough about the new CSC that I dislike that I’m going to break this out into a few posts:

Installation and Schema

CSC now requires it’s own schema. It also requires a bunch of schema changes to be made to CORE. Unfortunately the database installation scripts aren’t clear about where they should be run. service_production_all.sql throws a ton of errors regardless of where it’s run. The install docs are lacking, and point you to Service docs, Search docs, CA docs, etc… You also can’t really upgrade from Commerce Assist, so you lose all your old CSR related data.

Share on Facebook

Altin, who also provided an Albanian translation for 10MinuteMail (which will be launched very soon), wrote up a nice review of 10MinuteMail, which you can read here:

Avoid Spam With 10MinuteMail Service.

Share on Facebook

The JBoss Messaging service (at least on JBoss 4.3 EAP) defaults to using a local Hypersonic database. For production use you’ll want to switch away from Hypersonic to a real database, such as Oracle (in this example).

If you’re using XA datasources in general, it’s tempting to go ahead and create the new DefaultDS datasource definition as an XA datasource (like the example one jboss-eap-4.3/docs/examples/jca/oracle-xa-ds.xml ). However, I’ve just discovered that if you do that the JMS startup service won’t successfully create the tables it needs. The HILOSEQUENCES and TIMERS tables get created by the UUID key generator service, but the JMS table creation silently fails and then you get errors like this:

11:16:32,161 ERROR [ExceptionUtil] ServerPeer[0] startService
java.sql.SQLException: ORA-00942: table or view does not exist

Switch the DefaultDS definition to a non-XA version, and it will create all of the JBM_* tables successfully.

Share on Facebook

Video Camera by Andrew

The internet is constantly evolving.  The current trends of increased user bandwidth (upstream and downstream), mobile connectivity and bandwidth (3G/4G/WiMax/Ubiquitous WiFi), social networking/media/commerce, affordable video capture (cell phones, Flip cams, cheap HD video cameras), media driven websites (youtube, facebook, myspace) are all driving us toward a web future where user sourced video will be a major component to many web sites and web applications.  Everything from citizen based news and event coverage to social video sharing with friends and family to amateur movie production to media rich social commerce will be major growth areas over the next 36 months.

One critical component is video transcoding/encoding.  That is, taking video from your users in a wide variety of resolutions, formats, encoding, containers, and bit rates, and converting it into a single high quality, low size, web friendly format at one or more standardized resolutions.  Unless you’ve already worked on this problem yourself, you’d probably be surprised at how difficult it is to get a solution that works well with every major format and capture device your users are likely to use.  I’m no stranger to this problem having written my own transcoding system for How Good I Was.  Which while I’m proud of it, only handles some formats not all, and occasionally has some odd glitches.  The other big problem is even if you can build your own transcoding system, and are willing to keep up with new formats and new capture devices, the CPU usage for transcoding video is very high, and as more and more devices start capturing in HD it will get much worse.  If you start off on a single server, whenever you’re transcoding a new video your web site may be slow and unresponsive.  If you move to dedicated transcoding servers you’re incurring new costs and have to deal with distributing jobs to a farm of transcoding engines, etc… It’s a non trivial problem and can be quite costly.

Enter: zencoder!  Zencoder provides video conversion/transcoding as a web based or API driven service.  Zencoder’s primary focus is that they support virtually all video formats and encodings and will keep up with new codecs and formats as they are introduced to the market.  They are also focused on providing FAST encoding, minimizing the amount of time your users have to wait to see their uploaded videos on your site.  Zencoder pricing is very cheap and you can start off with a free plan and just pay by the minute if you like.  I will be using them for my next media driven project, and I highly recommend you check them out instead of trying to roll your own.

Share on Facebook