For some unknown reason ATG has replaced the standard Repository based search used for looking up profiles and orders with ATG Search engine based indexing and lookups. First off, I’ve never had any issues with the old Repository based search. It always seemed fast and worked fine. So I’m not sure why the change was warranted.

Search Issue 1: Supported Environments and Configuration

ATG Search is a 3rd party product ATG acquired so it’s not very “ATG-esque”. It’s also based on native binaries, which means it doesn’t run on Mac OS X, which myself and many others use for development. Configuring and tuning it is a real pain, and in my experience with hosting ATG Search using sites, the binary Search engines will occasionally die or have port conflicts, or otherwise foul up the works. The ATG Search used within CSC to index and provide searching on Profiles and Orders is sort of a sub-Search system. It’s not exactly full blown ATG Search, but it uses the same binaries, configs, and needs you to run Search patches, etc… In fixing a problem, which as far as I can tell didn’t exist, the new ATG CSC 9.1 has dramatically increased it’s install and runtime complexity by requiring ATG Search binary engines to be running.

Search Issue 2: Bulk Indexing Means Downtime

Another issue is if you are upgrading an existing environment, you have to run Bulk indexing on all of your existing Profiles and Orders in your CORE schema during the cut-over. What this means is that after you cut over to the new 9.1 site, using CSC 9.1 your CSR agents will be UNABLE TO HELP CUSTOMERS until the bulk indexing is completed. I’ve seen bulk indexing take ~6 hours per million items. Running profile and order bulk indexing at the same time does work, but also slows down the process a bit. So if you have 2 million orders and 2 million profiles, expect 12-18 hours of CSC downtime while the bulk indexing completes.

Search Issue 3: Incremental Indexing Falls Behind Production Activity

Newly created or updated Order and Profiles make their way into the CSC’s Search indexes by way of incremental indexing jobs, which run every 5 seconds. In testing, this works great. However in production it seems like it’s very easy to have the incoming item index events exceed the incremental indexing rate of processing. By which I mean the incremental indexing queue (SRCH_UPDATE_QUEUE) grows and grows, and your CSC Profile and Order indexes fall further and further behind your live data. Since most CSR calls are about orders placed in the last 24 hours, this becomes a serious problem very quickly as the CSR reps are unable to lookup orders or profiles created in the last X hours, where X continues to grow each day.

Part of this is due to badly planned default configurations, but part of it also seems intrinsic to the product. I am testing some post-patch 2 Search hotfixes, and we’ll see if they help or not. Another complaint I have is that no one told us about the hotfixes until after we’d gone live, and have this issue as a critical problem in our production environment. So if you are going live to CSC 9.1, make sure that ATG gives you any and all hotfixes they may have for CSC and Search BEFORE you go live.

Search Issue 4: Searching is More Limited

Because you can now only search on indexed properties your search options are more limited. For instance you can’t search for all orders in a date range (i.e. orders placed in the last 1 hour or 1 day).

Search Summary

Overall CSC Search is now more complex, harder to upgrade to, more limited, and has some significant production issues. Not a fan.

Share on Facebook

Commerce Assist 2006.3 is the ATG Customer Service application for ATG 2006.3 which allows CSRs to work with user profiles and orders. Commerce Assist 2006.3 is a relatively simple straight forward application. It’s essentially a standalone module. It can run against the existing CORE schema. The UI is simple but generally effective.

In stark contrast the new CSC is dependent on ATG Service and the front end is actually presented within the ATG Service UI. The UI is amazingly complex, which I’ll go into shortly. It also requires a new separate Service schema. It also has it’s own CA instance to deploy UI data… to itself…. It also requires it’s own Search engines.

For what it’s worth, before I met CSC 9.1, CA and Search were my two least favorite ATG products. Now CSC not only takes the cake as my #1 most disliked ATG product, but also requires #2 and #3.

There’s actually enough about the new CSC that I dislike that I’m going to break this out into a few posts:

Installation and Schema

CSC now requires it’s own schema. It also requires a bunch of schema changes to be made to CORE. Unfortunately the database installation scripts aren’t clear about where they should be run. service_production_all.sql throws a ton of errors regardless of where it’s run. The install docs are lacking, and point you to Service docs, Search docs, CA docs, etc… You also can’t really upgrade from Commerce Assist, so you lose all your old CSR related data.

Share on Facebook

Avoid Spam With 10MinuteMail Service.

Share on Facebook

If you’re using XA datasources in general, it’s tempting to go ahead and create the new DefaultDS datasource definition as an XA datasource (like the example one jboss-eap-4.3/docs/examples/jca/oracle-xa-ds.xml ). However, I’ve just discovered that if you do that the JMS startup service won’t successfully create the tables it needs. The HILOSEQUENCES and TIMERS tables get created by the UUID key generator service, but the JMS table creation silently fails and then you get errors like this:

11:16:32,161 ERROR [ExceptionUtil] ServerPeer[0] startService
java.sql.SQLException: ORA-00942: table or view does not exist

Switch the DefaultDS definition to a non-XA version, and it will create all of the JBM_* tables successfully.

Share on Facebook

Video Camera by Andrew

The internet is constantly evolving.  The current trends of increased user bandwidth (upstream and downstream), mobile connectivity and bandwidth (3G/4G/WiMax/Ubiquitous WiFi), social networking/media/commerce, affordable video capture (cell phones, Flip cams, cheap HD video cameras), media driven websites (youtube, facebook, myspace) are all driving us toward a web future where user sourced video will be a major component to many web sites and web applications.  Everything from citizen based news and event coverage to social video sharing with friends and family to amateur movie production to media rich social commerce will be major growth areas over the next 36 months.

One critical component is video transcoding/encoding.  That is, taking video from your users in a wide variety of resolutions, formats, encoding, containers, and bit rates, and converting it into a single high quality, low size, web friendly format at one or more standardized resolutions.  Unless you’ve already worked on this problem yourself, you’d probably be surprised at how difficult it is to get a solution that works well with every major format and capture device your users are likely to use.  I’m no stranger to this problem having written my own transcoding system for How Good I Was.  Which while I’m proud of it, only handles some formats not all, and occasionally has some odd glitches.  The other big problem is even if you can build your own transcoding system, and are willing to keep up with new formats and new capture devices, the CPU usage for transcoding video is very high, and as more and more devices start capturing in HD it will get much worse.  If you start off on a single server, whenever you’re transcoding a new video your web site may be slow and unresponsive.  If you move to dedicated transcoding servers you’re incurring new costs and have to deal with distributing jobs to a farm of transcoding engines, etc… It’s a non trivial problem and can be quite costly.

Enter: zencoder!  Zencoder provides video conversion/transcoding as a web based or API driven service.  Zencoder’s primary focus is that they support virtually all video formats and encodings and will keep up with new codecs and formats as they are introduced to the market.  They are also focused on providing FAST encoding, minimizing the amount of time your users have to wait to see their uploaded videos on your site.  Zencoder pricing is very cheap and you can start off with a free plan and just pay by the minute if you like.  I will be using them for my next media driven project, and I highly recommend you check them out instead of trying to roll your own.

Share on Facebook

White Books by Vicki

Initially I was thrilled that my local library has a website where I can request books from anywhere in the library network, and when they’re ready for pickup at my local branch, I get an e-mail. It makes things so much easier.  I love to read, read quickly, and don’t need to keep every book I read forever (although I have tons of books), so using the library more is better for my budget than hitting Amazon every time my bedside table is bare.

Yesterday I realized that while handy, there is so much more they could do on the website that would make things better both for the borrowers and the library system.

1. Netflix Style Queue

One issue is that you can add a number of requested books some of which might be available within the next day or two, some of which might not be available for many months. You also might add several books in a series, but have them become available out-of-order. Also, when books become available, you have to pick them up within a week, or you lose them and they vanish from your list. The result is that you might end up with 10 books ready for pickup on the same day, when you simply can’t ready all 10 books before they’ll all be due back. Or perhaps 5 of those 10 are books 2 through 6 in a series, but you’re still waiting on book 1.

I’d love a Netflix style queue where I can set a “simultaneous checkout” number, which is basically how many books I think I can read in 3 weeks (when they’re due back), and I can order my queue to ensure I don’t get books in a series out-of-order.

It would help me get the books in the order I choose and also not end up with more books than I can read. It would help the library in that I wouldn’t be holding onto 10 books when I’ll only read 3 of them, thus preventing 7 books from being checked out by people who can actually have read them. It would also allow them to evolve some smart management around popular books knowing people queue lists and historical book consumption rates.

2. Group Media Options

When you search for a book, using title, author, or other keywords, you get back a list of library items. Unfortunately each media type displays as a separate result. For instance if I search for an author, one of his books, Book A, might show up five or more times, randomly dispersed, on the search results page. Once for the standard paperback, once for the larger format trade paperback, once for the large print edition, once for the audio book on CD, once for the unabridged audiobook on CD, once for the downloadable audiobook, once for a different edition of the paperback, etc… By default the first results page shows 25 items, and there are times where that whole page only shows 4 different books. You might have to page through several results pages to find the book and format you want. Not only is it a pain, but sometimes I’ve clicked on the wrong media format because I wasn’t paying close enough attention and ended up with something I didn’t want.

They should show one result per book, and then give you format/media options once you select the book. This would not only make it easier to search and find what you are looking for, but would also allow the library to display the current queue sizes for each media type. For instance while I prefer the trade paperback format, if the standard paperback is instantly available, where the trade has 5 people waiting for it, I might opt to get the standard paperback instead, reducing overall waits for myself and others.

3. Summary, Ratings, and Reviews

Currently, some books have a summary in the system, but it’s not shown on the main book page. You have to click on the Summary link which takes you to another page. This is just bizarre. Put all the commonly desired information on the main product/item page. Make it easy.

Allow readers to rate and review books they’ve read, and allow me to read the reviews and see the ratings. Currently I use Amazon to check reviews and ratings for books, and then go to the library site to add them to my queue. This ties into my next point:

4. Integrate with Amazon

Partner up with Amazon to pull book images, summary, ratings, reviews, etc… And provide a “Buy this book” button that is an Amazon affiliate link. Also, after I’ve returned a book, send me an e-mail or prompt me when I hit the website next to rate/review/buy the book I just returned. The data would be very useful to have on the website (see #3) and the affiliate link would drive revenue to the library. Also suggest sending the book as a gift, via the Amazon link.

5. Redesign

The website looks VERY dated, and isn’t easy to navigate or use. It wouldn’t take much to improve it dramatically.

Share on Facebook

If you do, check it out: https://community.atg.com/docs/DOC-1871

Obligitory Spark::red ATG Hosting plug: The three founders all worked on AT&T’s massive ATG e-Commerce cluster, so we understand how to scale ATG to hundreds of nodes, and how to squeeze out maximal performance and stability out of clusters of any size.

Share on Facebook

You can read about it here: 10 Minute Mail – Recommended Temporary Email Service

Share on Facebook

In this newsletter we talk about the importance of improving your site performance, especially now that Google is using site performance as a search result ranking factor. We talk about Why, and provide several helpful links to help with How. I’d be remiss if I didn’t plug the fantastic ATG Hosting that Spark::red can provide, including extensive performance tuning at every level, web, app and database, using knowledge gained from 12 years of ATG experience.

We also reveal our PCI Compliant ATG Encryption Module which is the first PCI complaint credit card encryption option for the ATG eCommerce Platform. It handles strong encryption, key management, importing plain text and encrypted data, periodic re-keying/re-encryption, and more. It’s the fastest and most affordable path to being able to pass a PCI audit for your ATG eCommerce application. Contact us for more info: sales@sparkred.com.

You can see the whole newsletter here: Spark::red ATG Newsletter #1, and if you haven’t already, I recommend signing up so you don’t miss the next one: Sign Up for the Sparkred ATG Newsletter!

Share on Facebook

Which sounds good as I was wondering if there was stuff I could be doing with layout or ad types to help increase clicks, except this was from a random guy at a gmx.com address, with no company name. Googling his name and e-mail address turned up nothing. So I replied asking for his company info or references, etc…

Sorry, but our company has no web
site.
You will easteablish our credibility during partnership with us.
Lets just try it first, ok?
Put this code between the body tags on your 10minutemail.com
main page only:
<img src=”some russian site” />
It is an 1*1px transparent image.

Supposedly after I do that I’ll see the result in a couple of days.

So wow, little warning bells are now big warning bells. Adding a web bug can’t impact your Google adsense revenue. Adding a web bug to your site from a domain name with no website that’s registered by someone in Russia seems like a TERRIBLE idea! So of course I said no.

What do you think? Some sort of XSS attack, or cookie attack? I’m just not sure what the end game would be on this…

Share on Facebook