Session and Memory Leak on Wildfly 10

I recently re-wrote 10MinuteMail, my secure temporary email service, updating it from Seam 2 and JBoss 4.2 to Deltaspike and Wildfly 10.  Unfortunately I noticed a memory leak in the JVM.  During beta testing, the JVM heap usage would slowly grow, over 1-2 weeks until it reached an OOM (out of memory condition).

Memory Leak in the Old Gen on Wildfly 10

I performed analysis on heap dumps taken at various stages of memory consumption and discovered that the issue was HttpSessionImpl objects being held onto by the com.sun.faces.application.WebappLifecycleListener’s activeSessions property.  Since 10MinuteMail does some “manual” session expiration and management, and the new application is AJAX heavy, I figured I was doing something wrong.  But thanks to some great support on the JBoss.org forums, I soon learned that there is a bug in the Undertow sub-system that ships with Wildfly 10.0.0.FINAL which keeps old sessions around, causing a memory leak – 

[UNDERTOW-657] HttpSession never removed from activeSessions – JBoss Issue Tracker

The fix is to replace the Undertow modules that come with Wildfly 10 with the latest stable release versions.  You can find more details here – https://developer.jboss.org/message/959286#959286

Now my application works as expected with no memory leak or abnormal numbers of session objects.

Want an even better job? :)

While the ATG job opportunity at TOMS is pretty sweet, Spark::red is also hiring!  I might be a bit biased but I think Spark::red is an amazing place to work.

First, the jobs:  We’re looking for sys admins, jboss admins, or technical ATG folks (architects, deployment specialists, etc…) to join our sys ops and support/deployment/load testing/performance tuning/troubleshooting teams in Boston (Chelmsford, MA) and Seattle (Redmond, WA).  You can read more about what skills we’re looking for here: Spark::red Oracle ATG Commerce Hosting Careers.  Don’t worry about the which location for which job, we’re looking to fill the roles in either city.

Why would you want to work with us?  Here’s a list:

  • Amazing team of smart, talented, driven, funny people
  • Working with a combination of Fortune 1000s, Internet Retailer 500s, emerging retailers, hip companies, international trendsetters, and Hollywood award shows!
  • Working with happy clients!  We love our clients and they love us.
  • Great salaries – we want to get and keep the best in the business
  • Great benefits
  • Flexibility, power, huge career growth potential!
  • Large scale infrastructure
  • Interesting problems to solve
  • Halo!
  • You get to work with me:)


Seam Identity Management

During a recent coding getaway to Maine (see my post on the 2011 HackFestaThon) I decided to write a basic Seam project as a starting point for my future Seam based web applications.  The idea is to provide common features such as Login, Logout, Registration, Forgot Password, User Management, Audit Logging, Image Upload Handling, Video Upload Handling, etc… so next time I have an idea that I want to hack together I won’t have to re-write or copy-paste in basic functionality like that.

I spent about a day working on things before I discovered that I really should be using the Seam framework’s Identity Management feature.  So I threw out everything I’d done, and started by re-reading the docs, and went from there.  Seam’s Identity Management framework is VERY powerful, but is also a little complicated to get going and in many cases it seems like it would easier to just write stuff from scratch.  I’m banking on the powerful stuff being worth the initial learning curve and a little extra pain.

When I get the starter project in a more complete state I will be open sourcing the whole thing to help others along, but I wanted to share a few things I’ve learned so far:

In order to use the Email address as the login instead of a username, you need to remove the username property from your UserAccount entity and annotate the Email address property like so:

[java] @NotNull
public String getEmail() {
return email;

Actions like Registration need a RunAsOperation inner class to handle the fine grained security controls that the Identity Management framework enforces:

[java] public void register() {
verified = (confirm != null && confirm.equals(password));

if (!verified) {
FacesMessages.instance().addToControl("confirmPassword", "Passwords do not match");
new RunAsOperation() {
public void execute() {
try {
// Check if email address has already been used
if (identityManager.userExists(getEmail())) {
FacesMessages.instance().addToControl("email", "Email has already been used.");
identityManager.createUser(email, password, mFirstName, mLastName);
} catch (IdentityManagementException e) {
// TODO Auto-generated catch block
identityManager.grantRole(email, "member");

// Login the user

Populating custom properties on the User during things like registration requires observing events:

[java] @Observer(JpaIdentityStore.EVENT_PRE_PERSIST_USER)
public void prePersistUser(UserAccount pNewUser) {
// Setup additional UserAccount properties before the user is created
pNewUser.setRegistrationDate(new Date());

You can log audit events with the user’s IP address by doing things like this:

[java] @Scope(ScopeType.EVENT)
public class UserEvents {
private Log mLog;

public void loginSuccessful(UserAccount pUser) {
mLog.info("User logged in with email: #0", pUser.getEmail());
pUser.setLastLoginDate(new Date());
Contexts.getSessionContext().set("currentUser", pUser);
AuditEvent loginEvent = new AuditEvent(((ServletRequest) FacesContext.getCurrentInstance().getExternalContext()
.getRequest()).getRemoteAddr(), pUser.getId(), "Login Success", null);
Events.instance().raiseEvent("auditEvent", loginEvent);

Hopefully I’ll have the starter project ready soon and will share it with you all. In the meantime, happy hacking!

First brush with Ruby On Rails

Earlier this week I was hanging out with a friend talking about a project he was working on and I decided to poke at it a bit with him, and as such got my first hands on experience with Ruby on Rails.

Ruby on Rails or RoR obviously has huge buzz and is a very popular web application development framework lately.  Lots of people have praised it and lots of great sites have been built using it.  I’ve never bothered to learn it myself for a few reasons.  First I’m a Java guy (ATG and Seam) and have been for years.  Given limited time and limited brain capacity I’d rather learn more Java/Java Frameworks/etc… than try to learn a whole new language.  Secondly many trusted friends advised me that while RoR does somethings REALLY well and makes some things REALLY easy, once you need to try to break outside of the pre-imagined structure/features that RoR provides out of the box, things rapidly go downhill.  Those reasons aside, the high level of buzz has meant I’ve always been somewhat curious, so this opportunity to finally get my hands a little dirty with RoR was welcome.

Getting started with RoR on Mac OS X is very easy.  It’s pre-installed and works right out of the box.  However after upgrading Rails and the gems I ran into a known blocking bug which after some Googling I was able to fix by downgrading rake to 0.8.7 in the Gemfile.  So not a 100% smooth start, but not too bad.

The Rails generate scaffold commands make it very easy to create a data object, the related schema changes (managed through the rake migration mechanism), and related CRUD pages and controllers.  You can be up and running very quickly and creating, browsing, editing, and deleting records.  This can make it very easy to get a basic application laid out, and provides lots of plumbing automatically.

I didn’t get much farther than some simple controller modifications, outbound e-mail sending, etc…  so I’m far from a real RoR developer.  However I ran into enough pain points so far that I don’t think RoR is for me.  I don’t want to start any language/framework wars, but here is what I ran into:

A lot of the “magic” seems great at first, but as soon as you want go outside of the box or tweak how things are working, it becomes a massive liability.  For instance when using the generate scaffold command to create data objects you can setup relationships/foreign keys by passing in a column name that matches the form OtherClass_id:integer, which will be interpreted to be a FK association to the other class’s id column to join the objects.  This is great.  However, what if you want to add two relationships to the same Other class?  For instance an Message has a Sender and a Recipient, both of which are Users.  I can use user_id:integer for one, but how do I do the other?  How do I use column/property names that don’t fit that naming convention, for instance I’d want sender_id and recipient_id.  None of the getting started guides I was able to find covered that.  Googling for things like “scaffold multiple foreign keys” didn’t answer the question, etc…  I’m sure it’s possible, but finding out how wasn’t easy, and the default way hides and obscures the actual plumbing so it’s not easy to figure out how to make simple changes or additions.

Data object classes in the app/model area all extend ActiveRecord::Base and as generated by generate scaffold are completely empty.  There’s no clue or indication of the fields, any logic available, any relationships, property types, etc…

Emailer classes use magic mappings between method names and e-mail templates.  Because it’s “magic” I have no idea how to change a template file name if I wanted to.  App/helper classes are created, but they’re empty, so I have no idea what they are doing, or meant to do.  And so on.  If I was an expert RoR developer I’m sure I’d know, or if I read a few books I’d understand, but starting from scratch and trying to learn as I go, it proved very frustrating.

The much touted RoR Community proved to be more of a liability than an asset to me.  Especially when combined with the many, rapid, backwards incompatible, RoR releases that have come out so far.  When looking for information, guides, and answers related to RoR you end up finding things spread all over: blogs, forums, mailing lists, Ruby sites, RoR sites, groups, etc…  Most of these posts/documents refer to older versions of RoR.  Most of them don’t have dates or specify which version they are working with.  Many questions on forums are unanswered.  The end result is you find what you hope is a reasonable solution for an issue only to find that the code sample or directions are written for a previous version of RoR, and trying to follow the instructions or paste the code in the current RoR version results in weird errors or worse.

Each new version of RoR seems to massively change and/or break common APIs and change how things are supposed to be done, etc…  I ran into several situations where it seems like a method was renamed, with no backwards compatible alias left in place, for no other reason than they wanted to change the name, which breaks older code for  no real purpose.

I’m also not a fan of weakly typed languages.  Strongly typed languages provide compile time validation, IDE auto-completion, and easy to navigate API documentation.  With larger, more complex projects, or projects involving many developers, or projects utilizing many 3rd party libraries, these advantages become significant in my opinion.

So from my standpoint, JBoss Seam provides most of the advantages of RoR, including several improvements, without many of the liabilities.  Plus it’s in Java which is my strongest programming language.  I’ll stick with Seam, but I’ll still respect the creations of folks who use other tools, including RoR.


Swedish Translation courtesy of Andrey Fomin:  http://www.pkwteile.de/wissen/foersta-borste-med-ruby-on-rails