MSSQL, jTDS, NVARCHAR and Slow Indexes

Mr. Slow

Mr. Slow by Vox

An application I’ve built is going into production soon. It’s the first application I’ve been involved with which will be using MSSQL server in production, I have some learning about MSSQL to do. After some research, I ended up using the jDTS JDBC driver instead of the Microsoft JDBC driver (which is feature incomplete and has some serious open issues).

We recently began performance testing and saw some odd behavior. Initially the application was performing well. However after few runs of the stress test the performance went from good to awful. The main web service call went from 600 ms to 23,000 ms. The database server’s CPU was pinned, and the app servers were barely loading, spending all their time waiting for the database server to return queries. Stranger still, my local instance (running against PostgreSQL) performed well with the same code and same stress tests. Luckily a smart MSSQL DBA was able to figure out why the database was burning so much CPU and responding so slowly.

One of the primary queries is against a table which has been growing. The select query is simple and had an indexed column in the WHERE clause. Even as the table grew to over a million rows, it should have been a very quick query. Unfortunately it was taking several seconds to complete. My local instance had over 30 million rows in the same table in PostgreSQL and the query was lightening fast. The DBA discovered that the query execution was converting the indexed varchar column into nvarchar values for comparison with the query parameter used in the WHERE clause which was inexplicably coming over as an nvarchar. This datatype mismatch between the column definition and the query parameter meant that MSSQL was doing a scan of the million+ record index instead of the almost instant seek it should have been doing.

It turns out that jTDS sends String parameters over as Unicode (nvarchar) by default. It’s easily fixed by adding this property to your connection-url:

sendStringParametersAsUnicode=false

That immediately fixed the performance issues.

So, if you’re using jTDS and are using indexed varchar columns in your queries, you should add the property above to avoid your indexes being wasted and your queries running slowly.

Oracle Export (exp) and Initial Extent Size Issues

If you have a large database in Oracle, with a tablespace with say 2 gigabytes worth of data in it, and you then go in a delete a large number of rows from a large number of tables, and shrink it down to about 300 megabytes worth of data, and then you create an Oracle export using exp, you might expect you could then import this Oracle dump file, into another database, and have it take up 300 MB.

You’d be wrong.

The dump file ends up with all of the create table and create index commands using an INITIAL extent storage setting based on the size of the old table at its fullest. So when you run the import of the dump file, it basically eats up 2 gigabytes of tablespace for 300 MB of data. You can’t edit the INITIAL values in the dump file, since it’s binary, and if you edit it, you corrupt it. Oracle doesn’t seem to have any great ways to fix this, so here’s my hack:

  1. Do the full export, with compress=n (this is useful regardless).
  2. Generate a create tables script (I used my SQLDeveloper GUI client) that just creates the tables (no INITIAL settings)
  3. Generate a create constraints script (I used my SQLDeveloper GUI client) that just creates constraints/indexes
  4. Run the create tables script on the new database
  5. Run the import with these options: ignore=yes constraints=no indexes=no
  6. Run the create constraints script

Now you have a 300 MB database. If you export from this, you end up with an export file that will create other 300 MB databases and you can share it with your friends.

Good luck!

P.S. Oracle DBAs might have a better way of doing this. I don’t know.