April 12th, 2018

Things That Don't Scale: Entry #42

There are some approaches in programming that simply don't scale.

One of them, I am now reminded after running a bunch of timing tests, is reading large XML data using DOM. My only consolation is that I was not the one who decided that using DOM was a good idea.

I have suggested using a nice serial forward-only XML reader.

And that someone else can fix up the code to do so. :)

Hack and Slash

We've picked up a highly dimensional client file for testing here and have discovered that the calculations are running a good bit slower than they were in older versions of the product. Part of this is due to the fact that I eliminated a number of direct pointers in favor of map lookups, because I was told that maps would be very, very fast in our Java code. Well, maybe they are faster in Java than they are in C++, but they are still not fast enough in either case, because the calculations are way too slow.

So I've been putting back in various bits of caching in both the C++ and the Java code. With nothing cached, the calculations were taking a minute and 47 seconds. Caching two data items cut the time to 1:25, which was respectable, but still slower than I had in mind.

I have now cached a third bit of interesting data and have cut the calculation time down to 57 seconds.

This is good.

Tomorrow, I'll finish this up and try caching a fourth and final bit of data and we'll see where we end up. :)