DK 'Log

Remember the couple of posts I made back in November in the tuning section ? Well, I finally got the time to look into this issue again and have made some interesting discoveries the last couple of days. I'm really enjoying this.

The following table illustrates some comparisons between a stock Base 1.3.9 (ossim patched) and the tuned rewrite I've got running right now. These optimizations are now part of our appliance offering (updates for already deployed ones on the way) and will be released to the public afterwards. Thanks to everybody that has been helping me on this, specially to the people at #ossim in freenode ;-).

Just for statistical issues, the current contents of the database are (links are broken of course):

Here are the results of some tests I've made, you'll have to trust my word on these :-). The testing host is a Pentium M 2.0Ghz with 1GB RAM, with query caching disabled. If you want me to post some specific test or logs as proof of concept please don't hesitate to ask.

Those numbers are with a couple of days worth of data on a Mysql 5.0.32, not taking other optimizations into account which vastly improve that performance on the applliances such as:

• MySQL Partitioning.
• Multiple discs performance increase.
• Cache hints
• Some more query rewrites

Well, just wanted to share those numbers since forensic database performance has been a real pita for many years and I'm really happy it's starting to work right now on large amounts of data.

Happy spring break to everyone. Aaah, and check the GSOC page on monday, hopefully we're eligible as a mentoring organization for this years summer of code, check out our ideas here.