I’ve just finished committing the first alpha release of ClearCutter to the Alienvault-Garage repository on GoogleCode. It’s a tool born of necessity, for anyone whose spent a good amount of time on those ‘SIEM pre-processing’ tasks, neck-deep in sed, grep,awk, uniq.
Clearcutter (because it ‘clears a forest of logs’) is my work-in-progress combination tool for all those log pre-analysis tasks we here at Alienvault find ourselves doing repetitively. We figure if it’s useful to us, it will be just as useful to other people out there looking to add new device types into OSSIM and find the log entries they need to build effective correlation rules:
Here’s a short planned feature list
- Find Individual Log messages in a log file - ever tried to find all the unique message types in a log sample? this will do it for you.
- Make constructing OSSIM collector plugins easier - assistance in writing regexp’s for OSSIM plugin SID entries
- Find Sequences of Logs in a log file (complete behavioral actions) - need to find all the log messages that indicate a complete user session?
- Produce Sequence configs and process logs into sequence summaries
- Test OSSIM collector plugins against log samples - validate that you have a valid plugin config and show what gets parsed by what SID
- Profile individual regex’s/SID’s against sample logs to see which consume most CPU time - solve those performance mysteries.
You can find the alpha code for clearcutter here or check out via
svn co http://alienvault-labs-garage.googlecode.com/svn/trunk/clearcutter alienvault-labs-garage-read-only
Currently only the first function is implemented,though I’ll be updating this tool rather regularly in-between other things I’m working on, and I’m certainly interested in hearing back from people’s experiences with the accuracy and utility of clearcutter as it progresses. As it stands today, it’s not the fastest tool out there, but Donald Knuth tells me that Early Optimization is the root of all evil, right?
