[fuzzing] Commercial Fuzzers

[J. M. Seitz]

>So one of the other interesting things you've mentioned a few times ...
>are many of tools out there ready to go for the average QA guy or is there
almost always customization that needs to take place.  If so, what's the
typical nature of such 
>customizations?

For the most part the tools are ready to go in that you can fire them up,
give them an IP to blast away at, set your fuzzing points and have at it.
What I have found, from a QA perspective, is that you want a few things to
be at your disposal:

1) Something to catch the crashes, but do it in an automated fashion. Pydbg
is a great example of this, as you can have the application crash, do the
core dump, and then respawn it to continue with your next test case. After
you are finished your testing run you have an accurate picture of where the
problems lie.

2) Instrumentation - its incredibly useful to have the ability to watch
thread counts, file handle counts and memory usage.

3) Repeatable - most of the tools I have used are repeatable which is
awesome. A large part of testing in QA is being able to regression test and
ensure you aren't causing new bugs because of fixing old ones. A
pseudo-random fuzzer like GPF does a great job of this by providing a seed
value when you fire it up. Most of the other fuzzers that I have used
require static test cases which work well for this aspect.

Now granted you can always attach to a process with WinDbg and leave Taskman
running and you get the same thing, but generally when you are doing these
tests you want to be doing something else which requires human eye like
functional testing, etc. You never have the time to play catch-and-release
with breakpoints all day :)

JS