[fuzzing] OWASP Fuzzing page

[Charlie Miller]

I definitely don't know how to do it at this point, but I do agree we
need to target the implementation rather than the specification.  Do you
use the exact same test cases when testing sendmail, postfix, exchange,
and some third party mail server?  Or wouldn't you want to maybe tailor
the cases toward some of the specific differences between the
implementations?  If you limit your fuzzing to the specification, you
will only find bugs in the code of the target that deals with that
specification.  What if besides "GET", MS has some proprietary command
"MSGET" that it didn't bother telling anyone about?  (And MS would never
have undocumented features)  You'll obviously never test the code that
handles this command by following the HTTP spec.  And these are the
things most likely to have bugs, since nobody ever uses them.  Likewise,
what if they totally botched the implementation and while you are busy
being RFC compliant (or when fuzzing, almost RFC-compliant) they are
kicking out your test cases because they expect something slightly
different?  What if they didn't bother implementing some of the RFC,
you'll waste time fuzzing that (not a huge deal)

Ideally, you could take a target application, and have some tool which
RE's the thing and generates test cases for you based on the executable
(implementation) rather than the specification (what it claims to do).
I don't know how to do this or I'd be selling it.

Sure you can fuzz for the low hanging fruit, and probably be very
successful for a time.  But at some point, you're going to have to do
better and actually look at the program beforehand or at least analyze
how the program is reacting to your test cases.

Charlie