TDD & IDAPython


This is just a short note in which I want to share my experiences with writing test code for (IDAPython) scripts I use and produce on a daily basis.

The case for Test-Driven Development

A while ago, I thought it would be a good idea to advance my coding skills. So I had a look around methodologies that are popular in software development but that I had not tried myself.
The best candidate seemed to me Test-Driven Development (TDD) as I was familiar with the concept of unit testing but I was not able to believe that TDD can drive architecture and design decisions.

I started reading the Clean Code series of books by Uncle Bob. The books start out with general advice on how to structure your code in a way it is easier to understand and maintain. I recognize these books as an efficient way to lift your own coding habits to an acceptable level if you plan on publishing code.
In the later chapters the book focusses on TDD. I transferred the given example projects to Python (instead of Java, which the book uses) and really tried hard to embrace TDD as driving method for code generation.

Nevertheless, over trying TDD, I really started liking to have tests for my code for the following reasons:

So I regularly produce “test-covered” code now, instead of “test-driven” code which I’m pretty happy with. Should have done that with IDAscope as well but I’ll add tests for all future bugs I find, I guess.

Tests in IDAPython

So how to use this now in IDA? Here is my template file for writing tests:

import sys
import unittest
import datetime
from test import *

import idautils

class IDAPythonTests(unittest.TestCase):

    def setUp(self):
        pass

    def test_fileLoaded(self):
        assert idautils.GetInputFileMD5() is not None


def main(argv):
    print "#" * 10 + " NEW TEST RUN ## " \
        + datetime.datetime.utcnow().strftime("%A, %d. %B %Y %I:%M:%S") \
        + " " + "##"
    unittest.main()


if __name__ == "__main__":
    sys.exit(main(sys.argv))

In this template we have only one test in our test case IDAPythonTests called test_fileLoaded. Tests to be executed by the Python unittest testrunner use the prefix test_.
Normally you would not test directly against IDAPython’s API as in this example. But would rather test your own code through function calls, with your code usually being located in a different file and imported into the test case.

You can run this as a script within IDA while having loaded a file for analysis. This allows you to specifically test your code against IDAPython’s API on the one hand and using the contents of the file under analysis for verification on the other hand.
The output of the above script while having loaded a file and not having loaded a file to demonstrate the test’s behaviour looks like this:

---------------------------------------------------------------------------------
Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] 
IDAPython v1.5.5 final (c) The IDAPython Team <idapython@googlegroups.com>
---------------------------------------------------------------------------------
########## NEW TEST RUN ## Sunday, 18. November 2012 11:49:29 ##
.
----------------------------------------------------------------------
Ran 1 test in 0.010s

OK
########## NEW TEST RUN ## Sunday, 18. November 2012 11:50:06 ##
F
======================================================================
FAIL: test_fileLoaded (__main__.IDAPython_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Z:/ida_tests.py", line 14, in test_fileLoaded
    assert idautils.GetInputFileMD5() is not None
AssertionError

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)

Pretty much the IDAPython shell we are used to + the nice output from Python’s unit testing framework.

link to original post on blogspot.