Unit Testing Not a Panacea

I attempted to use unit testing on my recently finished HMAC implementation in an attempt to use what I have been told repeatedly and experienced first hand.  I first start by noting that when writing in C, unit testing libraries offer little more than a central block of code where tests should be placed.  I looked at some libraries here and they seemed suitable, but for my small HMAC implementation, I just made a function that calls some tests I came up with on the spot.

Here is where my problems began.  Unit tests can only test functionality that you explicitly write.  This is all that a programmer could ask for, unless someone implements some FSA to represent a function.  That would work on a small subset of useful functions (regular languages), but that is certainly a tangent.  I tested some utility functions I wrote by supplying input and the known output.  Example below:
   char *ascii_hex;
   unsigned char *binary;
   ascii_hex = (char *)malloc(4);
   ascii_hex[0] = 'A';
   ascii_hex[1] = 'B';
   ascii_hex[2] = 'C';
   ascii_hex[3] = '9';
   binary = ascii_hex_to_binary(ascii_hex, 2); /* 2 is size in bytes */
   if(binary[0] != 0xAB && binary[1] != 0xC9)
       printf("Error, ascii_hex_to_binary didn't work\n");

I was happy to know that my test passed, but I realized later that ascii_hex_to_binary was not behaving properly.  My test did not check the all possible casess of the input string, especially the one that caused my program to not function correctly. The lesson I suppose would be one test is certainly not enough to check the behavior of a nontrivial function.

No comments: