Why mock?
Regardless of whether you buy into Test Driven Development’s test-first approach, the practice of automated unit testing is generally viewed as something that improves software quality. Oftentimes there are portions of a software system that are relatively easy to unit test: typically reusable libraries and other modules on the fringes of the system’s functionality. However, it can be difficult to test the core functionality of a system, legacy code that was not written with testability in mind, or code that is tied to hardware behavior. Isolating functionality into small, testable units is not always easy in these cases.
Some of these problems could be resolved by refactoring the code, but realistically this is not always possible (think: obstacles like change management processes, time involved to refactor, risk in refactoring code that is not currently unit tested…). Non-object oriented languages (like C) can be particularly challenging to unit test, since the language does not provide interface primitives to easily transition between real and test harness code. Enter mock functions, the procedural language version of mock objects (“Endo-Testing: Unit Testing with Mock Objects”, Mackinnon, et. al.), that can help facilitate testing these hard-to-reach portions of your code.
In a procedural language like C there are a couple of difficulties in separating out units to test:
- Definition (and maintenance) of global variables and functions so that a module can be compiled and linked independently of the larger software system.
- Isolating units to be tested from the behavior of the functions that they call.
Mock functions are a technique for solving the problem of isolating a unit under test.
What is a mock function?
A mock function is an extremely simple function that has its behavior controlled by the unit test. cmockery provides the simplest approach that I have seen: the unit test simply states what each procedure will return during this test, for each invocation. The test code essentially provides a queue of values that the mock function reads from when it needs a return value.
Rather than generate a small artificial example, let’s take a look at some real software. The below figure shows CodeSonar’s visualization of the open source bash shell program (version 4.1), with the edges representing calls between functions.
We can see at the center there is a lot of core functionality inside bashline.c. Let’s just pick a function, command_subst_completion_function(), to demonstrate how mock functions help us unit test. The figure below shows the CodeSonar visualization of the forward callgraph of our chosen function. Most of the directly called functions are API functions, except for rt_completion_match() and test_for_directory(). We want to isolate command_subst_completion_function() from the behavior of these other functions, so that (1) we can test the smallest unit possible and (2) our unit test doesn’t have to link to the actual implementations of these functions (which may introduce more dependencies, and so on, until you are building your entire program).
test_for_directory() is actually a file static function, so we cannot mock it – we have to mock the two functions it calls, file_is_dir() and bash_tilde_expand() (both of which reside in files separate from our function). Below is how we might mock these with cmockery:
char ** rl_completion_matches(const char *text,
rl_compentry_func_t *entry_function)
{
return (char**)mock();
}
char * bash_tilde_expand(const char *s, int assign_p)
{
return (char*)mock();
}
int file_isdir(char *fn)
{
return (int)mock();
}
If any of these functions had ‘out’ parameters, we would set those to mock() as well. mock() simply denotes that the value returned will have been specified by the unit test that resulted in calling the function. Remember, we are only interested in testing the functionality in command_subst_completion_function(), not the functions that it calls.
The simplest test for our function might be to have no command completion matches, no tilde expansions, and no directories. Without understanding any of the details of bash‘s code, we would expect such a test to look like this:
static void test_command_subst_completion_function_simple( void **state )
{
// Our command to 'complete'
const char *our_command = “do_something”;
char *result;
// No completion matches for this test
will_return( rl_completion_matches, NULL );
// No tildes to expand
will_return( bash_tilde_expand, strdup( our_command ) );
// No files that are directories
will_return( file_isdir, 0 );
result = command_subst_completion_function( our_command, 0 );
// Since we didn't provide any expansions, we should get the same command back
assert_true( strcmp( result, our_command ) == 0 );
}
We have now “modeled” the behavior of the functions we depend on in the simplest way possible. To get good test coverage of command_subst_completion_function(), we would just continue generating test functions that provide different mock values.
Wrapping up
There are lots of other unit testing frameworks that support mocking (many of the ones from this huge Wikipedia article do), but hopefully these simple examples with cmockery have demonstrated why mock functions are useful for testing real programs.
Testing at this small granularity reduces the complexity of the behavior being observed, making it both easier to get good test coverage (inputs and outputs have much more straight-forward relationships at the unit level) and to find the cause of bugs (since the needle is in the very small haystack of the unit under test).
To quote one of the conclusions from A Survey of Unit Testing Practices[citation], “It’s hard to create a relevant test environment for modules interacting with a complex system state or a complex system environment.” Mock-based unit tests provide a way to alleviate this difficulty by isolating program functionality, though there is still a price to be paid in maintaining the compile and link-time dependencies that come along with a typical C program.
Interested in the differences between free static analysis and advanced static analysis? Check out our guide to “Advanced Static Analysis for C++” here:
{{cta(’42fd5967-604d-43ad-9272-63d7c9d3b48f’)}}
