Our partner, Vector Software, recently announced the official release of the VectorCAST and GrammaTech CodeSonar integration. This prompted this post to discuss the role of static and dynamic analysis in augmenting automated unit testing as a way to really push the emphasis on quality and security early in the lifecycle – make the “shift left” really happen. We discussed VectorCAST and CodeSonar integration before it was released in another post, check this out to get an overview of how the integration works.
What does it mean to “shift left” quality and security? The goal is to start analyzing and testing code as soon as possible – moving it closer to the beginning of the project which is the left side of the timeline. The motivation for this is to find and fix defects as early as possible, the sooner they are found, the cheaper they are to fix. Bugs found later in development or worse, in a delivered product, are much more expensive to fix, up to a 100 times more expensive. This post looks at how static and dynamic analysis help automated unit testing and, together, help shift testing for quality and security earlier in the development timeline.
Advantages of Automation in Unit Testing
Most developers agree that unit testing is a good idea but they don’t tend to do it. They often don’t have the time or motivation to do so and manual unit testing is time consuming and tedious. Also, unit testing requires stubbing and mocking which is an added workload. Unit tests are themselves code and add to the maintenance workload since changes in the unit under test can mean changes to the unit test code.
Automation is essential to remove the tedious, manual and error prone aspects of manual testing. Automation tools such as VectorCAST provide the test drivers and generate stubs and mock functions needed to isolate the unit under test. In addition, code coverage metrics are collected and along with test results are stored for later reporting and analysis. Increasing code coverage is increased by iterative improvements to test cases over time.
The Role Static and Dynamic Analysis in Automated Unit Testing
Static and dynamic analysis tools can be used at any time during development. However, they are particularly useful during code development and unit testing phase.
Static analysis tools are particularly good at enforcing coding standards and detecting bugs in the code that testing has missed. . For projects with established coding standards, static analysis automates most of the work involved, reducing the need for manual inspection. Unit testing is usually focused on verifying desired behavior and misses security vulnerabilities and tricky edge cases – something static analysis tools can detect. Static analysis tools can also detect poor coding practices and dead code which improves code quality and helps increase code coverage.
Dynamic analysis tools only work when code is running which is true during unit testing. Unit test frameworks only validate tests based on expected results even though other serious bugs have occurred. Dynamic analysis can detect serious runtime errors that occur when unit tests are run that aren’t detectable by the test automation software. Detecting these errors during unit testing means less errors in integration and system testing, saving time and effort.
The Value of an Integrated Comprehensive Solution
The real value of individual tools isn’t realized until they are integrated together. In that case of unit testing it’s critical that bugs found from dynamic and static analysis be reported alongside unit test results. The combined list of issues can be delivered to the appropriate developer to be worked on together. Rather than investigating reports from different tools in different user interfaces it’s more productive to work from one location. An example of this is the VectorCAST and CodeSonar integration – two best of breed tools brought together to provide an improved developer experience.
CodeSonar results integrated into VectorCAST
