The BSIMM is an annual study of the real-world software security initiatives – “SSIs” in the report – across the software industry drawing from data and experience from 130 organizations. Rather than repeat the aim of the study, this quote sums it up best:
“The BSIMM is a measuring stick for software security. The best way to use it is to compare and contrast your own initiative with the data about what other organizations are doing. You can identify your own goals and objectives, then refer to the BSIMM to determine which additional activities make sense for you.
… In the rapidly changing software security field, understanding what most, some, and few other organizations are doing in their SSIs can directly inform your own strategy.”
– Executive Summary, BSIMM11.
‘Measuring stick’ is the key term here. BSIMM is a way to measure where you stand and make a plan as to where you want to go. It is a way for software organizations to compare how they are doing in comparison to peer companies and to “discuss, implement, measure, report and improve.”
The BSIMM11 Framework
The BSIMM is organized into domains and security practices which encompass numerous activities that make up the security framework. This is illustrated below:
Source: BSIMM11 – Part Two – The BSIMM11 Framework
The maturity model aspect of BSIMM implies improvement and optimization and, in this case, it outlines key areas of practice that an SSI would fall under and as companies move from an ad-hoc approach to a more strategic one, they move along the maturity scale. In BSIMM these are defined as emerging, maturing and optimizing which, the study points out, isn’t necessarily linear and may not end up in the optimizing state.
For this post, I’m not going to delve into detail on all of these but there are clearly practices where SAST (static application security testing) and SCA (software composition analysis) has a role and then, only briefly – standards and requirements (SR), code review (CR) and security testing (ST).
The Role of SAST and SCA in BSIMM11
Recommendations in BSIMM, make it clear that tools and automation play an important supporting role in security and practice maturity includes more sophisticated use of them. Looking at the Governance-led Getting Started Checklist, it includes number 2, “inventory software”, an important role for SCA, 5, “do defect discovery” which implies detecting and discovering existing vulnerabilities, of which, SAST, SCA and other discovery tools play an important role. Number 6 is “Select security controls” which includes setting secure coding standards and prioritisation on detection and prevention of high-risk security vulnerabilities. Number 7 is “Repeat” which implies automation (including tools), cyclical processes and adoption of DevSecOps, for example, something that all modern tools need to integrate with. Although these are guidelines beyond the use of tools, it’s clear there’s an important role in security practice maturity.
In the standards and requirements (SR) practice, emerging practices include security standards which might imply certain constraints on developed software to reduce vulnerabilities. Maturing practices identify open source usage to determine their risk and exposure. Optimizing companies are using and enforcing secure coding standards, controlling open source risk, and securing their software supply chain.
Consider also the code review (CR) touchpoint: BSIMM notes that the emerging practice is the adoption of SAST to work alongside manual reviews. The maturing practice is the use of tailored rules and organizing target vulnerabilities into a Top N list (like their own OWASP or CWE list.) At the optimizing stage, organizations pursue the eradication of critical vulnerability types, automate malicious code detection and enforce coding standards (all of which SAST plays an important role.) As you can see, maturity in practice coincides with maturity of tool usage.
Inventory of software assets is highlighted in several locations (as above, in the getting started guidelines) as is monitoring and enforcing policies on the software supply chain. For example, third party software including open source should be accounted for as a possible attack surface (AM 1.3). SCA plays an important part in creating a software bill of materials and exposure to known vulnerabilities in the supply chain.
Maturing Companies are Optimizing SAST, SCA
It’s clear that tools play a part in security practice maturity and although it’s really about organizational improvement, the optimal use of tools where they make sense is an important part of this. These companies are effective in increasing the value of their tools and the ROI they receive as their practices mature. The BSIMM points out some themes from companies that are moving towards optimizing their practices and achieving maturity in their software practices. Not surprisingly, there is a role for SAST and SCA in each of these categories (among other tools, of course.)
- Software supply chain management: Mature organizations realize their software isn’t developed in isolation and has dependencies on many third-party software stacks. Understanding the real software bill of materials is essential as is the hidden dependencies within code from outside the organization. As the processes mature, they start to use the same security practices on their software supply chain.
- Top N risk reduction: Prioritizing security improvement efforts is key and maturing companies are creating their own target lists of vulnerabilities. These targeted vulnerabilities can be used to streamline and prioritize how to tune SAST and SCA and which warnings to take action on.
- Tool customization: This speaks for itself, maturing companies are tailoring tools to their needs. In terms of SAST tools, more sophisticated use included customized checkers and rule sets that provide the best ROI on their tool usage.
- Feedback loops: SAST tools provide important metrics plus a general gauge of product quality and security. SAST at the developer desktop provides instant feedback on newly introduced vulnerabilities. SAST and SCA tools in the CI/CD pipeline provides valuable feedback on product-wide vulnerabilities and monitor the evolving software bill of materials. Collection of data provides trends on improvement and areas of concern.
- Data driven governance: Extending the feedback SAST and SCA tools provide, the data created on a per-build basis provides an ongoing barometer of the state of the product. Analyzing this data for trends indicate the relative level of risk from specific types of vulnerabilities and specific areas of the software. Smart teams are using this data to drive the testing and security focus with each iteration. In products that require compliance to specific standards, SAST tools generate required reports to help achieve audit and compliance success. SCA tools help teams monitor the risk of third-party software as new vulnerabilities are detected in their dependent software.
Obviously, as an organization matures in terms of the security practices, their tools use and sophistication increase. They also increasingly use the data from these tools to drive decisions which increase productivity since resources are focused in the right place.
Summary
The BSIMM11 report provides interesting insights into the state-of-the-art security practices in place in the software industry. It also outlines a framework, based on observing companies at each stage of maturity, for organization to follow who are looking to mature their practices. Automation and tools play an important part in supporting more mature processes and companies use tools in a more advanced fashion.
SAST and SCA tools play an important role in software security improvement and the BSIMM shows that increasing tool integration into the security practices as organizations mature. In terms of advanced static analysis, detecting and preventing security vulnerabilities shift-left security improvement right to the developer’s desktop. SCA tools help inventory the software stack and identify areas of risk in the supply chain. Increasing integration and customization of these tools into existing workflows indicates more mature usage.
