CodeSonar’s users can sometimes come up with very interesting code properties to check. Last week I had a conversation with a customer that led to my writing a plug-in for CodeSonar described below.
The customer began by giving me a link to an article titled Data analysis of GitHub contributions reveals unexpected gender bias, which describes a study of how code contributions from men and women are treated differently. The researchers found that women’s contributions were accepted more often than men’s, but only if the gender of the contributor was unknown to the reviewer.
The customer emailed me after reading the study and coming to the not-unreasonable conclusion that women programmers tend to be more competent (or at the very least, more reliable) than men, and was wondering if we could take this into consideration in CodeSonar.
From the email: “I have a lot of source code that I have to review in detail, and I don’t know anything about the team that wrote it. Can we create a metric in CodeSonar that will tell me for each function, the likelihood that the author was a man? That way I can subject those functions to extra scrutiny because they are likely to have more bugs than those written by the women.”
Distinguishing “Maleness” of Code
I had heard that men and women write prose differently enough that it is possible to use statistical techniques to distinguish the styles, but I had never heard that the source code of programs could be used in a similar way. Fortunately, the customer was able to point to a helpful article that found that men and women choose different names for variables, and there is enough signal in most non-trivial functions to determine the gender of the author with a confidence of 95% [1].
It turns out that the test is fairly easy: genders use names that have different frequencies of letters. The greatest discrimination is that men select the letters “afilopr” much more often than women. The author of that study offered no theory as to why this should be, but I’d encourage readers to follow the link in the reference below to read how the study was conducted.
Using the findings of the study, it turns out that computing this metric in CodeSonar is fairly easy, so I wrote a small plug-in to do so. (CodeSonar allows users to author custom plug-ins in C, C++, Python, C#, Java and Scheme. I usually prefer C++, but this time I used Python for fast turnaround.) The full listing is provided at the end of the article.
Writing a CodeSonar Plug-In to Calculate “Maleness”
The entire program model is available through CodeSonar’s API, and most-plugins use a visitor pattern. Typically an author writes a callback that is invoked when CodeSonar’s analysis engine is traversing the program model. In this case the metric is a property of a procedure, so it could be written as a procedure visitor.
from cs import *
@procedure_visitor
def visit_proc(proc):
…
The shape of such a visitor is shown above. The first line imports the CodeSonar API, the second is a decorator that establishes that the following definition is the callback, which is given on the subsequent lines.
To tell CodeSonar that the plug-in is introducing a new metric, one writes:
met = procedure_metricclass_manager.create('Maleness',
'Degree of Masculinity')
The first parameter is the short name for the metric, the second is a longer more descriptive name.
With these in hand the procedure visitor can then compute the metric for the entire function by iterating through the set of names in the function (we look at local variables only; global variable names are less likely to be controlled by a single author which makes them more noisy), and by counting the number of characters.
syms = proc.local_symbols()
for sym in syms:
if sym.get_kind() == symbol_kind.USER:
name = re.sub(r'-[0-9]+$','',sym.name())
nchars += len(name)
nspecials += count_specials(name)
The first and second lines retrieve the symbols and start the iteration. The conditional is necessary because we don’t want to include special CodeSonar synthetic symbols (e.g., temporaries introduced by normalization) in the metric. The call to re.sub() removes any trailing label that CodeSonar may have added to the name (this is used to disambiguate variables with the same name in different subscopes).
Once the loop is done, it is easy to compute the final value of the metric and report it:
if nchars == 0:
maleness = 0.0
else:
maleness = float(nspecials)/float(nchars);
met.report(proc, maleness)
A one-line addition to the config file for a project causes the plug-in to be executed:
PLUGINS += maleness.py
The standard procedure view in CodeSonar then shows the metric:
I gave this to the customer who was pleased, but then wanted a warning to be issued if the metric exceeded a given threshold. This was easily done too.
Adding a “Maleness” Warning to CodeSonar
First I needed to add a warning class:
wc = analysis.create_warningclass('Function too Male')
With that, I could then test the value of the metric and report the warning if the condition is satisfied:
if maleness > 0.3:
wc.report(proc.entry_point(),
"This function is too male. " +
"It's maleness value is %5.2f" % maleness)
The screenshot shows an example warning report in CodeSonar. This particular function was clearly written by a man. He even left his name behind on line 346, alongside clear evidence that he blatantly chose to ignore a potentially serious bug. (Perhaps a female programmer would have done a better job)
If you’ve read this far, I expect you’ve realized what date it is today. Of course, the real point of this article is to demonstrate how easy it is to write a plugin for CodeSonar to extend its functionality, even if that functionality is a bit daft. Users who extend their static analysis tool can greatly increase the value it provides, so we encourage all our users to do so, regardless, of course, of gender.
Reference
[1] Poisson, d’Avril. Gender Differences in Variable Name Choices among Programmers. 5th International Conference on Computer Programming, 4/1/2014, Strabane, Northern Ireland.
Full listing:
