As application complexity increases, it is hard to imagine how development could keep up with demand without increasingly sophisticated compilers to translate our high-level ideas to low-level machine code.
But as compilers work to squeeze more and more performance out of your source code, they must make increasingly liberal assumptions about how your code will execute on the actual hardware. Though we think of the compiler as directly translating our commands to machine code, in fact there are many scenarios where the compiler may subtly alter the behavior of a program.
For example, online forums are packed with posts suggesting that GCC’s code generator must be buggy, because of code snippets like this:
#include <stdio.h> int main(void) { int x; int * y; float f = 0.0; y = (int*) &f; x = *y; printf("The binary representation of %f is 0x%xn", f, x); } |
Compiling with GCC 4.3.4 gives the following results at optimization levels 0 and 3:
$ gcc binrep.c -o binrep-O0 && ./binrep-O0 The binary representation of 0.000000 is 0x0 |
$ gcc binrep.c -O3 -o binrep-O3 && ./binrep-O3 The binary representation of 0.000000 is 0x61242fce |
Now, surely the IEEE didn’t change the standard representation of floating point numbers between our two compilations — so what is going on here?
The key to this issue is the C language’s often misunderstood strict pointer aliasing rules (leading to this MySQL bug, for example: http://bugs.mysql.com/bug.php?id=48284). In essence, this rule says that the compiler has the right to assume that if two pointers have a different type, then they do not point to the same location. The rules allow the compiler to make many optimizations to interleaved sequences of reads and writes to memory which may seem obvious at the source level but cannot be justified without the strict aliasing assumption. A detailed discussion of strict aliasing in the context of PowerPC machine code can be found here.
When GCC is at optimization level -O3 or with -fstrict-aliasing enabled, it argues to itself that the couplet
y = (int*) &f;
x = *y;
|
only refers to int-containing memory. Since int s aren’t float s, then it doesn’t matter if these instructions occur before or after the assignment to f — by the strict aliasing rules, they belong to entirely different universes. So these instructions are moved up to before f is initialized, causing x to contain a garbage value.
Of course, one man’s trash is another man’s treasure. The garbage value sitting in x before main() was called most likely came from the startup code, and could potentially contain some interesting information — maybe not the kind of information that we want to shout to the world. It is not too hard to see the security implications: even though the source code appears secure, the actual machine code which gets executed could potentially leak sensitive information into the world. [ Compare machine code ]
In fact, this exact class of bug bit Microsoft during their 2002 security push. Consider this fake bit of code to perform a secure operation:
void press_the_secret_button(void) { volatile char password[128]; ask_user_for_password(password); press_the_secret_button_using_credentials(password); memset(password, '#', 128); } |
This simple bit of code asks a user for their password, then uses that password to perform some secure operation. To ensure that the user’s password is not retained in memory, it cleans up by redacting the password buffer, overwriting everything with pound signs.
Or at least, that is what the source code suggests. But let’s have a look at the generated machine code, this time using Microsoft’s cl compiler at optimization level /O2:
_press_the_secret_button: ; Set up the stack frame and reserve space for password[] sub esp, 84h mov eax, dword ptr ds:___security_cookie xor eax, esp mov [esp+84h+var_4], eax ; ask_user_for_password(password) lea eax, [esp+84h+var_84] push eax call _ask_user_for_password ; press_the_secret_button_using_credentials(password) lea ecx, [esp+88h+var_84] push ecx call _press_the_secret_button_using_credentials ; Clean up the stack and return mov ecx, [esp+8Ch+var_4] add esp, 8 xor ecx, esp call @__security_check_cookie@4 add esp, 84h retn |
Notice that the call to memset() that was supposed to redact the password buffer has vanished from the function. The compiler has reasoned that the password[] buffer is never referred to after the call to memset(), so that call must be dead code. The “dead” redacting code is then removed, leaving the user’s unencrypted password in memory for an indefinite amount of time.
At GrammaTech, we have a baroque acronym for this scenario: What You See Is Not What You eXecute (WYSINWYX). It reminds us that analyzing the source code of a program may not be a sufficient guarantee of safety. Since the compiler may alter the behavior of a program in subtle ways, the safest bet is to directly analyze the generated binary code. Only then will you be analyzing exactly what the computer will be executing.
