next up previous
Next: Evaluation Up: Protecting from stack-smashing attacks Previous: Related work


Stack Protection Method

As described in the previous section, there are four areas that should be protected from a stack-smashing attack: the location of the arguments, the return address, the previous frame pointer, and the local variables.

A guard variable is introduced for preventing change in the first three areas. This technique was originally devised in the StackGuard project [3] and it is designed to insert the guard immediately after the return address. The difference in our method is the location of the guard and the protection of function pointers. The guard is inserted next to the previous frame pointer and it is prior to an array, which is the location where an attack can begin to destroy the stack.

We'll illustrates the method using a source code translation only for the purpose of a concise explanation. Given the source code of a function, a preprocessing step will insert the following fragments of code into the appropriate positions: declaration part of local variables, the entry points, and the exit points respectively.

The modified source program appears as Figure 3. Note that the guard must be located before a string buffer. The entry statement stores a value that is not known by the attacker and the exit statement verifies it. Unless the guard is preserved, it will cause an alert to the system and record information in the logging database.

Figure 3: After the Protection Code has been Added
\begin{figure}\centering\begin{verbatim}void bar( void (*func1)() )
void (*...
...uf, getenv (''HOME''));
(*func1)(); (*func2)();

We introduced the source code translation to explain how the protection is done. In fact, it it difficult to implement, because a function usually has many exit points and the guard must be located before the string buffer on the stack. We have implemented our system using the intermediate code translator of the gcc compiler. According to the specifications of the intermediate language, there is only one exit point per function and the location of each variables has been decided.

T main requirement of the guard value is that it must be a value that an attacker can't know. If he knows the value, the attacker could fill the part of the stack between the ``buffer'' and the previous frame pointer with the value. This would pass the verification test of the guard value, and then he could change the return address.

We chose a random number as the guard value. The random number is calculated at the initialization time of the application, which cannot be discovered by a non-privileged user. The number must be an unpredictable random number. Linux has a random number generator. It is implemented as a device, which name is /dev/urandom or /dev/random. It uses environmental noise to generate the number; thus, it is suitable to supply an unpredictable number.

Safety Function Model

We introduce a safety function model, which involves a limitation of stack usage 4 in the following manner:

Figure 4: Safe Frame Structure
\begin{figure}\centering \begin{tabular}{l\vert c\vert l}
& &  \cline{2-2}
...k pointer$\longrightarrow$ & & $\downarrow$ \\

This model has the following properties:

  1. The memory locations outside of a function frame cannot be damaged when the function returns.

    The location (B) is the only vulnerable location where an attack can begin to destroy the stack. Damage outside of the function frame can be detected by the verification of the guard value. If damage occurs outside of the frame, the program execution stops.

  2. An attack on pointer variables outside of a function frame will not succeed.

    The attack could only succeed if the following conditions were satisfied: (1) the attacker changes the value of the function pointer, and (2) he calls a function using the function pointer. In order to achieve the second condition, the function pointer must be visible from the function, but our assumption says this information is beyond the function scope. Therefore, the second condition can't be satisfied, and the attack will always fail.

  3. An attack on pointer variables in a function frame will not succeed.

    The location (B) is the only vulnerable location for a stack-smashing attack, and the damage goes away from area (C). Therefore, the area (C) is safe from the attack.

Pointer protection

Figure 5 shows another vulnerable function, which an attacker may use to take control by replacing the function pointer with the address of attack code.

Figure 5: Attack Against a Function Pointer
\begin{figure}\centering\begin{verbatim}void bar( void (*func1)() )
void (*...
...uf, getenv (''HOME''));
(*func1)(); (*func2)();

In order to protect function pointers from stack-smashing attacks, we change the stack location of each variables to be consistent with the safe function model.

The "C" language doesn't have restrictions on the order of local variables, but it doesn't allow changing the location of arguments. The restriction on arguments can be addressed by making a new local variable, copying the argument ``func1'' to it, and changing the reference to ``func1''to use the new local variable. Figure 6 shows the result of the translation.

Figure 6: Protection from Function Pointer Attack
\begin{figure}\centering\begin{verbatim}void bar( void (*tmpfunc1)() )
...uf, getenv (''HOME''));
(*func1)(); (*func2)();


In this section, we'll discuss optimization techniques for reducing the overhead of the guard implementation. The following assumptions are introduced for that purpose. When these assumptions are met, some guard code can safely be removed.

Assumption 1
The source code has the proper type declarations and follows the type conversion rules.

For example, an integer variable always stores integer values, and does not store string values. Such erroneous operations often causes memory protection errors during the execution of a program.

Assumption 2
Only character arrays can cause the buffer overflow.

The buffer overflow happens during the assignment operation to an array without a boundary check. The operation commonly uses a terminator for the detection of the boundary. Most terminators are used only by string functions; examples are a null character or a newline character for the gets function.

If Assumption 2 is met in addition to Assumption 1, the guard protection can be omitted except for the functions that declare string buffers as local variables or as arguments. This is expected to reduce the overhead, especially for numerical processing, because those functions usually use numerical arrays and a few functions are called many times in a short period.


The stack protection method is achieved by the program translation, which converts a vulnerable function to a non-vulnerable function. The conversion can't be always successful in some cases.

Comparison of Protection Techniques

Figure 7 shows the comparison of protection techniques, which prevent to pass the control of execution to code that has been attacked.

Figure 7: Summary of Detection Technique Characteristics
% latex2html id marker 358
\centering \begin{tabular}{\vert l\v...
...mic link library
\item[5] Intel processor only
\end{description} \end{figure*}

The protection effectiveness describes the protection coverage of the protected region for the stack and the coverage against string functions causing buffer overflow. The entry marked ``No'' in the protected region means that the particular region cannot be protected and vulnerable programs could be found in the future. For example, the exploit program for the program called ``SuperProbe 2.11'' attacks the function pointer variable pointed at from the argument variable, so StackGuard cannot block the attack and the attacker can gain access to the root shell.

Libsafe doesn't protect all of the string functions, nor does it from protect from the string operations that are using pointers directly; for example, the exploit program for the program called ``xterm distributed with RedHat5.2'' uses a buffer overflow in the tgetent() function of libtermcap.

The implementation characteristics shows that only our system provides a protection mechanism for a variety of operating systems and processors.

next up previous
Next: Evaluation Up: Protecting from stack-smashing attacks Previous: Related work