Understanding Buffer Overflow

Hardik Shah

The author explains how the register plays an important role in buffer overflow and the main cause for that is the EIP. Here we see how buffer overflow can influence the strings in making the program a big failure

This document is for educational purposes only and explains what a buffer overrun is and shows how they can be exploited on the Windows operating system.  Developer IQ nor me can be held responsible for misuse of the information provided in this article.

There are many hacking techniques. But one of the most popular is buffer overflow. This method is used by hackers to gain illegal access to the servers and most of the servers are exploited using this technique. In this article I try to show you what a buffer overflow is and how it can occur? We also take a look on how it works? We take a detailed look at the system level.

A Little Bit of Introduction

A buffer overrun will occur when a program allocates a limited memory of a certain fixed length and then tries to put too much data into the buffer which is more than the length of the buffer, hence overwriting critical information related to program.

Consider the following c-program:

Void main()
{ char name[8]; gets(name}; }

Now it is clear that, here no limits can be defined by the function gets(). The only terminating condition is an enter key or “\n”.
So, until a user does not press the enter key, the program takes the value into the array name[]. Now we declare the array as 8 characters long so that only 8 bytes allotted to this array in memory. Now the 8th byte of this must be NULL character or ‘\0’. This is the way c-language handles any string. The memory structure of the program is as shown in Fig 1.

This is a simple memory structure for an executable c-program. Now it is clear that if an user has no knowledge of the limit of the string to be entered and if he enters the string which is more than 7 characters, then buffer overflow may occur and it is possible that some critical memory location gets overwritten and program or operating system may show the incorrect behavior.

Looking into this problem at system level

We know that each processor has certain set of register associated with it. Like EAX, EBX, EIP, ESP, EBP etc. (here I assume that readers have a fair amount of assembly knowledge) EIP register is called instruction pointer (E means extended). This stores the instruction, which is next to being executed. EBP stores the base address of stack, ESP points to the Top of Stack. Now, when a buffer overrun occurs, it is possible that the contains of this register get overwritten (we shortly see how). If such a case occurs, then because EIP contains the instruction which is next to be executed, processor will look into the EIP register; but as the contains get overwritten, it contains a wrong instruction. Therefore program will crash.

Suppose in the following program if a user enters the string such as AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA, then the contains of EIP will get overwritten with the equivalent ASCII value of character ‘A’ which is 41. Hence the EIP register will contain 0x41414141 because EIP is 32 bits long. That’s why we can only store 4 A’s or 0x41414141. The processor then goes to address 0x41414141 and tries to read in the instruction found at that address. If there's no instruction, then we get our well-known Access Violation message popping up saying something like "The Instruction at '0x41414141' referenced memory at '0x41414141' performed an invalid operation. The memory could not be read."    

Another example:

Consider the following c-program with a function:

void fun(int a, int  b)
  char buf[10];
  // <-- the stack  is watched here
void main() { fun(1, 2); }

Now, when we execute this program, it will cause an access violation. Let’s have a detailed look on the inner working of this program.

When a procedure is called, the return address for function call, which the program needs to resume execution, is put into the stack. Looking at it from the attacker’s point of view, this is a situation of key importance. If the attacker somehow managed to overwrite the return address stored on the stack, then on termination of the procedure, it would be loaded into the EIP register, allowing any overflow code to be executed instead of the process code resulting from the normal behavior of the program. Let’s see how the stack behaves after the above program has been executed.

After the function fun() is entered, the stack looks like the illustration in Fig 2.

Here the function arguments are pushed backwards in the stack (in accordance with the C language rules), followed by the return address. From now on, the function fun() takes the return address to exploit it. fun() pushes the current EBP content (EBP will be discussed further below) and then allocates a portion of the stack for local variables of fun(). Here a user can directly modify the contains of the ESP register just like ADD ESP, 08H etc.

Now, as EBP register is a basic register that points to the stack bottom, it contains the address of the stack bottom as an offset, relative to the executed procedure. Each time a new procedure is called, the old value of EBP is the first to be pushed onto the stack and then the new value of ESP is moved to EBP. This new value of ESP held by EBP becomes the reference base to local variables that are needed to retrieve the stack section allocated for function call. Since ESP points to the top of the stack, it gets changed frequently during the execution of a program, and having it as an offset reference register is very cumbersome. That is why EBP is employed in this role.

Now it is clear that the register which plays an important role in buffer overflow is EIP. Thus buffer overrun deals with modifying the contains of EIP in a controlled manner. So, after reading the above description we can figure out the general algorithm for buffer overflow as follows:

1. Finding a code, which is vulnerable to a buffer overflow.

2. Determining the number of bytes to be long enough to overwrite the return address.
3. Calculating the address to point the alternate code.
4. Writing the code to be executed.
5. Linking everything together and testing.
Let’s look at an example for this. Here we have a program, firstly, we find the possible exploit and then exploit it.
The following code listing is an example of a code, which is vulnerable. Save this as victim.c (Code)

  #define BUF_LEN 40
  void main(int argc,  char **argv)
  char buf[BUF_LEN];
  if (argv > 1)
  printf(„\buffer  length: %d\nparameter length: %d”, BUF_LEN, strlen(argv[1]) );
  strcpy(buf, argv[1]);

This sample code has all those characteristics, which may cause a buffer overflow: a local buffer and an unsafe function that writes to memory.

Now using our new knowledge, let’s do a simple hacker’s task. As I discussed earlier guessing a code section potentially vulnerable to buffer overflow is simple. The use of a source code (if available) may be helpful; otherwise we can just look for something critical in the program to overwrite it. One approach will focus on searching for string-based function like strcpy(), strcat() or gets() in the disassembly (yes you can relate software reversing here). The common feature of such function is that they do not have any limit, i.e. they copy characters until a NULL byte is found. If, in addition, these functions operate on a local buffer then there is the possibility to redirect the process execution flow to anywhere we want. Another approach would be trial and error, by entering an inconsistently large batch of data inside any available space. Consider now the following example:


If the program returns an access violation error, we may simply move on to step 2.

The problem now, is to construct a large string to effectively overwrite the return address. This step is also very easy. Remember that only whole words can be pushed onto the stack. We simply need to construct the following string:

If successful, in terms of potential buffer overflow, this string will cause the program to fail with the well-known error message:

The instruction at “0x4b4b4b4b” referenced memory at “0x4b4b4b4b”. The memory could not be “read”.
The only conclusion to be drawn is that since the value 0x4b is the letter capital “K” in ASCII code, the return address has been overwritten with “KKKK”. Therefore, we can proceed to Step 3. Finding the buffer beginning address in memory (and the injected shell code) will not be easy. Several methods can be used to make this “guessing” more efficient, one of which we will discuss now, while the others will be explained later. In the meanwhile we need to get the necessary address by simply tracing the code. After starting the debugger and loading the victim program, we will attempt to proceed further. The initial concern is to get through a series of system function calls that are irrelevant from this task point of view. A good method is to trace the stack at runtime until the input string characters appear successively. Perhaps two or more approaches will be required to find a code similar to that provided below:

:00401045 8A08 mov cl, byte ptr [eax]
:00401047 880C02 mov byte ptr [edx+eax], cl
:0040104A 40 inc eax
:0040104B 84C9 test cl, cl
:0040104D 75F6 jne 00401045

This is the assembly snippet of an strcpy function. On entry to the function, the memory location pointed by EAX is read in order to move (next line) its value into memory location, pointed by the sum of the registers EAX and EDX. By reading the content of these registers during the first iteration we can determine that the buffer is located at 0x0012fec0.

Writing a shell code is an art in itself. Since operating systems use different system function calls, an individual approach is needed. It is dependent on the OS environment under which the code must run and the goal it is being aimed at. In the simplest case, nothing needs to be done, since just overwriting the return address causes the program to deviate from its expected behavior and fail. In fact, due to the nature of buffer overflow flaws associated with the possibility that the attacker can execute arbitrary code, it is possible to develop a range of different activities constrained only by available space and access privileges. In most of the cases, buffer overflow is a way for an attacker to gain “super user” privileges on the system or to use a vulnerable system to launch a Denial of Service attack. Let us try, for example, to create a shell code allowing commands (interpreter cmd.exe in WinNT/2000). This can be attained by using standard API functions: WinExec or CreateProcess. When WinExec is called, the process will look like this:

WinExec(command, state)
In terms of the activities that are necessary from our point of view, the following steps must be carried out:

1. Pushing the command to run onto the stack. It will be "cmd /c calc”.
2. Pushing the second parameter of WinExec onto the stack. We assume it to be zero in this script.
3. Pushing the address of the command "cmd /c calc”.
4. Calling WinExec.

There are many ways to accomplish this task and the snippet below is only one of the possible tricks:

sub esp, 28h ; 3 bytes
jmp call ; 2 bytes
call WinExec ; 5 bytes
push eax ; 1 byte
call ExitProcess ; 5 bytes
xor eax, eax ; 2 bytes
push eax ; 1 byte
call par ; 5 bytes
.string cmd /c calc|| ; 13 bytes
Some comments on this:
sub esp, 28h

This instruction adds some room to the stack. Since the procedure containing an overflow buffer had been completed, consequently, the stack space allocated for local variables is now declared as unused due to the change in ESP. This has the effect that any function call, which is given from the code level is likely to overwrite our constructed code inserted in the buffer. To have a function call save, all we need is to restore the stack pointer to what it was before “garbage”, that is to its original value (40 bytes), thereby assuring that our data will not be overwritten.

jmp call

The next instruction jumps to the location where the WinExec function arguments are pushed onto the stack. Some attention must be paid to this. Firstly, a NULL value is required to be “elaborated” and placed onto the stack. Such a function argument cannot be taken directly from the code otherwise it will be interpreted as null terminating the string that has only been partially copied. In the next step, we need a way of pointing the address of the command to run and we will make this in an ad hoc manner. As we may remember, each time a function is called, the address following the call instruction is placed onto the stack. Our successful (hopefully) exploit first overwrites the saved return address with the address of the function we wish to call. Notice that the address for the string may appear somewhere in the memory. Subsequently, WinExec followed by ExitProcess will be run. As we already know, CALL represents an offset that moves the stack pointer up to the address of the function following the call. And now we need to compute this offset.

  Listing 4 – Exploit of above program victim.exe 
  char *victim =  "victim.exe";
  char *code =  "\x90\x90\x90\x83\xec\x28\xeb\x0b\xe8\xe2\xa8\xd6\
char *oper = "cmd /c calc||"; char *rets = "\xc0\xfe\x12"; char par[42];
void main() { strncat(par, code, 28); strncat(par, oper, 14); strncat(par, rets, 4); char *buf; buf = (char*)malloc( strlen(victim) + strlen(par) + 4); if (!buf) { printf("Error malloc"); return; }
wsprintf(buf, "%s \"%s\"", victim, par); printf("Calling: %s", buf); WinExec(buf, 0); }

Voila, it works! The only requisite is that the current directory has a compiled file victim.exe. If all goes as expected, we will see a window with a well-known System Calculator.


In this article I try to explain buffer overrun in the simplest manner, but there are plenty of interesting buffer overflow issues, which are quite trickier and complex, which I have not discussed. My intention was to build a concept and bring out certain problems. I hope that this paper will be a contribution to the improvement of the software development and its quality through better understanding of the threat, and hence, providing better security to all of us and certainly the stable programs.


1. In practice, certain code-optimizing compilers may operate without pushing EBP on the stack. The Visual C++ 7 Compiler uses it as a default option. To deactivate it, set: Project Properties | C/C++ | Optimization | Omit Frame Pointer for NO.

2. Microsoft has introduced a buffer overrun security tool in its Visual C++ 7. If you are intending to use the above examples to run in this environment, ensure that this option is not selected before compilation: Project Properties | C/C++ | Code Generation | Buffer Security Check should have the value NO.

Hardik Shah can be reached on his email id at hardik_nt@yahoo.com.