A buffer overflow occurs when the program is trying to write data over a buffer capacity, resulting overwriting other memory locations. Overwriting the memory locations might give the attacker a chance to take control of the program execution.
I'll mostly use GDB (GNU debugger), a classic debugger in Linux, to see what is happening in the stack when executing a program and fuzzing some data on it. The debugger can display the stack, and the registers, function address, library used, etc. If you do not have any knowledge about how the stack works, please check out my tutorial about the stack here : How does the stack work
Among the listed functions, we see the function called "give_flag()", that might potentially what we want to call using the program.
Looking at the call instructions in the main function, we have setbuf(), signal(), puts() and gets(). Among those functions, the one that must be shining to your eyes are the gets() because if we look at the gets() manual, it is clearly indicating the reason to never use gets().
According to the manual, gets() will not check how many characters will be read so it might store some characters past the end of the buffer.
Here is our vulnerability. We are going to use gets() to overwrite some data and take control of the program execution. The goal will be to overwrite the stackframe and to modify the register RIP to the address of the give_flag() function.
The instruction leave will set rsp to rbp so the stackframe will be destroyed.
Continuing, the stackframe does not exist anymore and the next value of the stack is at the address 0x7fffffffe1e8 that has a value 0xf7e11d0a. We didn't reach 0x7fffffffe1e0 because the second part of the instruction leave is POP RBP, so the register rbp is taking the value of the top of the stack and remove it from the stack.
Looking at the register table, the register $rbp is taking the value "0x555555555300" which means that we didn't fuzzle enough to overwrite the base pointer of the stack, therefore we are missing 8 bytes to overwrite RBP. The next instruction is calling the RET function, which is similar to POP RIP, is taking the value on the top of the stack and store it on the register RIP. Since the register RIP is executing the next instruction then if we manage to know the offset to control the top of the stack at the moment, we also control the register RIP, which means that we can control the program execution.
$rip = 0x7f00deadbeef, we've managed to write "deadbeef" in the register RIP, we control the program execution. Therefore instead of writing "deadbeef", we can set the give_flag() address to the RIP register to achieve our goal.
The program is calling the RET function, the value on the top of the stack is 0x555555555203, so ret set this value to the register RIP, and the function give_flag() will be executed.
We see that the function give_flag() is calling the function fopen(). Fopen() function has 2 parameters :
The file passing as an argument to the function fopen() is flag.txt. Let's try to exploit in our local environment to see if that works.
The buffer overflow works, the program is not supposed to open the file called flag.txt, but we've managed to control the program execution by overwriting the stack and modifying the address that the register RIP is supposed to be. We have set the register RIP with our address to make the program to open a file.