What is Reverse Engineering?
Reverse engineering is the process of taking a compiled program and figuring out how it works without having access to its original source code. This usually involves looking at its functions, strings, assembly instructions, and general behaviour.
In CTFs, you might be given a program that checks for a secret password, changes your input in some unknown way, or hides a flag somewhere in its logic. Your goal is to understand what the program is doing and work out how to get the result you want.
How Does Reverse Engineering Work?
There are two main ways to analyse a program. Static analysis means looking at the program without running it. Tools such as Ghidra can turn machine code into more readable pseudocode, which helps you understand functions and follow the program’s logic.
Dynamic analysis means running the program and watching what happens. A debugger such as GDB lets you pause the program, step through its instructions, and inspect values while it is running. In practice, you will often move between both approaches while solving a challenge.
Start with the Basics
When you first receive a binary, start by running it and seeing how it
behaves. Simple commands such as file and strings
can also give you useful information before you open it in a larger tool.
Use a Decompiler
Try opening the program in Ghidra and following its main functions. Look for code related to user input, comparisons, or interesting strings. If something is still unclear, you can use a debugger to see what the program is doing while it runs.
Don't Sweat the Assembly
You do not need to know assembly before starting. Being able to recognise basic ideas such as functions, loops, conditions, and comparisons is enough for many beginner challenges. You can learn more assembly as you move on to harder problems.