Reverse › Module 3 › Lesson 2
Calling Conventions
System V AMD64 and Microsoft x64 name argument registers — follow calls on $RE_LAB, not unauthorized API hooking.
Visual · re_calling_conventions
Calling conventions literacy. $RE_LAB only. Original Cyberlium.
Opening
Arguments live in named registers — miss the convention and you misread every strcmp.
Linux x86-64 System V passes first integer args in RDI, RSI, RDX, RCX, R8, R9; return in RAX. Windows x64 uses RCX, RDX, R8, R9 for the first four. Callee-saved vs caller-saved registers define who preserves values across CALL. Stack aligns to 16 bytes before CALL on both. Cyberlium teaches convention literacy for tracing library calls in YOUR toy binaries on $RE_LAB. You will NOT hook commercial processes without authorization. Next: Control Flow.
1. System V AMD64 (named)
Arguments: RDI, RSI, RDX, RCX, R8, R9 then stack. Return RAX (RDX for 128-bit). Callee-saved: RBX, RBP, R12–R15.
strcmp(a,b) on $RE_LAB Linux toy binary: RDI and RSI hold string pointers at CALL — note in trace before stepping into PLT.
Command guide
Try these commands — System V AMD64 (named)
═══ TOOLS & WEBSITES ═══ Browse / read these (authorized learning only — stay in YOUR lab / program scope)
System V AMD64 ABI — https://refspecs.linuxfoundation.org/elf/x86_64-abi-0.99.pdf (calling conventions) objdump -d main — https://man7.org/linux/man-pages/man1/objdump.1.html LiveOverflow — https://www.youtube.com/c/LiveOverflow (calling convention explainers)
═══ INSTALL ═══
Linux (Debian/Ubuntu):
Command — copy this
sudo apt install binutils sudo apt install build-essential
macOS: Built-in (Xcode CLI) or brew install binutils
Command — copy this
xcode-select --install # or brew install gcc
Windows: Use WSL binutils Use WSL or MinGW
═══ LINUX / macOS ═══
Command — copy this
export LAB_RE=${LAB_RE:-$HOME/cyberlium-lab/t23-re}
objdump -d "$LAB_RE/bin/check" | grep -A20 '<add>:' | head -22
objdump -d "$LAB_RE/bin/check" | grep -A25 '<main>:' | grep -E 'call|mov.*rdi|mov.*rsi|mov.*eax' | head -10Primary tools to practice this lesson: objdump, gcc. Reference sites: System V AMD64 ABI (https://refspecs.linuxfoundation.org/elf/x86_64-abi-0.99.pdf); objdump -d main (https://man7.org/linux/man-pages/man1/objdump.1.html); LiveOverflow (https://www.youtube.com/c/LiveOverflow). Run every command in the box — install first, then the usage lines — only on YOUR lab / program scope.
2. Microsoft x64 (named)
First four integer args: RCX, RDX, R8, R9; shadow space on stack. Return RAX. Different order from System V — always check binary OS.
Windows crackmes on $RE_LAB need Windows guest or cross-disassembly with correct convention — never assume Linux register order.
3. Convention literacy boundaries
Forbidden: API hooking on neighbor apps, detouring commercial license checks on unauthorized software, or injecting trampolines outside $RE_LAB toy targets.
Ship: side-by-side table — System V vs Microsoft, first four arg registers each. Next: Control Flow.
4. What you ship: calling conventions comparison table
System V vs Microsoft x64 arg registers. $RE_LAB toy trace example. NEVER unauthorized hooking. chmod 600.
5. What you record before the next lesson
Date. Calling conventions table. $RE_LAB named. File t23-m03-l02-calling-conventions.txt chmod 600.
6. Wrong vs right: commercial piracy vs crackmes/CTF toys
Worked failure — same MSF word, opposite target. Right never needs a café Wi-Fi or classmate laptop.
Wrong
Assume RCX is first arg on Linux toy binary. Hook paid app license API on host.
Right
Write calling conventions table with YOUR toy trace note. Next: Control Flow.
Mission: compare two conventions
1) List first four arg registers for System V and Microsoft x64. 2) Note which convention YOUR toy binary uses. 3) chmod 600.
Stuck? Ask Cyberlium AI Mentor
OS and toolchain pick the convention — verify, don't guess.
Knowledge Check
APPLY: Calling convention literacy means:
Multiple choice
Knowledge Check
APPLY: True or False: System V AMD64 first integer arg is typically RDI.
True or False
Knowledge Check
APPLY: Microsoft x64 first integer arg register is:
Multiple choice