Calling Conventions
A more in-depth look into parameters for 32-bit and 64-bit programs

One Parameter

calling-conventions-one-param.zip
5KB
Binary
Calling Conventions - One Parameter

Source

Let's have a quick look at the source:
1
#include <stdio.h>
2
3
void vuln(int check) {
4
if(check == 0xdeadbeef) {
5
puts("Nice!");
6
} else {
7
puts("Not nice!");
8
}
9
}
10
11
int main() {
12
vuln(0xdeadbeef);
13
vuln(0xdeadc0de);
14
}
Copied!
Pretty simple.
If we run the 32-bit and 64-bit versions, we get the same output:
1
Nice!
2
Not nice!
Copied!
Just what we expected.

Analysing 32-bit

Let's open the binary up in radare2 and disassemble it.
1
$ r2 -d -A vuln-32
2
$ s main; pdf
3
4
0x080491ac 8d4c2404 lea ecx, [argv]
5
0x080491b0 83e4f0 and esp, 0xfffffff0
6
0x080491b3 ff71fc push dword [ecx - 4]
7
0x080491b6 55 push ebp
8
0x080491b7 89e5 mov ebp, esp
9
0x080491b9 51 push ecx
10
0x080491ba 83ec04 sub esp, 4
11
0x080491bd e832000000 call sym.__x86.get_pc_thunk.ax
12
0x080491c2 053e2e0000 add eax, 0x2e3e
13
0x080491c7 83ec0c sub esp, 0xc
14
0x080491ca 68efbeadde push 0xdeadbeef
15
0x080491cf e88effffff call sym.vuln
16
0x080491d4 83c410 add esp, 0x10
17
0x080491d7 83ec0c sub esp, 0xc
18
0x080491da 68dec0adde push 0xdeadc0de
19
0x080491df e87effffff call sym.vuln
20
0x080491e4 83c410 add esp, 0x10
21
0x080491e7 b800000000 mov eax, 0
22
0x080491ec 8b4dfc mov ecx, dword [var_4h]
23
0x080491ef c9 leave
24
0x080491f0 8d61fc lea esp, [ecx - 4]
25
0x080491f3 c3 ret
Copied!
If we look closely at the calls to sym.vuln, we see a pattern:
1
push 0xdeadbeef
2
call sym.vuln
3
[...]
4
push 0xdeadc0de
5
call sym.vuln
Copied!
We literally push the parameter to the stack before calling the function. Let's break on sym.vuln.
1
[0x080491ac]> db sym.vuln
2
[0x080491ac]> dc
3
hit breakpoint at: 8049162
4
[0x08049162]> pxw @ esp
5
0xffdeb54c 0x080491d4 0xdeadbeef 0xffdeb624 0xffdeb62c
Copied!
The first value there is the return pointer that we talked about before - the second, however, is the parameter. This makes sense because the return pointer gets pushed during the call, so it should be at the top of the stack. Now let's disassemble sym.vuln.
1
┌ 74: sym.vuln (int32_t arg_8h);
2
│ ; var int32_t var_4h @ ebp-0x4
3
│ ; arg int32_t arg_8h @ ebp+0x8
4
│ 0x08049162 b 55 push ebp
5
│ 0x08049163 89e5 mov ebp, esp
6
│ 0x08049165 53 push ebx
7
│ 0x08049166 83ec04 sub esp, 4
8
│ 0x08049169 e886000000 call sym.__x86.get_pc_thunk.ax
9
│ 0x0804916e 05922e0000 add eax, 0x2e92
10
│ 0x08049173 817d08efbead. cmp dword [arg_8h], 0xdeadbeef
11
│ ┌─< 0x0804917a 7516 jne 0x8049192
12
│ │ 0x0804917c 83ec0c sub esp, 0xc
13
│ │ 0x0804917f 8d9008e0ffff lea edx, [eax - 0x1ff8]
14
│ │ 0x08049185 52 push edx
15
│ │ 0x08049186 89c3 mov ebx, eax
16
│ │ 0x08049188 e8a3feffff call sym.imp.puts ; int puts(const char *s)
17
│ │ 0x0804918d 83c410 add esp, 0x10
18
│ ┌──< 0x08049190 eb14 jmp 0x80491a6
19
│ │└─> 0x08049192 83ec0c sub esp, 0xc
20
│ │ 0x08049195 8d900ee0ffff lea edx, [eax - 0x1ff2]
21
│ │ 0x0804919b 52 push edx
22
│ │ 0x0804919c 89c3 mov ebx, eax
23
│ │ 0x0804919e e88dfeffff call sym.imp.puts ; int puts(const char *s)
24
│ │ 0x080491a3 83c410 add esp, 0x10
25
│ │ ; CODE XREF from sym.vuln @ 0x8049190
26
│ └──> 0x080491a6 90 nop
27
│ 0x080491a7 8b5dfc mov ebx, dword [var_4h]
28
│ 0x080491aa c9 leave
29
└ 0x080491ab c3 ret
Copied!
Here I'm showing the full output of the command because a lot of it is relevant. radare2 does a great job of detecting local variables - as you can see at the top, there is one called arg_8h. Later this same one is compared to 0xdeadbeef:
1
cmp dword [arg_8h], 0xdeadbeef
Copied!
Clearly that's our parameter.
So now we know, when there's one parameter, it gets pushed to the stack so that the stack looks like:
1
return address param_1
Copied!

Analysing 64-bit

Let's disassemble main again here.
1
0x00401153 55 push rbp
2
0x00401154 4889e5 mov rbp, rsp
3
0x00401157 bfefbeadde mov edi, 0xdeadbeef
4
0x0040115c e8c1ffffff call sym.vuln
5
0x00401161 bfdec0adde mov edi, 0xdeadc0de
6
0x00401166 e8b7ffffff call sym.vuln
7
0x0040116b b800000000 mov eax, 0
8
0x00401170 5d pop rbp
9
0x00401171 c3 ret
Copied!
Hohoho, it's different. As we mentioned before, the parameter gets moved to rdi (in the disassembly here it's edi, but edi is just the lower 32 bits of rdi, and the parameter is only 32 bits long, so it says EDI instead). If we break on sym.vuln again we can check rdi with the command
1
dr rdi
Copied!
Just dr will display all registers
1
[0x00401153]> db sym.vuln
2
[0x00401153]> dc
3
hit breakpoint at: 401122
4
[0x00401122]> dr rdi
5
0xdeadbeef
Copied!
Awesome.
Registers are used for parameters, but the return address is still pushed onto the stack and in ROP is placed right after the function address

Multiple Parameters

calling-convention-multi-param.zip
5KB
Binary
Calling Conventions - Multiple Parameters

Source

1
#include <stdio.h>
2
3
void vuln(int check, int check2, int check3) {
4
if(check == 0xdeadbeef && check2 == 0xdeadc0de && check3 == 0xc0ded00d) {
5
puts("Nice!");
6
} else {
7
puts("Not nice!");
8
}
9
}
10
11
int main() {
12
vuln(0xdeadbeef, 0xdeadc0de, 0xc0ded00d);
13
vuln(0xdeadc0de, 0x12345678, 0xabcdef10);
14
}
Copied!

32-bit

We've seen the full disassembly of an almost identical binary, so I'll only isolate the important parts.
1
0x080491dd 680dd0dec0 push 0xc0ded00d
2
0x080491e2 68dec0adde push 0xdeadc0de
3
0x080491e7 68efbeadde push 0xdeadbeef
4
0x080491ec e871ffffff call sym.vuln
5
[...]
6
0x080491f7 6810efcdab push 0xabcdef10
7
0x080491fc 6878563412 push 0x12345678
8
0x08049201 68dec0adde push 0xdeadc0de
9
0x08049206 e857ffffff call sym.vuln
Copied!
It's just as simple - push them in reverse order of how they're passed in. The reverse order becomes helpful when you db sym.vuln and print out the stack.
1
[0x080491bf]> db sym.vuln
2
[0x080491bf]> dc
3
hit breakpoint at: 8049162
4
[0x08049162]> pxw @ esp
5
0xffb45efc 0x080491f1 0xdeadbeef 0xdeadc0de 0xc0ded00d
Copied!
So it becomes quite clear how more parameters are placed on the stack:
1
return pointer param1 param2 param3 [...] paramN
Copied!

64-bit

1
0x00401170 ba0dd0dec0 mov edx, 0xc0ded00d
2
0x00401175 bedec0adde mov esi, 0xdeadc0de
3
0x0040117a bfefbeadde mov edi, 0xdeadbeef
4
0x0040117f e89effffff call sym.vuln
5
0x00401184 ba10efcdab mov edx, 0xabcdef10
6
0x00401189 be78563412 mov esi, 0x12345678
7
0x0040118e bfdec0adde mov edi, 0xdeadc0de
8
0x00401193 e88affffff call sym.vuln
Copied!
So as well as rdi, we also push to rdx and rsi (or, in this case, their lower 32 bits).

Bigger 64-bit values

Just to show that it is in fact ultimately rdi and not edi that is used, I will alter the original one-parameter code to utilise a bigger number:
1
#include <stdio.h>
2
3
void vuln(long check) {
4
if(check == 0xdeadbeefc0dedd00d) {
5
puts("Nice!");
6
}
7
}
8
9
int main() {
10
vuln(0xdeadbeefc0dedd00d);
11
}
Copied!
If you disassemble main, you can see it disassembles to
1
movabs rdi, 0xdeadbeefc0ded00d
2
call sym.vuln
Copied!
movabs can be used to encode the mov instruction for 64-bit instructions - treat it as if it's a mov.