About
Articles
Book Store
Distributed RCE
Downloads
Event Calendar
Forums
Live Discussion
Reference Library
RSS Feeds
Search
Users
What's New
Customize Theme
bluegrey
blackgreen
metal
simple
Flag:
Tornado!
Hurricane!
Login:
Password:
Remember Me
Register
Blogs
>>
EliCZ
's Blog
Created: Friday, April 25 2008 09:21.42 CDT
Printer Friendly ...
Integer overflow
Author:
EliCZ
# Views:
10532
How to write integer(32) division in i386 assembly language using idiv instruction so that it doesn't raise an exception?
mov eax, Dividend
mov ecx, Divisor
idiv ecx
It can throw #DE, add Divisor == 0 check:
mov eax, Dividend
mov ecx, Divisor
jecxz DontIdiv
idiv ecx
It can still throw #DE, edx must be initialized for (i)div.
AMD64 manual : "To avoid overflow problems, precede this instruction (idiv) with a CBW, CWD, CDQ, or CQO instruction to sign-extend the dividend."
mov eax, Dividend
mov ecx, Divisor
jecxz DontIdiv
cdq
idiv ecx
It looks fixed (C compilers think so), can it throw #DE now?
Yes, it can - I didn't avoid overflow problems.
There is still one combination that will raise #DE:
Dividend = INT_MIN and Divisor = -1.
mov eax, Dividend
mov ecx, Divisor
jecxz DontIdiv
cmp eax, INT_MIN
sete dl
cmp ecx, -1
sete dh
test dl, dh
jne DontIdiv
cdq
idiv ecx
Other integer arithmetic instructions set OF only and the result for 0-INT_MIN is INT_MIN.
How do C compilers (CPUs, OSes) handle the integer overflow?
neg32.c:
int neg32(int x) {
return(-x);
}
neg64.c:
long long neg64(long long x) {
return(-x);
}
idiv32.c:
int idiv32(int x, int y) {
return(x/y);
}
idiv64.c:
#define __USE_ISOC99
#include <limits.h>
#ifndef LLONG_MIN
#ifdef LONG_LONG_MIN
#define LLONG_MIN LONG_LONG_MIN
#else
#define LLONG_MIN -9223372036854775808LL
#endif
#endif
#include <stdio.h>
long long idiv64(long long x, long long y) {
return(x/y);
}
int main(void) {
long long x = LLONG_MIN;
long long y = -1;
long long r = 1234;
r = idiv64(x, y);
return(printf("%lli\n", r));
}
Sometimes a (CRT) function is used instead of division performing instruction,
especially for integer64 division in 32bit builds.
(int)-INT_MIN == INT_MIN always.
architecture/os/compilers/builds neg32 | neg64 | idiv32 | idiv64
i386/winnt/msc|dmc/32 INT_MIN | LLONG_MIN | crash | LLONG_MIN
i386/winnt/wc/32 INT_MIN | LLONG_MIN | print exception info and exit | LLONG_MIN
i386/linux/gcc/32 INT_MIN | LLONG_MIN | "Floating point exception" | LLONG_MIN
amd64/winnt/msc/64 INT_MIN | LLONG_MIN | crash | crash
amd64/linux/gcc/64 INT_MIN | LLONG_MIN | "Floating point exception" | "Floating point exception"
ia64/winnt/msc/32 INT_MIN | LLONG_MIN | long pause and exit | LLONG_MIN
ia64/winnt/msc/64 INT_MIN | LLONG_MIN | INT_MIN | LLONG_MIN
power/aix/gcc|xlc/32 INT_MIN | LLONG_MIN | 0 | LLONG_MIN
power/aix/gcc|xlc/64 INT_MIN | LLONG_MIN | 0 | 0
sparc/solaris/gcc|suncc/32|64 INT_MIN | LLONG_MIN | INT_MIN | LLONG_MIN
See you at caro workshop.
Blog Comments
GynvaelColdwind
Posted: Friday, April 25 2008 11:46.03 CDT
Woah... INT_MIN / -1 ... I've never thought of that! Good stuff mate!
nezumi
Posted: Monday, May 5 2008 05:31.56 CDT
clever post! I just would like to add:
-INT_MIN/-1;
(INT_MAX + 1)/-1;
causes an exception as well.
(yeah, I know that -INT_MIN/-1 and (INT_MAX + 1)/-1 it's the same, however they look different at source level before pre-processing)
Add New Comment
Comment:
There are
31,312
total registered users.
Recently Created Topics
[help] Unpacking VMP...
Mar/12
Reverse Engineering ...
Jul/06
hi!
Jul/01
let 'IDAPython' impo...
Sep/24
set 'IDAPython' as t...
Sep/24
GuessType return une...
Sep/20
About retrieving the...
Sep/07
How to find specific...
Aug/15
How to get data depe...
Jul/07
Identify RVA data in...
May/06
Recent Forum Posts
Finding the procedur...
rolEYder
Question about debbu...
rolEYder
Identify RVA data in...
sohlow
let 'IDAPython' impo...
sohlow
How to find specific...
hackgreti
Problem with ollydbg
sh3dow
How can I write olly...
sh3dow
New LoadMAP plugin v...
mefisto...
Intel pin in loaded ...
djnemo
OOP_RE tool available?
Bl4ckm4n
Recent Blog Entries
halsten
Mar/14
Breaking IonCUBE VM
oleavr
Oct/24
Anatomy of a code tracer
hasherezade
Sep/24
IAT Patcher - new tool for ...
oleavr
Aug/27
CryptoShark: code tracer ba...
oleavr
Jun/25
Build a debugger in 5 minutes
More ...
Recent Blog Comments
nieo
on:
Mar/22
IAT Patcher - new tool for ...
djnemo
on:
Nov/17
Kernel debugger vs user mod...
acel
on:
Nov/14
Kernel debugger vs user mod...
pedram
on:
Dec/21
frida.github.io: scriptable...
capadleman
on:
Jun/19
Using NtCreateThreadEx for ...
More ...
Imagery
SoySauce Blueprint
Jun 6, 2008
[+] expand
View Gallery
(11) /
Submit