Using integers

Mixing signed and unsigned integers, or performing incorrect calculations on them, can result in a number of issues. These issues range from stability problems, such as Denial of Service (DoS), to arbitrary code execution in the affected program.

Best practices

There are a set of core rules that can help mitigate issues when dealing with integers:

  • Don't use signed integers, shorts, or chars when storing sizes that are supplied by network data.
  • Don't use signed integers, shorts, or chars for any variables that represent natural or countable items. Use signed variables only if a negative value makes sense in that context.
  • When performing arithmetic in conditional statements, ensure that the arithmetic is always balanced.
  • When casting, consider sign extension and zero extension in addition to truncation.
  • Where possible, avoid mixing types in expressions. Try to use consistent types or cast to consistent types, keeping the above guidelines in mind.
  • Remember that memory and copy functions in C/C++ treat all integers as unsigned.

Integer overflows and underflows

Integers can overflow and underflow. Consider the following function that results in an integer overflow:

#include <stdio.h>

int main(int argc, char * argv[]) {
     int intFoo=0;

     fprintf(stdout,"[i] %d %u\n",intFoo,intFoo);
     intFoo -= 1;
     fprintf(stdout,"[i] %d %u\n",intFoo,intFoo);

     return 0;
}

Calling this function results in the following:

[i] 0 0
[i] -1 4294967295

Since unsigned integers range from 0 to 232 - 1, subtracting 1 from 0 causes the value to wrap around and result in a potentially undesired value.

For applications that are written in a non-memory safe language like C and C++, these overflow issues can lead to memory corruption and catastrophic security compromises. If you do not catch exceptions and deal with them appropriately, integer underflow or overflow can result in a DoS and might lead to unexpected application behavior. In the case of security-relevant components, these issues can then allow other types of security breaches. For example, inducing an integer overflow can circumvent the functionality of an access control system.

Common issues

There are some common issues that can result when you use integers. For example:

  • Using signed integer variables for size values that are read from network data.
  • Casting between signed and unsigned integers.
  • Performing unbalanced calculations within conditional checks that allows for bypasses.

Reading signed integers from network-supplied data

In the following code sample, the type, size, and data are read from a simulated incoming network packet in the following format:

[TYPE][SIZE][DATA]

The size is a 32-bit unsigned integer. However, in the code sample, the size is copied into a signed integer.

int main(int argc, char* argv[]) {
     char strPktIncoming[] = 
             "\x01\x05\x00\x00\x00\x41\x41\x41\x41\x00";
     char *strPkt;
     char strPktSubPayload[256];
     char strType;
     int intSize;

     strPkt = strPktIncoming;

     // get the packet type
     strType = *strPkt;
     strPkt++;

     // get the size
     memcpy(&intSize,strPkt,4);
     strPkt += 4;

     switch(strType) {

         case 1:
             // check and copy
             if(intSize < 256) {
                 memcpy(strPktSubPayload,strPkt,intSize);
                 fprintf(stdout,"[i] %d %s\n",
                         intSize,strPktSubPayload);
             }
             break;

         default:
             fprintf(stdout,"[i] Unknown packet type\n");
             break;
     }

     return 0;
}

If you replace \x05\x00\x00\x00 in the incoming packet (the bit that denotes the size) with \x00\x00\x00\80 (-2147483648 as a signed integer), it would pass the condition in case 1. In this case, because memcpy() takes the amount to copy as an unsigned integer, it copies 2 GB of data into a 256 byte character array.

Casting between signed, unsigned, and different sizes

This section is based in part on examples drawn from The Art of Security Software Assessment.

Casting between signed and unsigned integers can have unexpected results. You should consider the following when casting:

  • Preserved bit patterns
  • Sign extension
  • Zero extension
  • Truncation

Preserved bit patterns

The following code sample demonstrates a preserved bit pattern resulting from casting an unsigned integer to a signed integer:

int intSignedIn=0xFFFFFFEE;
unsigned int intUnsignedOut;
int intSignedOut;

intUnsignedOut=intSignedIn;
intSignedOut=intUnsignedOut;

fprintf(stdout,"[i] %08x %08x %08x\n",intSignedIn,intUnsignedOut,intSignedOut);

Executing the code results in the following:

[i] ffffffee ffffffee ffffffee

All three are the same bit pattern, but they can have widely different meanings depending on how they are used.

Sign extension and zero extension

The following code sample demonstrates a sign extension. The value of -5 in a signed char (0xFB) is turned into 0xFFFFFFFB.

char chrFoo = -5;
unsigned int intUnsignedOutchr;

intUnsignedOutchr=chrFoo;
fprintf(stdout,"[i] %d %u\n",chrFoo,intUnsignedOutchr);

Executing the code results in the following:

[i] -5 4294967291

Zero extension can happen when you cast an unsigned char into a signed integer, in which case 0x05 would be turned into 0x00000005.

Truncation

In the following code sample, a larger variable is cast into a smaller one of a different signed type:

int intIn = -1000000;
unsigned short shrtOutput;

shrtOutput=intIn;
fprintf(stdout,"[i] %d %d\n",intIn,shrtOutput);

Executing this code results in the following:

[i] -1000000 48576

Because 0xFFF0BDC0 (a signed integer) is truncated to 0xBDC0 (an unsigned short), the meaning changes significantly. Even in cases where the small variable is signed, the negative number is reduced due to the same truncation:

[i] -1000000 -16960

A similar problem is the use of mixed types in arithmetic operations in C and C++. Compilers promote numeric variables to a common type before performing the operation. The way that this happens can often lead to security vulnerabilities.

When variables are promoted to a common type, the ANSI C standard requires that if a variable's full range of values can be contained in an integer type, it becomes an integer. Everything else becomes an unsigned integer.

Consider the following code sample:

int a = 10, c = -1;
unsigned int b = 10;

if( a + b < c ) return TRUE;

int a = 10, c = -1;
unsigned short b = 10;

if( a + b < c ) return TRUE;

The first if statement returns TRUE. The second if statement returns FALSE. In the first if statement, both a and b are converted to unsigned integers. This is because a signed integer can't contain the full range of values of b. So, the if statement becomes an unsigned comparison and c is treated as the unsigned value 0xFFFFFFFF.

In the second if statement, b is converted to a signed integer. This is because a signed integer can contain the full range of values of an unsigned short. As a result, the signed comparison evaluates to FALSE.

Unbalanced arithmetic in conditional statements

The following code sample illustrates a common issue involving unbalanced arithmetic:

int main(int argc, char* argv[]) {
     char strPktIncoming[] = 
             "\x01\x01\x00\x00\x00\x41\x41\x41\x41\x00";
     char *strPkt;
     char strPktSubPayload[256];
     char strType;
     int intSize;

     strPkt=strPktIncoming;

     // get the packet type
     strType=*strPkt;
     strPkt++;

     // get the size
     memcpy(&intSize,strPkt,4);
     strPkt+=4;

     switch(strType){

         case 1:
             // check and copy
             if(intSize > 0 && intSize - 5 < MAX_SIZE) {
                 memcpy(strPktSubPayload,strPkt,intSize - 5);
                 fprintf(stdout,"[i] %d %s\n",
                         intSize - 5,strPktSubPayload);
             } else {
                 fprintf(stdout,"[i] %d %d\n",intSize - 5,intSize);
             }
             break;

         default:
             break;
     }

     return 0;
}

In the above example, the problem is the following if statement:

if(intSize > 0 && intSize - 5 < MAX_SIZE){

If the user-supplied data for intSize (1 in our example above) is a value between 1 and 4, then it will cause a wrap. This issue results in memcpy() copying a very large amount of data (in this example, nearly 4 GB). This error occurs because the conditional checks for upper and lower bounds must check the value in the form that it will be used in, not necessarily in the form that it was provided in.

You can resolve this issue by ensuring that the checks mirror how the resulting value will be used in memcpy():

if(intSize - 5 > 0 && intSize - 5 < MAX_SIZE){

Last modified: 2014-05-14



Got questions about leaving a comment? Get answers from our Disqus FAQ.

comments powered by Disqus