How bad is a "controlled" signed int overflow and can it even be "controlled"?

The "why" mainly originates around signed integer representations that are not twos complement, such as sign-magnitude and ones complement.

On architectures that use these representations, it is possible that signed integer overflow can cause a hardware trap. Because of this, the C standard made signed integer overflow undefined behavior. The odds of finding such a system these days is pretty small however, so much so that starting with the C23 standard twos complement representation is the standard, although signed integer overflow remains undefined behavior.

Another thing about undefined behavior is that modern optimizing compilers will assume undefined behavior doesn't happen and perform optimizations based on that assumption. So it's possible that the compiler can generate code that would cause signed integer overflow to behave in a way that differs from unsigned "overflow" (which is actually a well-defined wraparound).

If you want to ensure that signed integers wraparound similarly to unsigned integers, GCC has the -fwrapv option which explicitly sets this behavior. Then you won't have any unexpected results such as the example in Lundin's answer.

Mainly the problem is this: if you add two positive signed numbers and there is an overflow, then a 2's complement CPU would wrap around the number to a negative value. C used to support various other exotic signedness formats however, that behaved differently. Therefore overflow (and underflow) was categorized as undefined behavior meaning anything can happen. Which in turn means that the compiler is free to make all manner of assumptions, like for example "addition of two positive numbers can never result in a negative number".

Take this as an example of how undefined behavior causes bugs:

#include <stdio.h>
#include <limits.h>
#include <stdlib.h>

int main (int argc, char* argv[])
{
  int a = atoi(argv[1]);
  int b = atoi(argv[2]);
  if(a < 0 || b < 0)
    return 0;

  int n = a + b;

  if(n>0)
    printf("%d is > 0\n", n);
}

With 32 bit int, I pass the command line arguments 2147483647 1. (The reason why I'm doing it with command line is to block various compile-time optimizations.) Then compile with gcc -O3.

The compiler concludes:

• At the point of addition, a and b are both positive or we wouldn't have gotten there.
• Thus n>0 is always true so that comparison can be removed.

And then the program output is: -2147483648 is > 0 - the printf statement got executed even though it shouldn't be possible.

Now if I remove optimizations, that doesn't happen, because the compiler then includes the comparison.

As for what can be done about it - if you don't use signed numbers but unsigned, then C does no longer call that overflow but wrap-around. Meaning unsigned numbers always behave well-defined since they wrap around.

How bad is a "controlled" signed int overflow and can it even be "controlled"?

Do not assume common compiler results will be the same in the future when dealing with undefined behavior (UB). Compilers have a history of taking advantage of UB found in old weak code. Don't rely on it else the code has a latent (expensive) bug for the future.

Many students (I included) do it some way that causes an overflow in case of INT_MIN ...

This can easily be avoided. So the bad overflow becomes a non-issue.

With very little re-work, OP's code can be adjusted to avoid the UB when the input string encodes INT_MIN.

The key idea is that rather than accumulate the result res as a positive intermediate value, accumulate it as a negative one. This takes advantage that the range [0...INT_MIN] is greater than [0...INT_MAX].

int my_atoi(const char *s)
{
    int neg;
    int res;
    size_t  i;

    i = 0;
    res = 0;
    //neg = 1;
    neg = -1;
    while (s[i] == ' ' || (s[i] >= 9 && s[i] <= 13))
        i++;
    if (s[i] == '+' || s[i] == '-')
    {
        if (s[i] == '-')
            // neg = -1;
            neg = 1;
        i++;
    }
    while (s[i] >= '0' && s[i] <= '9')
    {
        // res = res * 10 + s[i] - '0';
        res = res * 10 - (s[i] - '0');
        i++;
    }
    return (res * neg);
}

Other improvements possible too: No UB on any overflow, locale white-space, better names, ...

Note that C23 includes the header <stdckdint.h> (§7.20) which defines three type-generic macros:

#include <stdckdint.h>
bool ckd_add(type1 *result, type2 a, type3 b);
bool ckd_sub(type1 *result, type2 a, type3 b);
bool ckd_mul(type1 *result, type2 a, type3 b);

You can use these to check whether the computation will overflow.

If these type-generic macros return false, the value assigned to *result correctly represents the mathematical result of the operation. Otherwise, these type-generic macros return true. In this case, the value assigned to *result is the mathematical result of the operation wrapped around to the width of *result.

There isn't a macro for division because there's only one case that can produce overflow (division by zero is undefined behaviour, of course). If you're dealing with the int type, dividing INT_MIN by -1 overflows.

Signed integer overflow should be avoided because the compiler can make assumptions about it never occurring which in some cases can have unexpected consequences.

For example, this simplistic overflow checking code does not work:

    int n = 0;
    while (*p >= '0' && *p <= '9') {
        n = n * 10 + (*p - '0');
        if (n < 0) n = INT_MAX;
    }

The compiler may assume that given the above code, n cannot become negative without undefined behavior hence the if (n < 0) n = INT_MAX; can be ignored.

For this reason, it is recommended to check for potential overflow before the operation.

Here is a sample implementation of atoi() with reliable overflow control:

#include <ctype.h>
#include <errno.h>
#include <limits.h>
#include <stdio.h>

int my_atoi(const char *s) {
    int n = 0;

    /* skip optional initial white space */
    while (isspace((unsigned char)*s))
        s++;

    if (*s == '-') {
        /* convert negative number: use negative digit values */
        s++;
        if (!isdigit((unsigned char)*s) {
            errno = ERANGE;
            return 0;
        }
        while (isdigit((unsigned char)*s)) {
            int d = '0' - *s++;
            /* check for potential arithmetic overflow */
            if (n < INT_MIN / 10 || (n == INT_MIN / 10 && d < INT_MIN % 10)) {
                errno = ERANGE;
                n = INT_MIN;
                break;
            }
            n = n * 10 + d;
        }
    } else {
        /* ignore optional positive sign */
        if (*s == '+')
            s++;
        if (!isdigit((unsigned char)*s) {
            errno = ERANGE;
            return 0;
        }
        while (isdigit((unsigned char)*s)) {
            int d = *s++ - '0';
            /* check for potential arithmetic overflow */
            if (n > INT_MAX / 10 || (n == INT_MAX / 10 && d > INT_MAX % 10)) {
                errno = ERANGE;
                n = INT_MAX;
                break;
            }
            n = n * 10 + d;
        }
    }
    return n;
}

int main(int argc, char *argv[]) {
    for (int i = 1; i < argc; i++) {
        errno = 0;
        int n = my_atoi(argv[i]);
        printf("\"%s\" -> %d%s\n", argv[i], n, errno ? " (error)" : "");
    }
    return 0;
}

The function sets errno to ERANGE if the string is not a number and returns 0 or if the number is outside the range of int and returns the range limit of the same sign.