Weird numerical behavior

This will be because y1 is actually very slightly (due to floating point rounding errors) lower than 24 - and since the int function takes the integer part, it gets the number below.

You can often see this by using:

sstatus(fullPrecision t)
y1

but sometimes you might need one more digit than fullPrecision uses:

printf("%.17g\n" y1)

You can also see that if you do:

y1-24.0 

that you'll get a small residual negative value showing that y1 was slightly below 24 in practice.

This floating point rounding behaviour is well documented in any programming language that uses IEEE floating point numbers (C, Python, etc) and there are articles on support.cadence.com that cover it too. Put simply it's down to the challenges of representing a decimal number using a binary mantissa. In the same way that 1/3 can't be exactly represented in decimal, 0.1 (1/10) can't be exactly represented in binary (it's a recurring fraction) - and so you lost digits at the end of the mantissa with certain floating point operations.

Interestingly we've just been having discussions about a better way to make these rounding errors more evident.

Andrew

Thanks!

Thanks!