I wrote a program that returns incorrect results with large data sets. If the input data set is above a certain size, I see a negative result and the mathematical result can never be negative.
In one example, I get a correct result of 1176350207. But when the result should be2407905288, I instead get -1887062008. I suspect the problem is due to being unable to specify data type (as far as I know) and SKILL is apparently using signed int as defualt (at least for an integer variable). Note that:
log2(1176350207) = 30.13, requires 31 bits to represent
log2(2407905288) = 31.17, requires 32 bits to represent
If I could use an unsigned int, I should be able to represent 2407905288. I've never had a problem letting SKILL do it's dynamically typed goodness before. Perhaps sxtd() or zxtd() could help as they seem to be along the right track, but I don't understand (I'm a programming dunce).
So is there any way to specify variable data type in SKILL? Or am I out of luck with numbers this large?
I raised a service request on this in 2010 (SR 42278266 CCR 846991 Large number multiplication error)
This was part of the response:
The service request example is using a list “signed int” type which corresponds to a 32bit internal number.
220000000 is 0x0D1CEF00 in hex (28 bits set)
220000000*10 = 2200000000 or 0x83215600 (32 bits set)
An overflow occurs because the MSB is set which turns the number into a negativevalue.
To evade this problem, we could use a list double type which has the range needed but would have the following problem:
- checking for overflow or underflow if we are converting back to an int type.
- We can accommodate the approximation of floating point numbers on digital computers and
axlGeoEqual can help with this.
However, the SR was closed with no commitment to fix the problem. I have got used to working around the problem by ensuring that my code doesn't use such large integers.
Due to the danger of overflow, you should consider doing any math that involves multipication or division in floating point. If you need to convert back to integer at the end of the operations to address overflow, underflow, or rounding use the API round(x_float) --> x_int