Floating point rounding

Hi Bala,

It's not clear to me precisely what you're saying here. I can completely get that you might have a rounding error somewhere, but I don't understand what you mean by:

• "But I am getting 69.999* something". Which "actual y value" is 70.0? The result of the yCoord expression? The result of the floor expression? Which expression is giving 69.999?
• "When I use floor it is also coming like that only". Coming like what? 
• "Because of this one If I am trying to compare that y value with some other digit It is not passing the if condition but the opposite value is 70.0 only" - it sounds as if you're describing some if condition which you've not shown in the example. Note that you should never use == on floating point numbers unless you can guarantee the results have been computed in identical ways (and even then you should take extreme car). Either way, this sentence doesn't really make sense.
• The forum guidelines ask you to not use instance messenger abbreviations (such as "u" for "you" - assuming that is what you meant). I understand that you may not be a native English speaker, but as a native English speaker I get a bit grumpy seeing words unnecessarily abbreviated when there is no need.

So I'd love to try to help, but it's not clear what your actual problem is.

Kind Regards,

Andrew.

Hi Andrew,

First of all sorry for the inconvenience with my English.

What I am trying to ask is I have 2 inputs both are from Layout coordinates. For suppose a=70.0 and b=70.0(These are actual values which are the coordinates in Layout). But while reading the inputs from the layout one of those inputs is taking like 69.999999. Then if I am comparing those 2 variables I am not getting the correct value.

I used "floor(69.999999*10000)/10000.0" then also I am getting that value as 69.999999 only.

Is there any solution for this one?

Regards,

Bala

If both values came directly from the database (the same technology - i.e. with the same DBUperUU), then they absolutely would be equivalent. I'm assuming you did some manipulation of the data first before comparing them. The reason being is that the numbers stored in the actual database are integers, and then these are divided by the DBUperUU to give a floating point value, so the same computation must result in the same number. So I am rather doubtful that two coordinates from the same database which are both 70.0 would not be the same - I've never seen that behaviour before.

The floor expression you gave actually results in 69.9999 (not 69.999999) because it will multiply by 10000, round down, and then divide by 10000. Maybe you should use round() rather than floor()?

Comparing floating point numbers in general is better to be done using (say) abs(y1-y2)<0.0001 (if the numbers are coordinates and you have a DBUperUU of (say) 1000).

Regards,

Andrew.