Stats

  • Locked Locked
  • Replies 1
  • Subscribers 125
  • Views 12725
  • Members are here 0
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Strange result in SKILL arithmatic operation

RFStuff

Dear All,

In SKILL I did as follows:-

a=1045.0/5.0

 =209.0

int(a)

 =209

But when I did

a=1.045/0.005

 =209.0

int(a)

 =208

Why int() shows different results in the above two cases ?

Kind Regards

  • Andrew Beckett
    There are quite a few similar posts to this. This is normal floating point rounding errors which you get in any language using IEEE floating point representation. Try using printf("%.15f\n" a) and you'll see the reason. Essentially the binary representation means that base 10 fractions can end up as a recurring sequence (like 1/3 does in base 10).

Community Guidelines

The Cadence Design Communities support Cadence users and technologists interacting to exchange ideas, news, technical information, and best practices to solve problems and get the most from Cadence technology. The community is open to everyone, and to provide the most value, we require participants to follow our Community Guidelines that facilitate a quality exchange of ideas and information. By accessing, contributing, using or downloading any materials from the site, you agree to be bound by the full Community Guidelines.