Stats

  • Locked Locked
  • Replies 1
  • Subscribers 143
  • Views 2088
  • Members are here 0
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Floating number precesion in OCEAN/CIW

greatqs

I found a floating number in OCEAN or CIW can has only 6 digits precision after decimal point.

For example, in OCEAN shell, I type 1.123456789 but I get 1.123457 actually. So I get some calculation error when dealing this kind of floating point.

 

Could anyone advice me how to avoid this issue? Thanks a lot!

 

  • Andrew Beckett

    Actually the numbers in OCEAN (and SKILL, since OCEAN is SKILL) are stored in double precision floating point - which means they have about 15.5 decimal significant figures. What you're seeing is that the default print representation in SKILL is 7 significant digits. For example:

    defMathConstants('myMath)
    myMath.PI => 3.141593
    printf("%g\n" myMath.PI) => 3.14159
    printf("%.15g\n" myMath.PI) => 3.14159265358979
    sstatus(fullPrecision t)
    myMath.PI => 3.141592653589793

    So the sstatus fullPrecision flag can be set so that by default floating point numbers are displayed with full precision rather  than being output with "%.7g" format.

    You will not get rounding errors at the 7 digit level because of this normal truncation, since it's only at output. For example, if I do:
    myMath.PI-3.141592 =>  6.535898e-07

    so you can see the next digits being revealed. You will of course get rounding errors down at the end of the 52 bit mantissa though (i.e. around 15.5 digits).

    So it's not an issue to do with rounding, merely how the numbers are displayed.

    Regards,

    Andrew.

Community Guidelines

The Cadence Design Communities support Cadence users and technologists interacting to exchange ideas, news, technical information, and best practices to solve problems and get the most from Cadence technology. The community is open to everyone, and to provide the most value, we require participants to follow our Community Guidelines that facilitate a quality exchange of ideas and information. By accessing, contributing, using or downloading any materials from the site, you agree to be bound by the full Community Guidelines.