1
$\begingroup$

I run these lines:

a = 0.833
SetPrecision[a, 20]

and this is the output:

0.833

0.83299999999999996270

I expected to receive

0.83300000000000000000

Do you have any suggestion? Thank you!

$\endgroup$
8
  • 2
    $\begingroup$ 0.833'20 gives what you expect, but SetPrecision[a,20] is equivalent to N[FromDigits[RealDigits[a,2],2],20] = 0.83299999999999996270. MMA works with binary numbers. $\endgroup$ Commented Aug 25, 2019 at 16:40
  • $\begingroup$ It works with 0.833'20, but not with a'20 $\endgroup$ Commented Aug 25, 2019 at 17:59
  • $\begingroup$ The back tick is used in specifying numeric input. Using it on variables thus a`20 is not syntactically valid. The problem is that a = 0.833 evaluates 0.833 in machine precision binary and obtains the binary floating point number nearest to 0.833, but that number is only approximately equal to 0.833. $\endgroup$ Commented Aug 25, 2019 at 20:34
  • $\begingroup$ Related: mathematica.stackexchange.com/questions/55292/… $\endgroup$ Commented Aug 25, 2019 at 20:38
  • 1
    $\begingroup$ You don't need a'20. Just use a=0.833'20 and a will have 20 digits of precision from then on or until it is reassigned. That is supposed to be a back tick, but I can't write it that way because of the way comments treat back ticks. $\endgroup$ Commented Aug 25, 2019 at 21:58

1 Answer 1

1
$\begingroup$

I suggest SetPrecision[833/1000 , 20] or SetPrecision[.833 // Rationalize, 20].

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.