trying to understand why I get 2.2204e-16 rather than 0

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

trying to understand why I get 2.2204e-16 rather than 0

naryana.shankara
If I enter cot (pi/4)-1 I get 2.2204e-16
From my trig days I remember cot (pi/4) is 1 and 1-1 is 0
Octave also confirms cot (pi/4) is 1 and 1-1 is 0

so why does Octave spit out 2.2204e-16 when cot(pi/4)-1 is entered?
Reply | Threaded
Open this post in threaded view
|

Re: trying to understand why I get 2.2204e-16 rather than 0

James Sherman
On Wed, Aug 3, 2016 at 8:41 PM, naryana.shankara
<[hidden email]> wrote:

> If I enter cot (pi/4)-1 I get 2.2204e-16
> From my trig days I remember cot (pi/4) is 1 and 1-1 is 0
> Octave also confirms cot (pi/4) is 1 and 1-1 is 0
>
> so why does Octave spit out 2.2204e-16 when cot(pi/4)-1 is entered?
>
>
>
> --
> View this message in context: http://octave.1599824.n4.nabble.com/trying-to-understand-why-I-get-2-2204e-16-rather-than-0-tp4678993.html
> Sent from the Octave - General mailing list archive at Nabble.com.
>
> _______________________________________________
> Help-octave mailing list
> [hidden email]
> https://lists.gnu.org/mailman/listinfo/help-octave

Some background reading is probably the best:
https://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
Note the reason you're getting isn't Octave so much as a property of
floating point numbers.

Really the whole article is pretty good, but it boils down to
essentially that representing most numbers is an approximation because
of finite memory with decimal representation, let alone irrational
ones like pi.  This isn't even getting into things like how trig
functions are calculated.  To show that it is even more fundamental
than that, try

>> (1-1/3)-2/3

and see the result you get.

James Sherman

_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: trying to understand why I get 2.2204e-16 rather than 0

Dmitri A. Sergatskov
In reply to this post by naryana.shankara
On Wed, Aug 3, 2016 at 7:41 PM, naryana.shankara <[hidden email]> wrote:
If I enter cot (pi/4)-1 I get 2.2204e-16
From my trig days I remember cot (pi/4) is 1 and 1-1 is 0
Octave also confirms cot (pi/4) is 1 and 1-1 is 0


octave:6> format hex
octave:7> 1
ans = 3ff0000000000000
octave:8> cot(pi/4)
​​
ans = 3ff0000000000001

 
so why does Octave spit out 2.2204e-16 when cot(pi/4)-1 is entered?

​Dmitri.
--
 


_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave
Reply | Threaded
Open this post in threaded view
|

Re: trying to understand why I get 2.2204e-16 rather than 0

Maynard Wright
On Wednesday, August 03, 2016 08:59:15 PM Dmitri A. Sergatskov wrote:

> On Wed, Aug 3, 2016 at 7:41 PM, naryana.shankara <[hidden email]
> > wrote:
> >
> > If I enter cot (pi/4)-1 I get 2.2204e-16
> > From my trig days I remember cot (pi/4) is 1 and 1-1 is 0
> > Octave also confirms cot (pi/4) is 1 and 1-1 is 0
>
> octave:6> format hex
> octave:7> 1
> ans = 3ff0000000000000
> octave:8> cot(pi/4)
> ​​
> ans = 3ff0000000000001
>
> > so why does Octave spit out 2.2204e-16 when cot(pi/4)-1 is entered?
>
> https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
>
>
> ​Dmitri.


Although cot(pi/4)-1 returns 0, tan(pi/4)-1 returns -1.11022e-16.   One might
therefore suspect that cot() is not obtained by taking the inverse of tan(),
but 1 / (tan(pi/4))-1 returns 0, so the macro cot(x) = 1 / tan(x) may be used,
although I haven't checked.  

To see exactly why these anomalies occur, one would have to look into the
floating point representations of the numbers involved and the procedure used
to compute tan().   The references given by others in this thread should be
helpful in pointing out the limitations of representing numbers in floating
point format.


Maynard Wright


_______________________________________________
Help-octave mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave