On Wednesday, August 03, 2016 08:59:15 PM Dmitri A. Sergatskov wrote:

> On Wed, Aug 3, 2016 at 7:41 PM, naryana.shankara <

[hidden email]
> > wrote:

> >

> > If I enter cot (pi/4)-1 I get 2.2204e-16

> > From my trig days I remember cot (pi/4) is 1 and 1-1 is 0

> > Octave also confirms cot (pi/4) is 1 and 1-1 is 0

>

> octave:6> format hex

> octave:7> 1

> ans = 3ff0000000000000

> octave:8> cot(pi/4)

>

> ans = 3ff0000000000001

>

> > so why does Octave spit out 2.2204e-16 when cot(pi/4)-1 is entered?

>

>

https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html>

>

> Dmitri.

Although cot(pi/4)-1 returns 0, tan(pi/4)-1 returns -1.11022e-16. One might

therefore suspect that cot() is not obtained by taking the inverse of tan(),

but 1 / (tan(pi/4))-1 returns 0, so the macro cot(x) = 1 / tan(x) may be used,

although I haven't checked.

To see exactly why these anomalies occur, one would have to look into the

floating point representations of the numbers involved and the procedure used

to compute tan(). The references given by others in this thread should be

helpful in pointing out the limitations of representing numbers in floating

point format.

Maynard Wright

_______________________________________________

Help-octave mailing list

[hidden email]
https://lists.gnu.org/mailman/listinfo/help-octave