Re: [LAU] OT(ish): Strange coding problem (audio related)

From: Peter Nelson <peter@email-addr-hidden>
Date: Fri Jan 28 2011 - 14:03:56 EET

On Fri, 2011-01-28 at 12:28 +0100, Philipp Ãœberbacher wrote:

> Interesting.. would you mind explaining how this can be?
> How can 11-11 yield 4.80518e-16?

Because, rather than 11, log(2048)/log(2.0) is actually (to 40 d.p.)

7.6246189861593984035895533360399422488305 /
0.6931471805599453094172321214581765680755

But doubles don't contain 40 d.p.; it is approximately 16. Thus some
rounding errors may occur...

Peter.

_______________________________________________
Linux-audio-user mailing list
Linux-audio-user@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-user
Received on Fri Jan 28 16:15:04 2011

This archive was generated by hypermail 2.1.8 : Fri Jan 28 2011 - 16:15:05 EET