Subject: Re: [linux-audio-dev] Random thought on HDR latency compensation
From: Karl JH Millar (kmillar_AT_MIT.EDU)
Date: Wed Apr 26 2000 - 04:19:00 EEST
Hi Tom,
A while ago, when I had a Cyrix CPU, I started coding a multitrack mixer
(which I intend to do an initial release of shortly). Since Cyrix CPUs
have *terrible* FPU performance, I wanted to be able to do all the processing
using integer math. You can get away with doing this, but it involves writing
all your processors to multiply your signals with fractions. In order to
do this with errors that won't appear in 16bits, you need around 10 bits
in your numerator and denominator (using a partial fraction approximation; also
its much worse if the number you're using is close to an integer, which happens
in most low frequency filters. In that case you need to rearrange things
manually to get the filter to work at all).
On a 32 bit machine then, you can get 16bit resolution and 6bits of headroom.
This is barely adaquete.
Using floating point if possible is a fairly big win. Headroom is not an
issue, and with a long enough mantissa, the precision is there. I really
doubt that the rounding errors ever dominate the original quantization error
(especially since very few A/D/A converters are linear down to the last bit),
although I haven't personally checked if they add up in EQ filter code (I
doubt it's significant though). It's also worth remembering that quantization
or rounding errors are at -96/-144dB (for 16/24 bits respectively). There isn't
a whole lot of analog equipment that can claim noise thresholds that low
(although I haven't been inside a nice studio for a while). I think work in
digital recording has bigger problems with getting algorithms that sound really
nice than in getting the very last bit clean.
Anyway, as for my code, I have now acquired a CPU with better floating point
facilities, and simplified my code by dropping integer, and going to all
floating point arithmetic.
Karl.
This archive was generated by hypermail 2b28 : Wed Apr 26 2000 - 08:56:11 EEST