Subject: Re: [linux-audio-dev] pbd's votes on LADSPA
From: Paul Barton-Davis (pbd_AT_Op.Net)
Date: Mon Mar 27 2000 - 07:13:36 EEST
>I'm not familiar with whatever format you're talking about.
>Is it 24 bits or 32? i.e. is there data in the lower 8 bits that is not
>either zero or a replication of the higher order bits (a common technique of
>extending bit fields) ?
Its 24 bits packed into 32 bits, using the 3 most significant bytes
(well, actually, I suspect that its hardwired to use little-endian
ordering, but thats a detail). The lower 8 bits are always
zero. However, the sign bit is used as if the value were a 32 bit
integer. So it doesn't work to convert to 24 bits by bit shifting, at
least not if you want to write portable code since sign-extension
operations are inherently architecture/compiler dependent.
The rest of your message was somehow garbled. I got these 2 lines:
>equals bits out with a 24 bit mantissa in any case. So it doesn't matter if
>you convert to [-1,+1] or any other range reprensentable by a 32 bit float.
but nothing else. Even so, I think you may be wrong, for purely
mathematical reasons, but I am not particularly confident of my
reasoning. I'm a systems software not numerical person. The signed 24
bit integer value, when used as a sample value, is an implicit
fractional value between -1 and 1. The denominator is implied and is
either -(2^24) or (2^24)-1 (the min and max sample values) depending
on the sign of the value.
However, some of the implied fractions are irrational, and thus cannot
be represented accurately with floating point values of *any*
precision that do not use the correct base for the exponent. No IEEE
floats use either -(2^24) or (2^24)-1 as far as I know :). Other
values are not irrational but require more than 24 bits of mantissa to
be represented with an explicit denominator (2^N). In both cases, when
you convert to [-1,+1], you lose precision and can never get it back.
Am I way off base here ?
--p
This archive was generated by hypermail 2b28 : Mon Mar 27 2000 - 07:37:11 EEST