Subject: Re: [linux-audio-dev] 64 bit float or integer
From: Steve Harris (S.W.Harris_AT_ecs.soton.ac.uk)
Date: Mon May 21 2001 - 15:13:57 EEST
On Mon, May 21, 2001 at 11:15:21AM +0200, Jörn Nettingsmeier wrote:
> Steve Harris wrote:
> >
> > You do have to be careful when converting back to int from float, as
> > soundcard_buffer[i] = (int)(plugin_buffer[i] * 32768.0f) won't do what you
> > want, its incedibly slow too, but thats another story.
>
> just out of curiosity, could you give details ?
Well the truncation looses you 1 bit of dynamic range and distorts:
float using cast using round
-0.000300 -9 -10
-0.000270 -8 -9
-0.000240 -7 -8
-0.000210 -6 -7
-0.000180 -5 -6
-0.000150 -4 -5
-0.000120 -3 -4
-0.000090 -2 -3
-0.000060 -1 -2
-0.000030 0 -1
-0.000000 0 0
0.000030 0 1
0.000060 1 2
0.000090 2 3
0.000120 3 4
0.000150 4 5
0.000180 5 6
0.000210 6 7
0.000240 7 8
0.000270 8 9
0.000300 9 10
You can get the round results using:
inline int fi_round(float f) {
f += (3<<22);
return *((int*)&f) - 0x4b400000;
}
But that will return ints, I'm not sure if you could modify it to return
shorts, but the int to short cast must be cheap.
- Steve
This archive was generated by hypermail 2b28 : Mon May 21 2001 - 14:32:21 EEST