Re: [LAD] strategies for mixing streams of differing channel counts

On Sat, 2 May 2015, Adrian Knoth wrote:

> On Sat, May 02, 2015 at 08:30:50AM +0000, dave@email-addr-hidden wrote:
>
>> Could I get some ideas and advice on the subject of reconciling two
>> streams with different channel layouts?
>
> Hold on, weren't you the guy who couldn't mix two audio streams into one
> three days ago, and now you want to address surround?

I just want to make sure my Z-machine implemenation can sensibly deal with
whatever audio is thrown at it. The Z-machine was designed in the late
1970s. When it was extended in the 1980s to have sound effects, it was
mono only. Later came reverse-engineering the Z-machine and new
implementations. The Blorb spec was introduced to deal with the messy
problem of how to package up the game file along with sound files. By
including MOD and OGG in the spec, it implies that stereo is to be
expected. If the spec allowed only for AIFFs, I'd be done because there'd
be no mixing and I wouldn't touch the channel settings. However, the OGG
spec allows for surround sound. That's where I want to ensure
compatibility.

The engine has no capability of putting a sound in a particular place
within a surround sound environment. If that's desired, the author has to
work that out when creating the samples and the game itself. I don't know
if the Glulx VM is any different in this regard. I haven't dug that
deeply into its audio capabilities.

> How about taking a step back and getting the basics, first? Correct
> surround sound is a delicate subject, and entire companies have been
> founded to get it right.
>
> If you really want to, read some basics about positioning sound sources,
> maybe start with the many papers on spatial audio from the past Linux
> Audio Conferences. It won't get you anywhere near, but it will get you
> started. Expect some math to be involved in this.
>
>
>> with both channels carrying the same data. Mixing a one or two
>> channel stream with 5.1 or 7.1 also seems fairly clear: the audio
>> goes to the front left and right channels. Is this line of thinking
>> solid?
>
> No. Upmixing from stereo to any kind of surround can be arbitrarily
> complex. If you want it centered, send it to front-left, front-right,
> rear-left and rear-right with different gains depending on your desired
> position. Sum it to mono and apply an LPF with an appropriate cut-off for
> your .1 bass channel.

Unfortunately the Z-machine has no mechanism, intentional or not, to tell
the interpreter how a sound should be placed.

>> I understand that the chances of someone writing a game that would
>> go beyond stereo are rather slim,
>
> Quite the contrary, the industry is already doing it:
>
> http://www.iosono-sound.com/game-audio/

As I stated before, the Z-machine is quite limited in this regard.

>> I'm also considering extending the results of my work into a
>> general-purpose mixer library.
>
> Have you already figured out what mix buffers are and how you keep a
> single stereo stream open that is fed from multiple sources? To save you
> some time, here's how to do it properly:
>
> * keep the stream open all the time - send zeroes if you have nothing
> to output
>
> * don't rely on external software (like pulse) for mixing multiple
> streams. That said, send a single stereo stream from your
> application.

I detest pulse. That's a big reason why I want to mix internally.

> * use float32 as the internal representation, with audio normalised to
> -1.0 .. 1.0.
>
> * mixing is adding two floats:
> float a, b, mix; mix = a + b;
>
> * volume is multiplying with a scalar (usually 0..1, occasionally
> larger): float in, out, gain; out = in * gain;
>
> * you might find non-interleaved channels easier to handle:
> float left[512], right[512] instead of float stereo[1024]
>
> * ask your libs (like libsndfile) to return you floats, then sum the
> individual buffers and hand the result to the output stream. If need
> be, lower the volume as a final step.

My code that handles only AIFF samples works like this.
[snip]
> Cheers and good luck

Thanks.

-- 
David Griffith
dave@email-addr-hidden
A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev@email-addr-hidden
http://lists.linuxaudio.org/listinfo/linux-audio-dev