Subject: Re: [linux-audio-dev] LAAGA - how are we doing ?
From: Paul Davis (pbd_AT_Op.Net)
Date: Fri May 04 2001 - 14:13:15 EEST
>> in a correct implementation, there are no function calls other than
>> the required one to actually copy the data to a single channel. when
>> compiled with optimization, a system based on aes inlines everything
>> all the way down. we have to do just one call-by-pointer indirection.
>> the function called is handwritten to move the data from
>> non-interleaved float format to/from whatever the channel actually is
>> (e.g. interleaved 16 bit, noninterleaved 24-in-32bit, etc.)
>
>I see a problem here - if we've got 4 plugins writing to one bus that
>happens to be a 16-bit bus, then we're converting float->short 4
>times, and mixing in 16-bit, which is not a very good idea. If more
>than one plugin is writing to a bus, then the bus needs to be a
>`float' bus, converting to 16-bit afterwards.
>
>Anyway, I thought this was going to be float all-through. Are this
>alternative format busses some kind of optimisation for efficiency ?
I don't know how you concluded that there was anything other than
floats everywhere. The only time we convert from/to a non-float is
when reading/writing to a physical audio output where the underlying
h/w isn't floating point (i.e. just about all audiointerfaces).
Everything else is float.
There's no way I can see to avoid the situation you describe with a
physical channel without keeping an entire set of buffers around to
"mirror" the physical channels and then copying them when the engine
cycle is done. I've tried to avoid this so far, preferring to work
directly into the mmap-ed "DMA buffer" of the audio interface. This
means that in the common case for ardour (which probably the most
demanding characteristics of an AudioEngine plugin), data moves
directly between the diskstream ringbuffer and the hardware "DMA
buffer" in one step.
OTOH, there are several reasons to consider the intermediate buffer
step, and its something I am definitely thinking about.
The "mixing in 16 bit" is precisely what I meant by "if you send to a
physical channel and the normalized sum exceeds 1, then you clip".
>However, in the case of working to/from a `float' bus, it actually
>forces additional copying because it stops the plugin from reading
>directly from an input bus or adding directly to an output bus.
yes, there is additional copying. but ... there has to be a copy
somewhere if there are LADSPA (or VST) plugins, since the LADSPA
plugins are run in-place. they cannot be connected to the output
channel (since they expect to be able to write to it directly in
non-interleaved float format).
it would be crazy having the internal objects working directly on the
(physical) channels, because they could not be (efficiently) portable
among different audio h/w with differing sample widths, interleave
status, etc. thats precisely what the engine is for. so yes, the
engine converts from the h/w format into float during
read_from_channel when the channel is a physical channel, and converts
back to the h/w format from float when the channel is a physical channel.
hence, nobody except the inside of the AudioEngine ever gets to have
any clue what the format of the channels are: they all appear to be
floats.
>> the whole point of request_channel() and release_channel() is that the
>> setup some internals in the server so that calls to
>> read_from_channel() and much more importantly write_to_channel() do
>> the right thing.
>
>I would guess that these are also to decide the order of processing
>plugins.
The "plugins" at this level are AES plugins, and no, they are run in
the order in which they attach to the AES.
This may change in the future. For precisely the reasons you get to next:
We're not permitting feedback loops, are we ? My impression
>from this talk of busses and stuff is that a plugin will process a
>chunk of data from one bus at time `t' and write it to another bus
>also at time `t' -- not at time `t+chunk_size'. This means that
>everything that writes to a bus must be processed before something
>that reads from it. This means that some combinations of plugging
>things together are illegal. Is this the idea ? Am I on the right
>track here ?
Yes. Each relay through a bus will cause a "frames_per_cycle" delay. I
am working on some ideas on how to fix this. Its non-trivial. The
solution is something like:
foreach bw in bus-writers {
bw->do_your_writing (nframes);
}
foreach br in bus-readers {
br->do_your_reading (nframes);
}
(note that a plugin that is doing bus-reading and bus-writing will
have to subdivide its own internal work this way too; clearly, some
combinations will always generate a cycle's worth of delay)
this breaks the simple, single "process(nframes)" model, which is why
i've been reluctant to go there. but i think it may be necessary.
>> >The use of the bus metaphor seems
>> >limiting, however. Why not have a complete signal flow graph similar to
>> >the Max family of languages?
>>
>> we do. its only the terminology that's hanging you up. i have been
>> drawing some diagrams (on paper, alas) to try to make this clear. here
>> is rough ascii rendition:
>
>I'm struggling a little here to understand - I was also imagining a
>signal flow graph kind of thing. I'm trying to find a model to help
>me understand how you're thinking about this. I'm wondering if this
>corresponds to a studio patchbay.
No. The diagram shown (for ardour) corresponds to a mixing console,
plus the connectors on the back. I don't want to encourage this
metaphor too much because it seems to distract people from the
possibilities of the model, but its the simplest way to think about
it.
In a typical mixing console, you have a series of channel strips. Each
strip has input(s), inserts, pan, gain and output(s). You set up the
mixer (either physically, for an analog board, or via software for a
digital system) so that a given mixer strip is:
1) receiving input from the place you want it to
2) sending output to where you want it to
a Route is roughly equivalent to a mixer strip.
So, the way I think of the channels supported by AES is a pool of
"line ins", and "line outs", plus a set of internal busses for doing
submixes and other tricks. When an application/plugin starts to use
AES, *it* (the plugin) is responsible for providing a way for the user
to select the channels to use. AES can't do it, or rather, AES can't
do it in a way that is necessarily appropriate for each
application. AES is "dumb" - it has the channels, but the application
has to do the "connecting". The UI for AES that I wrote allows only
the following operations:
* change the engine cycle (frames per cycle)
* change the sample clock sync mode
* request bypass monitoring ("All Monitor Input")
* pause/restart the audio engine
* quit
On hardware with optional support for h/w monitoring, there's a switch
for that as well. Buts that all. Don't confuse AES with Ardour. The
diagram I showed was for Ardour, not AES.
To try to summarize the diagram with words:
There are two kinds of objects that receive input from a channel,
a DiskStream and a Route. There is 1 kind of object that sends
output to a channel, a Route.
A Route may receive input from either a channel, or a
DiskStream.
The data received by a DiskStream is (sometimes) streamed to disk.
The data received by a Route moves through a processing flow, with
the option of the output(s) at each stage being sent back to a
channel.
--p
This archive was generated by hypermail 2b28 : Fri May 04 2001 - 14:50:08 EEST