Subject: Re: [linux-audio-dev] ladspa GUI round 2
From: David Olofson (david_AT_gardena.net)
Date: Sun Apr 15 2001 - 13:44:22 EEST
On Thursday 05 April 2001 01:14, Steve Harris wrote:
[...OSC...]
> I still believe that string passing is neccesary for file loading
> etc. And that the GUI needs to be able to deal in comms with the
> DSP plugin that can't be expressed in LADSPA ports. I am open to
> being convinced otherwise however.
Speaking of string passing and LADSPA ports (or other non-function
call mechanisms), note that passing non-constant strings requires
special attention. I discovered this the hard way when trying to
figure out useful semathics for some MAIA Protocols. :-)
The problem becomes most obvious when thinking about a plugin
expected to respond to a request by sending an internally generated
string [or a buffer of some other non-constant data] somewhere. I can
see two ways of doing it, but I don't like either of them:
1. Use dynamic memory allocation to create the data buffer,
fill it in, and send it off.
2. Make a function call to the transport layer (which isn't
normally done during the execution of a MAIA plugin
callback - nearly everything is done with simple inline
code), making sure it grabs the data, copying it to some
safe place for later use if required.
The first approach could be managed internally by plugins (some sort
of buffer cache), emulating the nice and simple const behavior (ie
just pass the buffer by reference), but that would require that
someone tells the plugin when each buffer becomes free for reuse.
Even, worse, that might not even happen during the lifetime of the
plugin, which means that the plugin must allocate it's "internal"
buffer as separate memory blocks that can be left around if neeed.
(Note: This problem applies to const strings as well, but in that
case, you can just pipe string passing events through the host to
copy the strings, or lock the const data areas of all used plugins
until it's certain that no one holds a pointer to data inside them.
Simple hack for the basic host: Never unload plugins that can send
references to const data once they've been used...)
Now, MAIA already has a Protocol that handles sharing, transferring
etc of buffers, including allocating new buffers from the host (hard
RT if the host can provide it), but this Protocol wasn't meant to be
required for such a basic thing as passing strings around. I don't
see a nice way of supporting non-const strings without it, though;
hardly even const strings...
> > 3. When does everything happen? How does operation differ when
> > running offline to when running live?
>
> It doesn't as far as I'm concerned. There should be nothing
> stopping a GUI from being used even if it is not connected to a DSP
> part.
Although it might be a very good idea making sure that a GUI plugin
doesn't get the idea that it should refresh every N samples, thinking
it's visualizing a 44.1 kHz stream, while it's in fact hooked up to a
plugin that runs off-line 10 times faster... :-)
> > 4. Does the plugin exist within the host's GUI or externally? ...
> > and how?
>
> Externally. It is running in its own process, so it has it own
> connection to the X server (if thats what it wants, could equally
> be doing MIDI things or whatever)
IMHO, running the GUI externally shouldn't be the only option. Just
as for real time audio processing, tons of individual rendering
processes will cause trouble when trying to achieve smooth real time
visualization.
There is the X/toolkit/event loop issue, though... Can only be solved
"nicely" with a VSTGUI style wrapper or toolkit, it seems.
> > E [executable]. Plugins exist in a common directory as binary
> > files that can be run to extract parameter/configuration
> > information and then executed by hosts locally or remotely.
> > Communication is then maintained using OSC, IPC, sockets or pipes
> > to stdin/stdout. This could be quite fast and makes the
> > multitasking aspect explicit. Problems:
> > i. The overhead for host and plugin design might be quite
> > significant (although a library of standard plugin code could be
> > provided).
>
> I haven't tried to measure this, but I supect it isn;t that bad,
> given that a user is not going to have a large number of GUIs open
> at one time.
Hmm... What is "a large number of GUIs"? :-)
I can fit an awful lot of XMMS size GUIs on this screen, and I can't
say I'll never want to actually do it... I just hate fiddling with
windows. It's bad enough as it is with a single mouse with 4 buttons
without adding another level of frustration in the form of switching
views or something similar. Too many keystrokes and/or clicks per
operation.
[...]
> > v. Potentially high start-up cost (relatively).
>
> This is certainly true, I don't know if it will be an issue or not
> though.
Start-up cost seems to be high for anything that isn't either very
simple and/or will turn out to be very messy and unhelpful in the
long run... If nothing else, it takes time to figure out how to use
the existing tools together in the right way.
> > vi. Makes me nervous for other reasons I've not put my finger on
> > yet.
>
> Fair enough ;) Seems like the right thing to me, leaves the GUI
> designer a free hand to do whatever they want, and the user a free
> hand to use whatever GUI they want.
This is a double-edge sword. Choice is Good, but it doesn't come for
free - and the end users are usually the ones who have to pay the
price.
Now, in this case, this is basically about whether or not it's right
to expect users to run X (windowed mode; ie not DGA fullscreen or
anything like that) and install whatever toolkit every plugin uses,
or not. As long as the user *is* running X in windowed mode, this
isn't much of an issue, but anything else basically means that no
custom plugin GUIs will work.
> > SOME TEST CASES
>
> I'l argue these in favour of the seperate process + OSC approach...
>
> > A working API should ideally handle all the following (somewhat
> > random) cases. Standardised wrappers for the existing LADSPA
> > layer are required for a couple of them (e.g. a MIDI Sysex
> > convention), but nothing controversial.
> >
> > Some hosts (imaginary, any similarities coincidental etc etc):
> >
> > 1. Rackmount 386 FX processor with custom low-latency kernel,
> > small graphical LCD display and no X-Window.
>
> The seperate process approach would work well here (SDL + bitmaps
> or whatever). Only one GUI process would be runnign at any one
> time, and you are free to use whatever drawing library you like. Of
> course the GUI would need to be coded specially, but I htink that
> is unavoidable.
Not if you have some API like VSTGUI, possibly with a canvas widget.
Could be implemented on top of any of the major toolkits, or directly
against SDL, GGI, svgalib, DirectDraw, GDI, DrawSprocket, OpenGL or
whatever.
We're not talking about a toolkit for normal applications here (ie
not GTK+ or Qt), but rather a dynamic form of WinAmp/XMMS skins, so
this shouldn't be too huge an effort to implement, compared to a
"normal" toolkit.
> > 2. The above, but without the LCD display. Has MIDI though...
>
> Ditto, but the GUI is infact MIDI CC's or whatever.
IMHO, MIDI is more like a "Driver Plugin" in MAIA terms. Then again,
a plugin GUI is also a Plugin in MAIA terms, and both are very
similar to DSP plugins from the API POV. (Same module loading system,
same event transport layer, same low level protocols etc. You could
actually do it all in a single Plugin, although a hard RT host would
most probably have plenty of evil thoughts about such a plugin.)
> > 6. The host system has a core that runs on a remote Sun box while
> > its GUI component runs and displays on a local machine over a
> > busy network (while streaming audio & video for a gig).
>
> This could be achieved either via X (running the GUI processes on
> the sun and dipslaying on a remote x server, or by running the GUI
> processes on the remote machien and connecting via OSC.
This is basically designed into every MAIA Protocol, so it should be
possible to place the GUI plugin on either system - just put it where
it'll produce the least traffic over the network. (Of course, this
assumes that the GUI plugin is using a Protocol that corresponds to a
toolkit or rendering API, rather than directly doing rendering using
some API or toolkit.)
> > Some plugins:
> >
> > 5. Audio plugin is a 512 channel spectrum analyser, GUI plugin is
> > to display these on meters.
>
> I nice implementation would be using control outs from the DSP part
> and rendering those values graphically.
Yep. Custom protocols are strongly discouraged, and this looks like
just another case where standard interfaces can be used. Make it a
512 subchannel interleaved stream "port" if you don't want 512
individual connections.
> > 6. Audio plugin is a synth with four different FM configurations
> > and eight envelope generators, all of which require graphical
> > editors with five breakpoints.
>
> You could build a paged GUI with whatever toolkit took your fancy.
That introduces some potential GUI logic problems that you may not
want to deal with. What happens if you flip to the "Advanced" page,
adjust some parameters, flip back to the "Easy" page and then adjust
some parameter that wraps the "Advanced" parameters you just changed?
I've seen a few attemps to solve this in various applications, and
have been known to express non-printable opinions about some of
them... :-)
> > 7. Audio plugin is a complex general purpose filter with two GUI
> > plugins. The first GUI plugin has just a cutoff frequency and
> > resonance and sets the filter coefficients to operate as a
> > Moog-style resonant low-pass filter. The second GUI plugin
> > presents vowel diagrams to the user and sets the filter
> > coefficients to generate the appropriate formants.
>
> This is where its gets a bit vague, we havn't really discussed how
> the hosts finds GUI binaries that are used with a given DSP plugin.
With MAIA GUI plugins, it would be possible (but perhaps not very
useful) to just gather the interface info from all GUI plugins, and
make a list of matches with DSP plugin interfaces. Controls are
described using an enumeration similar to the MIDI CC definitions,
and something similar should be used for other things that doesn't
have an implicit "kind" quality. Perhaps I'll just throw it all into
the same enumeration, or use strings or char[8] identifiers or
something...)
Anyway, it's probably a very nice idea to have a way of linking GUIs
and DSP plugins together. It's probably more useful for the GUIs to
reference DSP plugins than the other way around, although both styles
have some uses... (Simple/advanced GUI for a single plugin vs. common
GUI for a bunch of plugins with the exact same parameters.)
How about allocating unique IDs for GUIs as well as DSP plugins, and
then allow both GUIs and DSP plugins to provide a list of sensible
matches? (Of course, GUI plugins could be assumed to have IDs from a
separate pool, but that seems like a *very* bad ide, considering that
MAIA is using the same low level plugin API for both kinds of
plugins. Might couse serious plugin management and connection
trouble, especially in single threaded hosts.)
//David
.- M A I A -------------------------------------------------.
| Multimedia Application Integration Architecture |
| A Free/Open Source Plugin API for Professional Multimedia |
`----------------------> http://www.linuxaudiodev.com/maia -'
.- David Olofson -------------------------------------------.
| Audio Hacker - Open Source Advocate - Singer - Songwriter |
`--------------------------------------> david_AT_linuxdj.com -'
This archive was generated by hypermail 2b28 : Sun Apr 15 2001 - 15:21:17 EEST