Subject: Re: [linux-audio-dev] [ardour] custom graphics + MIDI control for LADSPA plugins
From: David Olofson (david_AT_gardena.net)
Date: Fri Dec 01 2000 - 01:59:44 EET
On Thursday 30 November 2000 15:13, Steve Harris wrote:
> On Thu, Nov 30, 2000 at 04:43:24AM +0100, David Olofson wrote:
> > Audio ports have Data Type Descriptors, that hosts have to match
> > when making connections. Converter plugins, libraries or inline
> > code can be used to convert data if needed.
>
> I've allready made my feelings clear on this point, and no-one has
> yet given a good reason for it.
Right, there is no good reason for integer formats and the like for
audio these days, as long as you stick to *professional* quality
music systems.
However, there are many other places where plugin systems can be
used. One size does not fit all indeed, but if we have LADSPA and
then add MAIA, and *still* cannot cover anything but pro audio and
very similar forms of signal processing on high end CPUs properly, I
think we're headed in the wrong direction.
That kind of specialization is simply not called for in this case,
IMHO, especially as there is LADSPA when you can't accept and/or
don't need an extra integer field per port for data type management,
the timpstamped event system that MAIA is entirely built around and
so on.
> > I'm thinking about allowing plugins to use these for some kinds
> > of dynamic allocation. The event pool can actually be used that
> > way right now - call it a bonus feature...
>
> If the event pool is in 32byte chunks it wouldn't be fast enough
> for use in a lot of plugins, you really need big lumps of
> contiguous RAM for wavetables and stuff.
Right. If I'll officially allow this at all, it's for sequencers and
other plugins to buffer up events and other small structs internally;
not for audio data. (It would be insane to allocate an event pool big
enough for that, especially as this pool cannot safely be extended at
run time in RT hosts!)
> > > - data ranges - currently both control data and audio
> > > data are not normalized to any specific range"
> >
> > This can't really be done in any useful way with audio plugins.
> > The most sensible thing along these lines I can think of is
> > dynamic integer formats. That is, basically FP formats where an
> > entire buffer shares a single "exponent" field. That's a
> > nightmare to work with, and doesn't belong inside DSP nets.
>
> Buts thats what DSP programmers spend a lot of time doing, ensuring
> that an array of floats shares the same exponent allows you to
> shortcut table lookups, for example.
What has this got to do with the signals that are routed between
plugins? I wouldn't think it makes sense to renormalize all audio
data before feeding it into a plugin, or to require all plugins only
to output normalized data... but perhaps it does, or what's your
point?
> > *What* do people actually want with this? I've still not seen any
> > sensible mathematical or other explanation why one would want a
> > normalized range. To me, it looks like "math utopia", but I might
> > still be missing the point...
>
> A fair number of audio techniques *need* bounded range input. Of
> the top of my head: ringmodulation, waveshaping, distortion, click
> removal, dynamics processing and transient processing.
Uhm, "normalized range" cannot have been what I meant to write.
"Normalized signal levels" was what I meant.
Anyway, sure, that kind of plugins need to know in what approximate
range the input signals will be - that's the point with the 0 dB
level.
As to "bounded", I can't help interpretting that as "limited" or
"clipped", which definitely doesn't make sense in FP systems.
> > MAIA will simply define a "recommended 0 dB level" for signal
> > data types, so that audio plugin writers will have some idea
> > about what to expect.
>
> This would probably be enough, but ideally the hosts should be
> required to conform to it (obviously other plugins might junk the
> data, but thats another problem). Floats bounded to (-1,+1) are
> ideal, as that is what 95% of DSP routines are designed with.
So you basically have to clip signals, or apply some AGC algorithm on
them if the should exceed this level? I don't see how that could
work, and I don't see any similarities with analog signal processing.
Yes, indeed many plugins will clip or otherwise distort the signal if
it exceeds a certain level, but I don't see how that maps to
requiring that plugins can never ever get such signals. On the
contrary, to me that sounds more like breaking the engine, making it
appear strange and very different from what users are used to.
Besides, why should the host waste time on this when dealing with
plugins that *don't* care about absolute levels?
(Remember that gain controls, EQs, modulators and many other small
units are also plugins! MAIA is designed to make it possible to
implement practically *any* application using only plugins in a
standard engine, so there should basically be no signal processing at
all built into the host. This design is a fundamental requirement for
the ReWire style "run an application as a plugin" architecture to
work.)
> > > As far as I remember, only one serious problem came up. James
> > > McCartney reported that his SuperCollider software absolutely
> > > requires 3 (plugin instantation must be rt-safe).
> >
> > This is part a plugin implementation issue, part an API issue.
> > The host may have to provide RT memory management, plugins need
> > to be deterministic WRT instantiation time/CPU usage, and plugins
> > have to tell the host about it for any of this to make any sense
> > without forcing *all* plugins to be RT instantiation capable.
>
> Why shouldn't plugins all be RT instantiation capable? Usually all
> they need is an RT safe way of allocating memory. All my plugins
> would be RT safe in instantiation execpt that I have to call malloc
> a lot.
I agree that it would be nice if all plugins were RT instantiation
capable, but I don't think it's all that easy. The memory management
is tricky to implement, but OTOH, it's a do-once thing on the host
side. We could even hack up one or two reference implementations that
people could paste right into their hosts. (If anyone is to be
writing a host at all, except for the system wide one, that is - see
above on the ReWire stuff.)
However, just dealing with the memory management isn't enough - truly
hard RT instantiation capable plugins must also be able to do all
init processing (create, init, prepare, start etc...) in about the
same time as an average run()/process() call in the actual net. This
might be a problem for plugins with large buffers or LUTs that are
generated at init time, especially in very low latency settings. (A
buffer could be 32 *bytes*...) This becomes very critical in systems
that may occasionally instantiate lots of plugins during the same
cycle.
Now, one could "cheat" some, and accept that some plugins need one or
more cycles to fully initialize, but that would require that the init
code can be chopped up in arbitrary size bits. Possible but not
beautiful...
Another solution would be to run plugins with too heavy
initialisation in another thread, but that adds another chance of the
OS scheduling latency hitting the engine by occasionally delaying the
initialization of plugins. That is, instantiation easilly becomes
undeterministic, even if the plugins *can* instantiate in a well
defined amount of time!
Basically, the only way a plugin can be hard RT instantiation capable
is that it's instantiation time
* is bounded and deterministic
* is in the same order as the average run() time
* scales with the buffer size
Note that this still implies that it takes one engine cycle to
instantiate a plugin! Basically, that means that a soft synth (or
whatever) will add one fragment to the latency for any operation that
requires a plugin to be instantiated. Both instantiating *and*
processing one buffer would take too long...
Well, yeah, hosts may be clever about this by somehow limiting the
number of plugins that can be instantiated within a cycle, and then
allocating some CPU time margin to use for instantiating plugins
before the net is processed. However, this starts too remind of the
older generations of hardware synths, where it took several ms to
start one note, and only one note at a time could be started, due to
underpowered MCUs...
> > Right; I don't see how the above can be incorporated into LADSPA
> > without giving it the VST 2.0 syndrome (multiple interfaces with
> > overlapping functionality), and I don't see how anything with a
> > sample accurate event system can be as simple and efficient as
> > LADSPA.
>
> Actually, with sample accurate events, you could write some really
> efficient stuff, one of the main speed problems with LADSPA is the
> tradeoff between detecting parameter changes and recalculating
> tables etc. for every chunk.
Well, that's the point with the event system; it's very efficient
when it's put to work. You can rest assured that nothing unexpected
will happen without both plugin and host knowing about it.
What I meant was that it can't ever be as fast to queue up an event
that is later decoded and interpretted by the plugin, as it is to
simply write a new value to a LADSPA control port.
But considering the late thread on parameter interpolation and that
kind of stuff, it might just be that the event system will eliminate
checking and/or unnecessarry calculations in the majority of quality
plugins, thus being faster nearly all the time.
//David
.- M u C o S -------------------------. .- David Olofson --------.
| A Free/Open Source | | Audio Hacker |
| Plugin and Integration Standard | | Linux Advocate |
| for | | Open Source Advocate |
| Professional and Consumer | | Singer |
| Multimedia | | Songwriter |
`-----> http://www.linuxdj.com/mucos -' `---> david_AT_linuxdj.com -'
This archive was generated by hypermail 2b28 : Fri Dec 01 2000 - 04:23:46 EET