Subject: Re: [linux-audio-dev] Synth APIs, GluttonAudio
From: David Olofson (david_AT_olofson.net)
Date: Fri Dec 13 2002 - 22:17:00 EET
On Friday 13 December 2002 20.07, Sami P Perttu wrote:
> On Fri, 13 Dec 2002, David Olofson wrote:
> > > What is that? If a plugin got an audio port event it would just
> > > update a pointer. Past samples should be kept in temporary
> > > variables and future samples should not be looked at.
> >
> > Correct. But consider this example:
> >
> > out[s] = left[s] + right[s];
> > ++s;
> >
> > Now, if you get a new buffer for left in the middle of the
> > buffer, what happens...? This is what I'd call "non-obvious
> > implications".
>
> Oops, I didn't consider that. Solution: require that pointers point
> to start of block even though there may be no data there :).
Well, you'd have to adjust the pointer yourself, since the host can't
know how your plugin is implemented.
And BTW, the host that does this kind of stuff may well bo doing it
as a way of avoiding data copying to merge fragments into complete
buffers. Then, there is no obvious logic in this at all. :-)
> > IMHO, this is complicating the API for no good reason.
>
> Then we are back to square one: multiple versions of plugins that
> differ only in control and audio port details.
Why? Sample accurate buffer splits won't help, since it only applies
to Audio Ports. You need control<->audio converters either way. There
is no way we can merge audio and control into one kind of port
without either doing it all as audio rate streams, or actually doing
conversions somewhere, or forcing plugins to have "dual" inputs.
> If the plugins will,
> as you say, indeed be higher level than CSound then it won't be a
> problem.
It will be much *less* of a problem, but I'm 100% conviced that
people will still want to do "weird" stuff with the plugins. Even VST
users dream about this! ;-)
(They might even be doing it to some extent; dunno... MIDI
CCs<->audio? Hahaaa! :-)
> > > I don't see anything physical about audio ports in XAP,
> > > although it might be better if they were actual objects.
> >
> > typedef XAP_sample *XAP_buffer;
> >
> > Do you need an object for that? :-)
>
> No, sorry, I should make myself clearer. I was thinking about the
> pull model again.
Ah, I see. Well, see my post on asynchronous streaming; that's
basically what you get if you want to do it in a real time context.
(And that's not even a complete solution, as it doesn't include QoS
control.)
> > BTW, I am indeed going to look into some sort of "process only
> > this range" feature for the rendering engine. Some sounds take a
> > while to render, and there's no point in waiting for the whole
> > sound to render when you're just tweaking the attack.
>
> ...or let the rendering run in a background process.
Doesn't work, because the current system is still totally imperative,
and not block based. One statement at a time operates on the whole
buffer, so no part of the waveform is ready for playback until the
script has returned.
Speaking of which, switching to implicit full script block based
processing might be a rather performance hack. Should reduce cache
thrashing quite a bit!
> > How? It's not the host that sends these events in general; it's
> > other plugins. The host never touches the events in the normal
> > case.
>
> Okay. I'm lost here because I don't know what a XAP app would look
> like. I only know trackers :).
Well, it's not *that* different. Just think of the different parts of
a tracker (MIDI input, song, pattern, "classic" effects, synth, DSP
effects, audio output) as separate plugins, running under a host that
does pretty much nothing but loading, connecting and running plugins.
The sequencer becomes a plugin, rather than a part of the host, and
you can do the same with I/O and pretty much everything.
> > > Logically, controls are like ports: they have a definite value
> > > at each point in time.
> >
> > You *could* say that - but don't forget that many plugins are
> > oversampling internally...
> >
> > internally. As a result, the filters and oscillators see zipper
> > noise
> >
> > Now, if control was seen as continous rather than sampled, it
> > would [...]
> > That said, the problem is that you cannot get away without
> > actually implementing this. If there are cubic interpolation
> > events, all [...]
> > Well, you could just use the extra argument as "slope", to
> > suggest the desired slope when "target" is reached. So, for
> > chained sections, you get <value, slope> for start and end of
> > each section. Either just ignore the slopes and do linear
> > interpolation, or do something more sophisticated, to take the
> > slopes in account as well. (Basic splines; "smoothest path".)
>
> What happens between sample values. Now we are too far in the
> hair-splitting territory :).
*hehe*
Does matter with some oversampled algos, though. They may not be very
happy at all with the zipper-noise, since it lands well inside the
frequency range the algo is supposed to work with. This may result in
distortion that affects the final downsampled result.
Think of it as running all audio at 48 kHz, but updating controls
only every 3rd sample. (The "feeping" should be pretty annoying at
that rate.) Now, downsample to 16 kHz. Can you be *sure* that all
distortion goes away...?
> If we are dealing with 3rd order polys then both point-slope and
> coefficient formats contain the same information. I guess we should
> pick a representation that allowed lower-order approximations to be
> made efficiently. If plugins can't handle anything but constants
> they would use the average value; linearly ramping plugins would
> use the initial value and the slope as you describe. 2nd order
> approximation is probably useless. Plugin picks the order and
> invokes a macro to get the coefficients. Conversion between 3rd
> order poly coeffs and point-slope is pretty trivial. Take your
> pick. Perhaps some spline expert could comment on this?
Yeah, suggestions are welcome. Something that easily translates to
"coefficients" for this:
for(...)
{
...
value += dvalue;
dvalue += ddvalue;
ddvalue += dddvalue;
}
or this:
for(...)
{
...
value += dvalue;
}
where both should start and end at the same respective values.
If you have:
start_value, start_dvalue (from current state)
end_value, end_dvalue (from spline event)
segment_length (from spline event)
(*_dvalue are the slopes at the respective *_value), it's trivial to
do linear:
d_value = (end_value - start_value) / segment_lenght;
for(...)
{
...
value += dvalue;
}
To do the a "reasonably smooth" (*) spline, you'll need to calculate
dvalue, ddvalue and dddalue in a way that solves this:
value[0] = start_value;
dvalue[0] = start_dvalue;
value[segment_length] = end_value;
dvalue[segment_length] = end_dvalue;
value and dvalue are given, obviously. We need ddvalue and dddvalue.
(I was looking into that before, but got interrupted, and lost track.
Shouldn't be too complicated, though. Just a basic 1D polynomial
spline. :-)
(*) We have to assume something reasonable as a "vague reference"
here, so plugin authors have some useful common guidelines.
Anything stricter than that will just make optimizations harder
and more risky, and may have authors of control generators
assume a bit too much about the shape of splines.
//David Olofson - Programmer, Composer, Open Source Advocate
.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
.- M A I A -------------------------------------------------.
| The Multimedia Application Integration Architecture |
`----------------------------> http://www.linuxdj.com/maia -'
--- http://olofson.net --- http://www.reologica.se ---
This archive was generated by hypermail 2b28 : Fri Dec 13 2002 - 22:24:48 EET