Subject: [linux-audio-dev] App intercomunication issues, some views.
From: Juan Linietsky (coding_AT_reduz.com.ar)
Date: Mon Jul 22 2002 - 23:18:46 EEST
We've been discussing this issue a lot recently on the #lad irc
channel (irc.openprojects.net)
and i thought it's worth a posting in the LAD list...
There seems to be two strong postures on how to develop audio
software, and how to proovide linux
with a "modular" audio desktop. Such postures aim to solve the common
linux-audio problem
on how to do things "more modular" and improve app reusability. We can
look at windows/mac apps
such as Reason/Logic/Sound Forge/etc and wonder "Why dont we have this
for linux?". And I
think the answer is "not many in the community have enough time to
focus on such a huge project".
But at the same time, the community is full of developers which, may
not have an eternity
of time, but have enough to write _something_ and the will to do so.
This could
be pictured as a classical "bandwidth vs. speed" issue :). Because of
this, I think
it's extremely important to focus development on modularity and
inter-communication
as much as possible. Discussing this with irc folks, several people
grouped into
two postures.
The first posture is to write big apps, with extra functionality added
as host processing.
This is the de-facto standard in windows & mac. Using this model, all
kind of "extra" processing
features (such as audio DSP, midi instruments/ports, and even some
midi drivers) are realtime
loaded into the program through a dynamic library model. This model
has clear
advantages in the sense that it can produce solid monstruous apps rich
in functionality,
and in general userfriendly for the new user who comes from other
operating systems.
But for us to develop these apps, we find ourselves with huge
disadvantages, the first one
is that, no matter how many host dsp is loaded, the app will end up
being huge, and whoever
develops it will need an enormous amount of time. The other issue with
this is managing a common
interface for the host plugins. The good thing also about this design
is that it is compatible
with the second posture mentioned (modular apps), since a host app can
be easily turned
into a network node.
The second approach is to develop smaller and distributed apps, In
this model, apps offer specific
functionality over a solid/well defined api which proovides good way
of intercommunication with
other apps. This is probably the closest approach to the current
status of linux audio right now,
with apis such as JACK and ALSASEQ. The good thing about developing in
this way is that it
takes full advantage of the "bandwith" of developers in the community.
The problem
with this approach is that since we're using many apps instead of one,
management of them
can become annoying, specially at the time of saving our work/project
and layout.
I think this can be solved by developing a metadata protocol between
apps, so the can intercommunicate
status and other things, and having a "master" app that manages
projects and things like that, by just
retrieving/storing status in the other apps.
Examples of common problems, how can they be solved by both
approaches:
1- We are writing some music, we want to use a sequencer to do some
midi, which controls
a couple of softsynths and beat machines, then we want to add effects
to the output, maybe equalize it,
and from time to time, when we see that cpu usage is eating the
machine, dump some audio tracks
to a multitrack recorder/player. When we are done we save the project.
Problem we have right now:
Apps exist to do this, Muse/jazz++/etc work as a sequencer.. but we
start to get into problems. Softsynths and beat machines can be
triggered using midi and the aseq api, but we cant capture the output.
Also each softsynth
proovides its own set of effects, or its own host app interface
(ladspa), so when you use them all together the CPU usage grows to
insanity. Finally, a track can be recorded back by capturing the
a specific track playing as solo using JACK (or the output->input
feedback of the soudncard, if supported), but no sync is proovided so
it has to be done by hand. At save time, we have to save
what we did in each app and put it in a certain directory :) or even
write a shellscript!
Using the first approach:
In this approach, the softsynths/beat machines would be host apps in
the sequencer, and probably
would have to be written as libs... the sequencer takes care of
configuring them. Then also the sequencer takes care of the audio
routing by offering a graphical interface where you trace audio
routes,
place ladspa plugins, etc. Finally, the sequencer also proovides a
multuitrack recorder. At save time,
you just write a sequencer project file. As seen, this requieres more
work on the side of the sequencer programmer, but offers an app rich
in functionality. This is how Windows/Mac software works. For this,
a set of standard libs/apis will need to be defined, so each host app
can be written accordingly.
Another advantage of this approach is that existing wrappers can be
easily written to be able to use VST/DXi plugins. This approach is
good, and probably the most usefriendly, but forces you to build
big monolithic apps that take care of just everything, and we'll also
need to develop a set of apis
also for what is missing. Another problem is that we'll end up with
high difficulties in "importing/exporting" projects between apps, and
in finding a common gui for working on this.
Using the second approach:
We start out a sequencer, the sequencer "connects" to:
Alsa SEQ api: here we'll connect to the softsynths, the beat machines,
and the multitrack recorder.
The softsynths/beat machines will connect to the JACK api, prooviding
output ports for
the main mixing and for the effect send buffers, they wont do global
effect processing (ie chorus/reverb).
Using an interface to JACK (is there any aviable?) the user is able to
connect the send buffers
of the softsynths to a program which does in->out processing through
LADSPA plugins, in this case,
the chorus/reverb/sends of the softsynths will connect here. At the
end, we connect those and the global
mix buffers of the softsynths to a multitrack recorder (and why not,
to the output). At play time,
the multitrack recorder uses midi sync to record the final result. All
the programs used will be connected
to a "metadata server". This will be able to spawn/kill clients
(programs), and retrieve/store configurations (and maybe other
stuff?). So this will be a "Project Manager" App, and with it you will
be able to save/load projects and go back to them in no time. A good
thing about this is that it resembles
more how studios work, with all their equipment interconnected, so in
some way this should make easier
to integrate external stuff to our "projects".
So, what do you think about this issue? I'd really like to hear views
on this since i guess it's probably
the biggest problem the audio community is facing.
Regards
Juan Linietsky
This archive was generated by hypermail 2b28 : Mon Jul 22 2002 - 23:35:48 EEST