Subject: RE: [linux-audio-dev] present state of affairs with streaming fra meworks
From: Steve Baker (sbaker_AT_chello.com)
Date: Sun Jul 01 2001 - 22:59:58 EEST
Before beginning I should say that I have been working on getting LADSPA
working in GStreamer and generally working on audio support.
> >I'd like to pose a question to those in the know: What is
> the difference
> >between LAAGA (or the goals of LAAGA) and GStreamer?
>
> GStreamer is, according to its website, a developer's tool for
> building self-contained applications that feature processing graphs
> and streaming media.
To build on this I'll just list some features which make GStreamer unique in
the context of this discussion:
- GStreamer supports audio and video (and whatever else people want to write
support for)
- it supports many audio and video formats - audio formats including
interleaved int and non-interleaved float
- media flows through an in-process graph - in GStreamer terminology data
"buffers" flow between "elements" via "pad" connections
- one of GStreamer's target environments is embedded systems - the core is
small and fast
- GStreamer is written in C and now uses the GObject type system (which
means no more X dependency)
- connecting pads involves capabilities negotiation so that elements can
agree on the format of data to pass (yes, the implementation is complex. no,
plugins developers will not usually have to deal with that complexity)
- the preferred method for achieving concurrency in the graph is by using
cothreads. These are low-overhead user-space threads which will should
behave very well in a low latency environment. Unlike threads, cothread
context is explicitly controlled by the app, so you don't have to worry
about locking or any of the other non-deterministic ugliness you get with
pthreads.
> LAAGA is an API and a server to facilitate connecting multiple
> applications so that they can share data and avoid the typical
> interactions with audio interface h/w. An application would not be
> expected to use LAAGA internally to handle a processing graph.
No arguments there. I might as well say at this point that I think LAAGA and
GStreamer are in different and complimentary problem domains. If LAAGA
takes off I would like to see everyone writing their LAAGA apps with
GStreamer! If they don't it doesn't matter, but I would like to see LAAGA
support in GStreamer when LAAGA matures a bit.
> As far as I understand GStreamer, it doesn't offer any support for
> real-time out-of-process "graph nodes", which is a basic requirement
> for LAAGA.
Yep, although at some point we would like to have some kind IPC (maybe
CORBA/Bonobo) for GStreamer apps so we can easily construct distributed
graphs for fun things like render farms.
> Its does have a more sophisticated streaming model
> including support for parallelization and so forth. In this sense,
> GStreamer must also be compared to GLAME, which is significantly more
> mature than GStreamer at this point
In terms of end-user graphical apps then yes, GLAME is more mature. However
GStreamer has been in active development for almost 2 years now. The graph
system is very advanced and there has been a mind-boggling number of exotic
media formats and shared libs integrated.
> though not really a developer's
> tool as much as an application.
Quite - although soon enough we'll be encouraging the great unwashed to
write apps with gstreamer (many of course haven't needed encouragement)
> I would have to leave the GStreamer folks to comment on whether their
> graph model shares the properties that Richard and I discussed here at
> some length recently. GLAME's model will fail to produce any output
> when some upstream connections are broken, which at least a couple of
> LAD folk felt was user-unfriendly at the very least.
The GStreamer scheduler should keep elements in line and control when each
element should produce or consume a buffer. Also graph manipulation overhead
is very low and could even be used for things like muting elements.
> You could easily write an application that used GStreamer for its own
> internal graph processing, but used LAAGA to connect to other
> applications and some audio h/w.
Yes. While I said that GStreamer supports many formats, when used for LAAGA
apps I would expect that app writers should have preference for:
- support non interleaved float data bound by -1.0 -> 1.0
- prefer elements which don't re-malloc their buffers
- use cothreads for concurrency
I think the int audio formats are important for consumer audio apps and
there will need to be some int effects plugins written for these, but I can
see the benefit of sticking to float for pro audio.
I suspect the best way to support LAAGA's callback api is to write a
gstLAAGAbin which contains the gstreamer graph inside it and exposes the
bin's external pads as LAAGA ports. The normal way of supporting this sort
of thing in GStreamer would be to write a LAAGAsrc and LAAGAsink but I'm not
sure this would fit with the callback model.
I hope other GStreamer developers pitch in to this discussion (guys?).
Their opinions may differ and they may be able to provide more insight as
well.
cheers
This archive was generated by hypermail 2b28 : Sun Jul 01 2001 - 23:04:57 EEST