On Wed, 2011-03-02 at 14:20 +0100, Olivier Guilyardi wrote:
> On 03/02/2011 12:22 AM, David Robillard wrote:
>
> > On Tue, 2011-03-01 at 19:36 +0100, Olivier Guilyardi wrote:
>
> >> Hmm, what platform specific native code? In my idea, the host would handle
> >> setting up the GL viewport and such.
> >
> > ... "the host would <platform specific native code>" ;)
>
> No, not really.
>
> Apart from mobile web apps, a way to build cross-platform apps is using a
> lightweight toolkit, with OpenGL for all graphics, and a couple of other entry
> points such as an audio I/O callback, etc..
>
> With this method, notably used by game devs, there's one code base, with thin
> platform drivers.
Thin platform drivers, i.e... platform specific native code.
Can we PLEASE stop beating this dead horse? My point is that NO SOFTWARE
NEEDS TO BE INSTALLED AT ALL WHATSOEVER AT ALL PERIOD AT ALL. Get it?
Good. Let's move on.
> What you're talking about makes sense if the host itself runs in the browser...
> But switching from a native UI to a browser repeatedly is going to look rather
> odd IMO. And if your idea is mainly intended for browser-based hosts then it
> will not integrate properly with a large range of existing software.
The idea is to put the entire performance/control/et UI in the browser,
so it can be remote. You've missed the remote idea. Remotes good, even
on a desktop. People go out and spend money on special purpose devices
for this all the time. Tablets can give you a completely generic
reconfigurable... that.
The "host", as in the plugin host that does audio things, is obviously
not in the browser, that's on the machine doing audio things, C, Jack,
etc.
> That said, for instance on Android, you can embed a small WebView into the UI of
> a host. So it could make sense.
Yep.
> But (http) network operations are going to bring
> some more overhead. All of this really isn't going to be smooth.
It's not a network operation if it's local. Local socket communication
is very fast... and something that's arriving in the browser.
http://en.wikipedia.org/wiki/WebSockets
As far as I'm concerned a non-crap app design should be capable of doing
engine <=> UI communication over a <whatever, e.g. socket> anyway, even
if you only intend to use it on a local machine. You WANT a huge
meaningful wall between those two things. For one thing, if your UI
stuff crashes, your audio doesn't, and your set doesn't trainwreck.
The other win of UI<=>audio separation is that not all of our DSP code
runs on a computer with a monitor at all. Little embedded boxes that do
DSP stuff don't have big screens... but they usually have USB ports (or
network, or MIDI, or something).
That UI <=> Plugin separation is necessary is, IMO, a given, and not up
for debate. You could throw it all out the window and gain virtually
nothing in return, but... well, doesn't sound wise, does it?
> >> The plugin would expose a draw() callback
> >> in which it would only perform standard OpenGL calls.
> >>
> >> I'm not an expert about GL portability, but AFAIK this might allow the UI
> >> provided by a plugin to run unchanged on Linux, iOS, Android, Mac, Windows, etc..
> >
> > Probably, since the host deals with all the non-portable things. What
> > you describe here sounds like how a GL UI type should indeed work. It
> > would be better if there was some equivalent of a "widget" the UI could
> > return, but I guess that is not the case for GL.
>
> The UI would draw into a rectangular area which could:
> - either be embedded in a larger GL scene, if the host UI is GL based,
> - or be displayed by QGLWidget, rendered into a GdkGLWindow, etc..
>
> Also, what's rather funny is that there would be some kind of symmetry between
> the run() and draw() functions exposed by a plugin. One for audio, the other for
> graphics/video... And both called in a cyclic manner by the host.
Plugin and UI are not the same thing, nor should they be.
> > So far, nobody has showed up with a desire to write a GL plugin UI.
> > Extensions don't get invented until there's an actual use for them. If
> > someone does, drop by lv2-devel, and we'll get this done (or don't, and
> > just do it).
>
> Yes, I've developed a long time for the desktop, and I never used OpenGL before
> I got myself into mobile development. Now that I'm familiar with it, I quite
> enjoy it, and am amazed by the perspectives in terms of portability.
OK, well, do you have any active plans to do OpenGL plugin UIs any time
soon?
> > The next SLV2 will abstract away UI types, so if/when this gets
> > invented, it will magically just work in existing hosts, since embedding
> > GL in most toolkits is possible.
>
> Sounds good. This is where LV2 extensibility shows some of its strengths I guess.
In one sense. The problem it's solving also shows some of its
weaknesses.
> >> I'm confused now. You are talking about UIs automatically built by the host from
> >> the plugin parameters, ports, metadata, etc..
> >
> > Yes, partially...
> >
> >> To me, this is not a plugin UI,
> >
> > ... except a plugin could provide such a (plugin specific) UI as well.
> > My "dirg" plugin provides a web-based grid UI in the browser, for
> > example. Currently it just does this by running its own web server, but
> > this needs to be pushed up to the host for obvious reasons.
>
> Hmm, own web server, I don't know really.. This sounds a bit bloated to me.
... Yeah, that would be the obvious reason. ;)
> >> this is the absence of plugin specific UI, and the kind of thing which you can
> >> already do with LADSPA plugins, etc.. So, automatically generated web UIs from
> >> plugin metadata is of course possible, but I wasn't talking about that.
> >>
> >> I'm talking about those cases where the plugin needs to provide a specific UI,
> >> as it's currently being done with LV2 using GTK, for example in Calf plugins,
> >> etc.. But here you mention a mix of automatically generated UIs and "special web
> >> UI fragments" provided by the plugin.
> >
> > "special web UI fragments", or, in other words, a plugin specific UI.
> > Yep.
>
> And how does one draws a frequency curve, even one which doesn't update in
> realtime? With SVG? Do you see how heavy this is all going to be?
Yep. I see how heavy it is, and I also see that it is the only
technology stack you have, if you want to do this kind of thing. I also
see it's getting a whole lot less heavy extremely rapidly.
Anyway, go look at some modern canvas experiments. Drawing a curve
quickly is well within what works fine right now. Drawing and animating
a whole ton of them smoothly is current reality.
Why you are trying to pick apart web UIs in the same email as you're
arguing where one size does not fit all I don't know... I want a remote
control that works on any device out of the box. It's about as blatantly
obvious as anything can be that web is literally the only way to go for
that, because the browser is actually on those devices. QED.
Do I think /all/ desktop PC hosts and plugin should use it? No. It's for
"remote control" things (even if not remote). Is GL good? Yes. I sure
wish I used it for the Ingen canvas. Has anyone actually showed up yet
who wanted to write a GL UI? No. Unless you're one, talking about it is
a waste of time.
> >> I think that I understand you idea of doing this with a browser. No need to
> >> setup OSC, ports, map URIs, etc.. But this sounds like a web-based controller
> >> project. It's application level IMO, not sure it belongs to LV2, apart from the
> >> UI fragments idea, but this all sounds like fundamental research to me.
> >
> > My LV2 host /is/ an LV2 plugin ;) Think about the implications of that
> > for a minute and a big lightbulb should go off (Hint: Want to make
> > plugins with universal UIs? Don't want to write code?)
>
> Not sure what you mean here, but I'm a big No Silver Bullet fan, and generally I
> don't believe in no-need-to-code-anymore claims.
You just mentioned controller apps where users build their own UIs
(touchOSC etc), sooo... I guess you don't believe in reality? ;)
> Okay, well, this Web UI idea would certainly be good in certain cases. And the
> truth is that I'm not really convinced about the OpenGL idea, because it may
> result in inconsistent appearance, with widgets style which do not match each other.
This is audio. Give people the ability to pixmap at all and they're
going to make some big retarded quasi hardware photo thing, each
entirely different, and usually horrific screen real-estate wasters.
Consistency between UIs ain't going to happen. GL would probably be even
more consistent than the toolkits, ironically, because it tends to push
you towards vectorey things.
> But this later problem could be solved with a simple audio-oriented UI toolkit,
> which would render using OpenGL. A such toolkit could actually be very
> lightweight. For instance you do not necessarily need to embed entire charsets.
> A couple of characters (digits, "dB", etc..) could be sufficient in many cases.
Aaaand now we're entered the realm of many years of work that nobody is
going to do. Unless JUCE counts.
Speaking of doing tons of work, GL gives you an open-ended way to draw
things... and that's it. One you actually need to make those things
interactive, you need something else. Something else that isn't just GL,
and probably isn't portable, and is a ton of more work regardless. The
DOM inherently makes it trivial to deal with events, and that's all
built-in. A mess of little toolkits, all alike, is not a fun situation,
and plopping it on a GL backend doesn't make it go away.
-dr
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev@email-addr-hidden
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Received on Wed Mar 2 20:15:04 2011
This archive was generated by hypermail 2.1.8 : Wed Mar 02 2011 - 20:15:04 EET