Re: [LAD] Portable user interfaces for LV2 plugins

On 03/01/2011 12:58 AM, David Robillard wrote:
> On Mon, 2011-02-28 at 21:51 +0100, Olivier Guilyardi wrote:

>> Actually, on current mobile platforms, when one wants a portable UI, there is an
>> alternative to Web UIs: OpenGL. This runs everywhere, and as smoothly as can be.
>> All you need is the plugin to expose draw() and mouse motion functions.

>
> Yep... and it's not remote, and involves writing a bunch of platform
> specific native code. The whole point is avoiding that.

Hmm, what platform specific native code? In my idea, the host would handle
setting up the GL viewport and such. The plugin would expose a draw() callback
in which it would only perform standard OpenGL calls.

I'm not an expert about GL portability, but AFAIK this might allow the UI
provided by a plugin to run unchanged on Linux, iOS, Android, Mac, Windows, etc..

> Do I think GL is the thing to use _if you want to, and can deploy,
> implement device specific native code for the UI_? Absolutely. Is that
> suitable for all (or even most) cases where browser UIs shine? Nope.
>
> As one example, I want to have a machine controlling the audio rig, have
> people arrive with their tablet (or whatever), go to a particular
> address and participate in the jam. This is be a pretty
> awesome/novel/unique possibility. Non-realtime audio is even a
> possibility if their device can do such things. Obviously, the only way
> of doing this is web UI. As a nice plus, when you do it that way, hey,
> you get a PC appropriate network transparent UI for free. From the
> perspective of someone who needs this anyway, some very tangible reasons
> would be needed to make rewriting the whole UI in GL as well not an epic
> waste of time. Note that most of realising this dream will be done by
> the host, only certain plugins would need special web UI fragments. The
> rest just need to provide sufficient information for the host to make
> sense of their parameters (as they need to regardless).

I'm confused now. You are talking about UIs automatically built by the host from
the plugin parameters, ports, metadata, etc.. To me, this is not a plugin UI,
this is the absence of plugin specific UI, and the kind of thing which you can
already do with LADSPA plugins, etc.. So, automatically generated web UIs from
plugin metadata is of course possible, but I wasn't talking about that.

I'm talking about those cases where the plugin needs to provide a specific UI,
as it's currently being done with LV2 using GTK, for example in Calf plugins,
etc.. But here you mention a mix of automatically generated UIs and "special web
UI fragments" provided by the plugin.

This sounds like abstract maths to me ;) Sorry, I can't follow you, it's not my
field.

Regarding remote controllers.. Well this is already in the wild, tablets
included, with OSC and MIDI. For example, there is TouchOSC (sorry, not free,
but a great project) which even has a UI editor.

I think that I understand you idea of doing this with a browser. No need to
setup OSC, ports, map URIs, etc.. But this sounds like a web-based controller
project. It's application level IMO, not sure it belongs to LV2, apart from the
UI fragments idea, but this all sounds like fundamental research to me.

That said, I don't think that I can help some more here. I'm more into simple
user needs and every day music/audio tasks.

--
  Olivier
> 
> If you want to do some sort of experimental fancy 3D plugin UIs rendered
> in the same 3D universe or whatever (right now, i.e. not using webGL),
> where it is necessary for a plugin to have special UI code (i.e. the
> host can't generate it) sure, this is not the way to go. Use
> GL/Clutter/whatever. Unless you actually need the performance advantage
> of native GL, though, browser is better.
> 
>> Using such functions glScissor(), it would even be possible to embed a plugin UI
>> in a host UI while making sure that it doesn't draw out of its bounds.
>>
>> A toolkit on top of OpenGL may often be useful especially for drawing texts and
>> widgets. For example, there is Clutter: http://www.clutter-project.org/
>>
>> But this is no constraint, every plugin would be free to use the toolkit of its
>> choice internally when rendering within its draw() function.
>>
>> There are subtleties though. For example on Android, there are so-called screen
>> densities, theoretically ranging from 120dpi to 320dpi according to the device.
>>
>> But that's easily solved with OpenGL, the host can scale the displayed plugins
>> if needed. The plugins do not need to know what the ratio between a point and a
>> pixel is.
>>
>> And if there existed some specialized toolkit for audio UI development, and even
>> a UI builder in the lines of JUCE's Jucer, then creating portable and
>> high-performance plugin UIs would become very easy.
>>
>> That said, I have also investigated other solutions for simple portable plugin
>> UIs, but I think that OpenGL is worth considering.
> 
> I fully support people doing openGL (or clutter, or whatever) UIs,
> nothing's stopping you. It won't solve the problem I am trying to solve,
> though. This is another example of why LV2 doesn't cram a toolkit down
> your throat. It is unclear right now whether the web UI stuff will even
> have anything to do with the UI extension.
> 
> Anyway, the more interesting/important/pressing issue is how the UI
> communicates with the plugin, because that part really should be the
> same in both cases, and a solution is needed for currently existing
> things that are kludging around the lack of it with e.g.
> instance-access. Solution here coming soon.
> 
>>> I think I am going to create a "LADSPA metadata" LV2 bundle with a
>>> (hand-curated) data file with extra info about various LADSPA plugins,
>>> particularly classes (plugin categories) for plugins without lrdf.
>>> LADSPA plugins not being categorized in the menu is the user-visible
>>> kink in integration right now...
>> Hmm.. Does the current bridge rely on LADSPA rdf files to be present? If so then
>> I would say that it doesn't really provide backward compatibility from LV2 to
>> LADSPA. AFAIK, ladspa.h is the specification, and I would hope that the bridge
>> is able to detect and load any plugin which follow this specification.
> 
> It does if you want categories etc. (obviously, since LADSPA binaries
> simply don't have that information in them). They are not required for
> plugins to be operational.
> 
> -dr
> 
> 
> 
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev@email-addr-hidden
http://lists.linuxaudio.org/listinfo/linux-audio-dev