Dave Robillard wrote:
> "extensible" like SYSEX. It isn't 1980 anymore...
Which doesn't mean we need to waste CPU cycles on comparing URIs in
event processing loop :)
I already gave you one example how it can be handled beyond what SysEx
offers (by using interfaces). And another (by binding command numbers to
URIs, preferably on plugin load). Read my previous mail more carefully.
By the way, how old is IP? It is old, it is crufty, still, it won with
competing protocols for some reasons :)
> Your extension is too complex, this is primarily why I don't like it.
> Why are there two uint8_t's in the header that are probably completely
> useless for essentially everything but MIDI? It's crufty.
Don't tell me you haven't heard about alignment. Please. Basically, the
"useless" fields serve as padding when they aren't used. You'd have to
have them there anyway, just as "reserved".
Optimization often introduces cruftiness. If you looked at IP header,
it's full of nasty, unelegant stuff. It also serves its purpose pretty well.
> A stamped chunk of "some sort of event" is simple to understand,
> guaranteed 100% extensible and generic without any weird sysexey
> 1980'sness,
Are you sure you are solving right problems here? Look at what purpose
is it meant to serve.
> and is very, very similar to how LV2 ports themselves work
> (and therefore obvious and easy to understand for anyone who knows LV2).
The difference is that ports are usually connected once per plugin load,
or once per playback on/off, or comparably often. Not several times per
256-sample period.
> Your efficiency claims make no sense, frankly. Put everything in a flat
> buffer and voila, it's exactly how MIDI events work right now. Your
> proposal has no inherent performance advantage over a more obvious and
> cleaner one.
I've already said why it is more efficient. Please give statements that
refer to details, not just vague "make no sense". For example, tell me
why reading more data from DRAM to cache has no impact on performance in
your opinion.
I think we make a serious mistake of not trying to sketch out some code
for reading a buffer, writing a buffer, handling binary data etc.
> Exactly. You are making assumptions about what people want to put in
> there. This is a Bad Thing(TM). If it can be avoided, it should be
> (and it can definitely be avoided).
I should be avoided, but not at cost of efficiency.
> I have said this a lot, and I will continue saying it more until the end
> of time because it's important: the fact that ports can
> contain /anything/ is the fundamental core idea behind LV2, and it's a
> good one. A good generic event extension must do this as well.
But, preferably, it would not _mandate_ wasting CPU cycles. It should
promote good programming.
> too much is on the table to be figured out at one time (how far would
> the Internet have come if every high level protocol was defined in the
> IP specification?)
That's a good argument... almost. But there are counterarguments:
- whenever you add more layers, you decrease efficiency
- complexity of TCP protocol favours making it a separate layer, we
don't have this level of complexity in our event types
> Your optimisation comments (the bulk of your email) are based on a
> misunderstanding. I don't mean the data pointer to point somewhere else
> in memory, it's a flat buffer (ie almost identical to how MIDI works
> right now).
You didn't make it possible for the MIDI data to follow the event
directly (well, you could say that if data pointer is equal to the first
byte after event header, then size+padding should be added to header
pointer, but that would decrease the elegance of your proposal).
Also, as I said, you always use 16 (or 24) octets+payload length for
every trivial event like MIDI. That's what I wanted to avoid.
> For large data, that flat buffer can contain a pointer or
> whatever (again, see LV2 port extensions that can e.g. contain structs
> with whatever you feel like cramming in there).
Oh, we have 1980 SysEx right here in your proposal :)
Basically, that's the same thing you criticize me for doing. Let's do
something better.
As I said, you didn't even provide any way to transfer the ownership of
the buffer to the plugin, so that it doesn't have to copy it.
> In short: if generic events is the goal, then lets make the events
> generic!
Generic events is one of the goals. Not the only one.
Another goal, as I said, is not wasting speed (by wasting CPU-DRAM
bandwidth, causing unnecessary cache thrashing etc), at least in the
"expected" case.
Another goal, is not wasting memory, but that's a low priority one -
unless we're wasting speed by wasting memory.
> I'm adamant about this for pragmatic reasons: There are some more
> difficult problems here if we really want to crank up the performance of
> this, especially random event access and constant time buffer traversal
> which needs an lookup table to be around somewhere.
Well, please explain when do you need a random event access - I didn't
factor it in during designing my solution, because, frankly, it's
rarely, if ever, used in practice. You can always traverse the buffer
first and make fixed-length index for binary search, if you really need
random access.
You're free to favour those unusual cases and degrade performance for
everyone else, but do you really want to?
> real problem at hand and ignore the (completely orthogonal) details of
> event types. The transport part alone is a problem that needs
> solving /first/.
Okay, that's a good start. What do you propose?
> spirit of LV2. "Note that commands 0x00-0x6F and 0x73-0x7F are unused"
> is more the spirit of MIDI circa 30 years ago.
Only if you ignore the remark about plugin declaring that its
understanding of command 0x73 maps to uri http://blah.blah/mpeg2-video.
Then the host will fill the command field with 0x73 whenever it needs to
send the mpeg2-video data type.
It's more work during plugin load, but then the plugin doesn't need to
strcmp any uris or anything stupid like that in the audio processing
loop - just compare command with 0x73.
There is no central authority here, everything can be done in "spirit of
LV2" with URI-based mappings.
I was aware of the central authority problem, and as you see, it got
solved. Right in the first paragraph of the second mail.
> better.. having to appeal to some (nonexistant) central authority for
> doling out a numeric range for features is completely out of the
> question (we learned that one with LADSPA IDs).
No doubt. Relying on people to handle that is asking for trouble,
especially if they have to work for free. Plus ego, differing visions
etc. all contribute to problems.
> who almost certainly will to). Preserve extensibility in LV2 wherever
> possible, it's a Good Thing :)
Preserve extensibility - yes. But do it in a reasonable way, taking care
of more factors than just extensibility.
Anyway... Let's try to LV2ize my proposal, just as some starting point:
struct LV2_EVENT_HEADER
{
uint32_t timestamp; // timestamp
uint8_t event_type; // event type number
uint8_t args[3]; // short-form arguments or padding
};
Event types are defined by the plugin like this:
@prefix ep: <http://foltman.com/lv2/ep> .
<http://someevent> ep:eventType [
ep:eventNumber 128 ;
ep:eventName "CH1:Note off" ;
ep:eventData ep:shortData ;
ep:eventContent
"http://www.midi.org/about-midi/specshome.shtml#ch1-noteoff" ;
].
(it's probably full of syntax errors, but at least you'll get the
meaning if you try)
Where:
ep:shortData means that command arguments are contained entirely within
args array
ep:mediumData means that the 8-octet block follows the event header
ep:longData means that the 8-octet block containing the
IDataBuffer-derived pointer follows (where IDataBuffer would be
something more or less like my LV2_BINARY_BUFFER proposal, with
serialization added, perhaps); this is to ensure that we can pass
structures with pointers, allow zero-copy approach etc.
What exactly _is_ in those data (the semantics, as opposed to just data
size and format) is dependent on eventContent. With longData,
eventContent describes a specific interface (the only requirement is
that it derives from IDataBuffer, so that it can be serialized, acquired
etc easily).
The host is responsible for sending right event numbers as required by
the plugin (so the plugin might wish the channel 1 note off command be
delivered as type 0, while http://foltman.com/lv2/ep#song-lyrics would
be delivered as type 1). Of course, every sane plugin on the planet will
just map usual MIDI command URIs to their usual numbers, but it won't be
mandatory. You can also decide to drop support for commands like
polyphonic aftertouch, and have some other types of events in the same
range of numbers.
This is kind of silly (extensibility gone crazy), but you kind of asked
for it :)
It has all the extensibility you wished for, and the (hypothetical)
plugin code looks pretty straightforward to me. It is also pretty tight
memory-wise.
You (as a plugin author) can have commands with 3-byte data, 11-byte
data, interface pointers - with whatever meaning you attach to them. The
host would have to have each event types for each plugin easy
accessible, but that's inevitable in any approach that doesn't use fixed
number assignments.
Now, what do you think? I think you might like it, because it's very
similar to your "generic event" structure, just with more compact memory
layout, and with zero-copy approach to large data. The MIDI-oriented
folks might like the fact that they can pass MIDI data without that
separate block of memory, while the rest can still do their advanced
stuff with relative ease (using mediumData and longData).
Again, that's just a starting point.
Regards,
Krzysztof
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev@email-addr-hidden
http://lists.linuxaudio.org/mailman/listinfo/linux-audio-dev
Received on Wed Nov 28 16:15:03 2007
This archive was generated by hypermail 2.1.8 : Wed Nov 28 2007 - 16:15:03 EET