Subject: Re: [linux-audio-dev] Introducing DMIDI
From: John Lazzaro (lazzaro_AT_CS.Berkeley.EDU)
Date: Thu Dec 20 2001 - 00:34:44 EET
[note to AVT folks: MWPP discussions on linux-audio-developers are
ongoing, including posts by Dominique Fober, whose paper:
http://www.grame.fr/~fober/RTESP-Wedel.pdf
describes MIDI network coding; this message is my reply to these
discussions, CC'd to LAD]
Hi everyone,
Responding to MWPP-related comments made in multiple email
postings here ...
> Dominique Fober writes:
>
> There are some interesting ideas in these works and I would like to
> add a contribution to the discussion:
Yes, please do -- more specifically, please consider joining the IETF
AVT working group mailing list, and becoming part of the consensus
process; we have many network experts there, but we're short on MIDI
expertise. See:
http://www.ietf.org/html.charters/avt-charter.html
For details on how to join the mailing list (its a low volume list).
In general, the main thrust of the comments on MWPP at IETF52 in Salt
Lake was that it was important to figure out the minimal amount of
MWPP semantics that needs to be normative, so that implementors can be
free to innovate under MWPP, as long as interoperability can be
maintained.
The next rewrite of the MWPP (probably in Feb 2002 timeframe) is about
figuring out what to remove and what to add to meet this goal;
figuring out what to remove and what to add to permit the
RTESP-Wedel.pdf ideas from working in the MWPP framework would be a
good contribution to this effort.
>
> Therefore, a mecanism to compensate for the latency variation seems to
> me to be necessary.
>
I don't think its always necessary -- MWPP should provide the freedom
to implement latency variation compensation, but I don't think it
should mandate it. In our experience playing over the CalREN 2
Berkeley-Stanford and Berkeley-Caltech links, jitter compensation
wasn't necessary; "play when received" worked well, as long as the
"outliers" of very late packets were handled separately, using
semantic rules (which is what the "ontime" and "late" flags codify).
> [... a later message ...]
>
> Sure, it may be used for this but it isn't: MIDI events are played
> at reception time. In MWPP for example, the timestamp is only used
> to determine wether a packet is too late or not.
Note that "play when received" isn't a normative part of MWPP.
Quoting from page 7 of the MWPP I-D, first paragraph:
The most common occurrence is for packets to be normal/ontime. In this
case, the receiver schedules all commands in the MIDI command section
of the MWPP payload for execution on the local SA decoder. The
details of scheduling are implementation-specific: simple decoders
(such as [3]) may execute MIDI commands as soon as they arrive, but
other approaches are possible.
So, MWPP itself is agnostic about when you actually play notes which
are deemed "ontime" -- an implementor is permitted to use a playout
scheme more sophisticated than "play when received", that handles
latency variation. And since the algorithm for deciding when a note is
late or ontime is also implementation specific, I don't see any
show-stoppers for implementing the latency variation ideas in
RTESP-Wedel.pdf in an MWPP-compliant implementation (although I
haven't read the paper yet so I can't say for sure).
In practice, though, any scheme that compensates for latency variation
is going to do so by delaying the playing out some notes, relative to
a "play when received" mode -- jitter reduction has a cost in added
latency. The question is whether the cost is worth the benefit -- and
it would be best if MWPP left that decision in the implementors hands,
rather than making it for them.
> Another point is the efficient use of the transmitted packets: sending
> one packet for each event is probably not the best solution.
Note that MWPP has a "MIDI Command Section" with a 6-bit length field,
and that except for the first MIDI command, running status coding is
permitted:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|R| LEN | MIDI Command Payload ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Recovery Journal ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
So, schemes that code more than one MIDI command per packet are
supportable under MWPP; sfront puts one command per packet in its MWPP
implementation, but its not a normative part of the spec. What is lost
in this simple approach is information about the temporal delay
between commands -- as MWPP is written, the sender is claiming that
all commands in the MIDI Command Payload happened at the moment of the
RTP timestamp.
One addition that could be made to MWPP, is a simple, efficient way to
code this delta-t information, that doesn't impact the packetization
for the "single-command-per-packet" case. However, the AVT culture is
not to "add a feature because it might possibly be used" -- if the
bandwidth reduction (relative to sending one-command-per-packet, each
with its precise RTP timestamp) achievable by adding this complexity
isn't seen as serious issue by the working group, the consensus might
well be to leave the feature out.
> In this case and due to the underlying protocols overhead, the useful
> information part of a packet may become less than 10% of the packet
> size.
We talked about efficiency of MWPP at the Working Group meeting in
Salt Lake, and there was a sense that looking at efficiency without
regards to the absolute bandwidth of the packetization is a mistake.
For fast piano work, we're measured the median MWPP RTP payload
bandwidth (i.e. everything in the bitfield diagram above, but not the
RTP, UDP and other headers) to be about 4700 bits per second. Since
this compares favorably with the lowest-bandwidth speech codecs that
RTP currently supports, there wasn't much support in the room for
re-engineering the packetization to have better efficiency if there
was a price to be paid in other areas.
> Considering a 44 bytes overhead (IP + UDP) + the 4 DMIDI header bytes
> intended to address a specific node and device, sending a full MIDI
> data flow (about 1000 3-bytes events per second) requires nearly 400
> kbs when the MIDI rate is 31.25 kbs.
We've clocked sustained, two-handed, fast piano improvisation at about
20 events per second -- significantly less than the 1000-event "maxing
out the MIDI cable" number above. For the MWPP target application of
network musical performance of musical groups, the "number of events
produced by a human" metric seems more appropriate than the "max out
the cable" metric. BTW, note that using controllers seems to decrease,
not increase the number -- its hard to generate data faster than fast
fingers sweeping across a piano keyboard.
This fast, two-handed playing is the sort of input that yields the
4700 bits per second figure for MWPP's payload (the RTP way of
comparing bandwidth is to use packetization payloads, since there are
good header-compression techniques for RTP, UDP, and IP headers that
can be brought into play if saving every bit is important).
If you wanted to send these "max out the MIDI cable" flows through
MWPP, you'd have two alternatives using the existing scheme:
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|R| LEN | MIDI Command Payload ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Recovery Journal ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
-- Place one MIDI command per RTP packet, to minimize time skew
at the expense of bandwidth.
-- Pack multiple MIDI commands into each MIDI Command Header, and
pay the price in systematic jitter.
The alternative is a more sophisticated coding of the MIDI Command
Payload, that codes time deltas between commands; this information
wouldn't be of use for simple "play when received" implementations,
but would be of use for implementations which did latency variation
removal. This would be an example of the sort of thing that could be
added to a future MWPP I-D revision ...
-------------------------------------------------------------------------
John Lazzaro -- Research Specialist -- CS Division -- EECS -- UC Berkeley
lazzaro [at] cs [dot] berkeley [dot] edu www.cs.berkeley.edu/~lazzaro
-------------------------------------------------------------------------
This archive was generated by hypermail 2b28 : Thu Dec 20 2001 - 00:29:11 EET