[LAD] Tracing malloc() and mutex usage in realt-time software

Heya!

Not sure if anyone of you already read this announcement about my
mutex profiler "mutrace" I wrote a couple of days back:

http://0pointer.de/blog/projects/mutrace.html

Since I wrote that blog story I have added a couple of features to
mutrace that are particularly relevant when developing real-time
applications. I'd like to draw your attention to these, since this
might be of some interest for the audio community.

"mutrace -r" can be used to track down any mutex usage in real-time
threads. As everyone probably knows it's a good idea to abstain from
mutex usage entirely in RT threads. However, actually doing that is
not always realistic, and sometimes the more complex your project gets
the harder it becomes actually making sure that no hidden mutex usage
creeps into your program. Too verify lock-freeness or track down the
culprits you may use "mutrace -r". It will tell you not only if am RT
thread used a mutex, it will also tell you how often it was contended
and how often it changed ownership (i.e. how big the risk of
contention is even if it didn't contend in the specific run.) In
addition it will show you the mutex protocol, so if you are using a
mutex from an RT thread and really cannot do without it, you can at
least make sure it is a PRIO_INHERIT mutex,

And there's also now a second tool called "matrace" in the tarball
that tracks down malloc() usage from RT threads. It's a very simple
tool, which just catches the memory allocations/frees, checks the
scheduling and complains if necessary.

With these two tools two of the biggest donts in RT programming may be
tracked down: use of mutexes and use of malloc.

How does the output look like? As an example I simply ran jackd through
mutrace -r and matrace, for 10s without doing anything else. The
result for mutrace you may find here:

http://0pointer.de/public/mutrace-jackd.txt

(The interesting table is at the end, so scroll down)

Not sure what to make of this, as I don't know the Jack internals too
well. However mutrace reveals that at least one mutex was used from
an RT thread and even contended. Moreover that mutex was not
PRIO_INHERIT.

And here's matrace:

http://0pointer.de/public/matrace-jackd.txt

Seems free() is called a couple of times from an RT thread.

These might, or might not be problems. The tools can just inform you
what is done. If there's something to fix is up to the eye of the
beholder.

Get the tool here:

http://git.0pointer.de/?p=mutrace.git

Have fun,

Lennart

-- 
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/           GnuPG 0x1A015CC4
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev@email-addr-hidden
http://lists.linuxaudio.org/mailman/listinfo/linux-audio-dev