Subject: [linux-audio-dev] disk-based realtime sampler prototype released ! please try it.
From: Benno Senoner (sbenno_AT_gardena.net)
Date: Thu Jul 13 2000 - 01:41:29 EEST
Hi,
tadaaaaaaaa.
After 3 days of hard work of hacking an thinking , I was able to finish my
linux disk-based sampler prototype !
(how should we call it ? linuxsampler , linuxhdsampler ?
any ideas ?
(please read the entire mail before downloading)
http://www.gardena.net/benno/linux/linuxsampler-0.0.1.tar.gz
(please do not post this on sites like freshmeat etc, since this
is really a release for developers only , although it's trivial to compile
and testdrive the app even for a newbie)
I recycled the streaming and lock-free fifo technology from hdrbench,
so I saved some work.
It is a still only proof of concept, but is fully playable via MIDI keyboard,
and with the default settings the MIDI Note-on to audio latency should be
about 3msec.
For testing purpose I wrote a small tool which generates a 120MB instrument
which is basically a square/sine/sawtooth sample which is stored on the
harddisk note by note. (each note is about 20secs)
The first part is held in RAM , and the nice thing is that the 120MB instruments
gets loaded within 1-2secs. No more waits loading long samples.
I made some stress test like pressing on the MIDI keyboard with the entire
forearm as many keys as possible, and holding them down for 10secs.
About 20 notes get triggered on my MIDI keyboard, the disk begins to stream
nicely while the audio thread plays back the buffered data without any problem.
Not to mention the beautiful work the linux file buffer cache does:
the first time you trigger certain notes, there are disk accesses, but as soon
as you trigger the same notes again , the read() calls get the data out of the
file buffer cache thus causing only very light disk IO.
The utility which generates the test instruments, uses a really simple file
format: a small header, then a structure which points to the beginning and
endings (offsets) of each sample within the file, and at last the samples
themselves.
All the disk IO is done using a single filedescriptor using lseek()s to get
the relevant parts during streaming. It works great, and I think a mulithreaded
/multifile model does not offer any advantage, since we have to jump around
the disk to retrieve the samples anyway.
So please test the application, stress it as much as you can (by sending tons
of midi events to it using a 2nd PC and a sequencer etc etc),
read the code, (it is commented and very short and easy to understand)
let me know about flaws, bugs , what you like, what you dislike etc.
if you have problems with the default 3x128 fragmentsize change it using the
commandline options. (I had some problems with fragsize=128 on a SBLive (some
time ago, but perhaps this was due to old drivers, on the SB AWE64 3x128
works great.
BTW: can someone with golden-ears , eg those which are good in feeling
the goodness of note-on latencies tell me
if they can feel the difference between 3x128 and 3x256 buffersize.
Simply test with the default and then with the --fragmentsize 256 option,
and then tell us if they feel that the 3x128 response is more tight than 3x256.
I tried, to me 3x128 seems better , but I can't tell exactly.
I attached the README which contains useful stuff
and describes how the algorithm
works (lock free-FIFOs , audiothread, diskthread , midithread).
please read it (it is not that long) so you can get a better picture where
we are.
We should all join our forces and knowledge and try to transform it in a full
fledged professionally usable sampler.
How many are interested in contributing to this project ?
We need knowledgable people in the field of DSP processing,
(who wants a sampler without FXes or filters?) , sample library
formats/layouts ( eg Richard, Garth)
, plus people who heavily use hardware/soft samplers
in the realworld and who know their strengths and drawbacks, (they do not
need to be high skilled programmers) having clear ideas about what features
are needed and what is superfluous etc. (eg Tom)
Waiting for TONS of feedback and comments.
(PS: please leave out the boring patent discussions let's invest our time
in coding and hacking which is more productive)
PS2: hopefully (in not to distant future) Mac users (under Linux :-) ) will
finally enjoying to have have a disk-based sampler available,
contrary to our "competitor" which does not plan to release a Mac version
because it would have to recode it from scratch.
his happens when developing on a kludge-OS like windoze.
( a G4 with Altivec is too powerful to throw it away and use it as a mere
MIDI sequencer conneted to a x86 PC which acts as soft-sampler )
Benno.
README
---------------
disksamp 0.0.1 disk-based sampler for Linux
(c) by Benno Senoner (sbenno_AT_gardena.net)
This software is distributed onder the GNU GPL License (see LICENSE file),
and may not be used in commercial applications without asking the
author for permission.
Warning: for now, this is code is only a proof of concept demonstration.
It will later evolve to a full fledged disk-based sampler.
It is written in C++ since encapsulating the diskstreams and audiovoices
into objects helps to keep the data separated while losing nothing in
terms of efficiency.
The development will take place among the linux-audio-dev group.
If you are a developer and interested to improve this software
you should join the linux-audio-dev list and help us out !
(see http://www.linuxdj.com/audio/lad )
disksamp is a program which, when run on a low-latency Linux kernel,
can achieve MIDI NoteOn to output latencies of about 3-4ms at par
with a hardware sampler, with the difference that it can stream
the samples from disk in realtime.
That means that the size of the sample is only limited by the
size of your harddisk / max filesize imposed by the filesystem.
Therfore with this approach you can have samples of multigigabyte
in size and still fully playable from your MIDI keyboard.
The loading of a 120MByte sample takes about
1-2secs which is very fast since you don't have to load all the data into mem.
What are the current features/limitations of this demonstration code ?
limitations:
- Only simple linear velocity to volume mapping
- No envelopes yet, (ADSR etc), that means when you press a key
the sample is played as is, and when you release it , the note is
stopped immediately.
- No resampling supported yet, that means in order to being able to play
an instrument via midi keyboard, you have to supply sampled data for each
semitone. (resampling keyzones , velocity zones etc will be added later)
- No looping yet : the sample is simply played from the begin to the end
(or until you release the key). Looping will be introduced in order
to save diskspace and to allow complex effects.
- the fileformat is very simple (therefore limited in its features) since
it was introduced for the sole purpose to create simple instruments which
can be loaded by the democode.
- the voices are simply added up and then clipped, therefore too loud
signals may introduce distorsions. (to solve this we need to add
dynamic range handling code , a dynamic compressor perhaps ? )
- the code makes some assumptions about the sample size like
requiring the preload size and the total sample size to be a multiple
of the BUFFER_GRANULE value (in order to skip addidional checks and
helping to speed up and simplify the code)
features:
basically you can load a prebuilt instrument into the sampler and play it
from a MIDI keyboard.
the initial part of the sample is kept in RAM in order to overcome to
the latency of the harddisk which is not able to ensure 3ms latencies.
The nice thing is that since use the regular read() system calls when
streaming the samples, we can fully exploit Linux' file buffer cache.
That means when you press a key the first time, the sample will be streamed
from disk as soon as you hold the key down long enough to exceed the
RAM buffer part. But when you press the key a second time, the data
will be cached in the file cache, thus there will be almost no disk accesses.
This greatly improves the performance, since the most frequently used samples
will be always in RAM helping to avoid that your disk get stressed too much.
It seem strange, but hacking on your MIDI keyboard like crazy isn't that
stressful for the application as you mean: since you are frequently releasing
the keys, you will almost always play cached data.
The most stressful activity is to play many notes simultaneously, and
hold the keys down for 10secs or so.
In this case the limitation is the disk bandwidth/seek performance, just
like when doing harddisk recording.
I stressed the application by pressing on MIDI keyboard with the whole
forearm. About 20 notes got triggered and I kept the forearm down for
about 10secs. The disk was quite busy but no dropout occurred.
(depending on read chunck sizes an IBM ide disk can achieve up to 60
simultaneous tracks).
But keep in mind that this is worst case: very seldom this situation happens,
since music is a dynamic thing and not a static playback of 60 notes
thus making the achievable number of MIDI voices much higher.
On a fast machine , 100+ voices will be easily possible.
Theory of operation:
the application uses 3 threads with different SCHED_FIFO priorities:
audio thread:
highest priorty, is responsible to receive commands
from the MIDI thread and send commands to the disk thread and to
render the audio voices which consists to fetch the
samples from RAM / stream buffers , sum them up
and send the result to the audio device.
midi thread: lower priority than the audio thread
sits in a loop and waits for MIDI messages on the MIDI in port.
Formats the messages and send them to the audio thread in order to trigger
notes.
disk thread: lowest priority
is responsible to manage the disk streams and receice commands from the audio
thread.
The disk thread has to ensure to keep all the diskstream ringbuffers
as full as possible and to react with the highest possible speed to
commands from the audio thread.
Notice that all messages queues are implemented by using lock-free FIFOs
in order to avoid blocking (which could hurt the audio thread) and
to achieve maximum possible speed.
What happens when you press a key on the MIDI keyboard ?
the MIDI thread wakes up, reads the MIDI message and reads note velocity and
pitch. At this point it sends a (similar) note-on message to the audio thread.
The audio thread checks for new messages after every new audio fragment.
That means every 1-2ms depending from the fragment size.
Now the message is parsed and the pitch is used to figure out which
sample to play. The voice gets activated and simultanously a message is
send to the disk thread which tells him to activate the corresponding
disk stream and begin to refill buffers.
The note release case is similar: midi thread wakes up gets note-off message,
sends message to the audio thread which then deactivates the note
and informs the diskthread that it can shut down the associated stream
since it is not longer needed.
The refilling algorithm of the diskthread is based on the technology
used in my hdrbench harddisk recording benchmark
(http://www.linuxdj.com/hdrbench)
In practice at each processing loop we check what are the most empty tracks
(eg the tracks which are more likely to go in underrun status)
and refill these first.
This is achieved by sorting the tracks by available buffer space and
process only the top ones.
INSTALLATION / COMPILATION AND USAGE:
- be sure to have a low-latency kernel running
(see http://www.linuxdj.com/audio/lad for more infos)
- tune all IDE disks with hdparm -d 1 -c 1 -u 1 /dev/youridedisk
- disable APM
compilartion:
a simple "make" should suffice
it will create two apps
gensamples and disksamp
gensamples is a program to generate simple instruments playable by disksamp
the harddisk-based sampler.
gensamples generates a file which is about 120MBytes long.
Every note as a length of about 20secs and every MIDI note
ranging from midi-pitch 24 to 84 is stored in the sample.
That means when you press different keys on your MIDI keyboard,
differens parts of the file get accessed.
gensamples has two options
./gensampls FILENAME WAVETYPE
FILENAME is the output file ( for example test.dat )
WAVETYPE can be either:
sine (generates a pure sine wave)
saw (generates a pure sawtooth wave)
square (generates a pure square wave)
for example
./gensample test.dat saw
generates the instrument test.dat in the current directory using the
sawtooth waveform.
now to play the disk sampler,
simply invoke: (as root or it will not be able to run with realtime
privileges which means no dropout-free operation possible)
./disksamp test.dat
the default midi in port is /dev/midi00
the default number of fragments is 3
the default fragmentsize is 128
(total buffer size 3 x 128 = 384 bytes)
optional parameters:
--fragmentsize x sets the fragmentsize to x
--numfragments x sets the number of fragments to x
--midiport x sets the midi input device to x ( for example /dev/midi01 )
since the output is stereo (stereo features not used yet)
128/ 2 / 2 = 32 samples per fragment.
Processing 32 samples at time gives you a nice event granularity
of about 700usec, that means that eventual controller data
(like volume , filter etc) will sound ok, since we are able to process
the data faster than the MIDI cable is able to deliver it to us.
which corresponds to a 2.1ms buffer size
If your MIDI card is not flawed summing up the 1.1ms it takes
to deliver a Note-on message (wire speed) , you should
get a total Note-on to audio-out latency of 2.1+1.1 = 3.2msec
(plus eventual analog delays)
I must say the Note-on response feels quite good
(try it with the sawtooth sample).
I'd like to hear from you how the Note-on response feels to you.
(especially if you have the possiblity to compare it to hardware
samplers / synths).
Please let me know what do you think about this software,
eventual flaws, bugs , what could be improved, ideas
algorithms etc etc.
happy hacking !
Benno
sbenno_AT_gardena.net
This archive was generated by hypermail 2b28 : Thu Jul 13 2000 - 02:30:56 EEST