[LAU] Music Editors for Blind Users?

Hi!

A visually impaired Linux user here. I recently came across this great
thread[1] and decided to write this message. My goal is a little
different from the original topic author (generating MIDI via code VS.
recording via a keyboard) so I decided to start a new discussion.
Here's my situation (warning: very long post!):

First of all, I'm just an amateur and a beginner with nearly zero
knowledge about music production. I had very little contact with DAWs,
even in Windows... The closest I've gotten to a "DAW" that I tried to
use was Anvil Studio (many years ago), which wasn't accessible either.
The little that I know about MIDI editors and DAWs is from what I read
looking for such programs in Linux.

I'm using Slackware Linux, with Speakup[2] as my console screen reader
(with Espeak[3] through the Espeakup[4] connector, because I don't
have a hardware voice synth), and Orca[5] as the screen reader for X
(note: I use only ALSA, without Pulseaudio).

I'm trying to create some kind of audio setup for quite some time
(since around 2010). Back then, I read everywhere that it was needed
to recompile the kernel with a realtime patch. Then I found the
courage and did just that, but:
* I never got JACK to really work. It complained that the soundcard
was already being used, probably because of my screen readers which
are softsynths (again, I don't have a hardware synth nor a Braille
display), and I suspect they take all the soundcard for themselves.
JACK only got to start if I used a second soundcard, an M-Audio Fast
Track Pro (which I bought because it seemed to work on Linux), but it
didn't go too well either. Only much later (last year, 2013) I came
across this post
http://joegiampaoli.blogspot.com/2011/06/m-audio-fast-track-pro-for-debian-linux.html
It seems the Fast Track Pro is already supported on newer (3.X)
kernels, but I still didn't test that.
* Most audio related apps were (and still are) made with QT GUIs.
Though QT's accessibility API exists for a long time (and is well
documented[6]), there were no means for Orca to communicate with it
yet, so all those apps were completely inaccessible.
* With the recompiled kernel, my video driver installation broke (I
wasn't even using proprietary drivers or anything). X only showed a
blinking cursor on the screen. Orca worked just fine, but it was a
problem when a sighted person needed to use the computer.

So, I went back to the stock kernel (where the video driver worked)
and gave up, for the most part. I settled on doing everything from my
keyboard (which requires some sighted assistance), then sending the
result as a wav file to my computer, then processing it with SoX[7]
(which I use all the time) and/or Ecasound[8]. But recently I began
thinking of using sounds from softsynths or other programs (for
example, LinuxSampler[9]) instead of my keyboard's internal sounds,
and for that, I think it would be best if I could record MIDI directly
into my computer. So, the search started again, and I began testing
some command line applications (most of which were already mentioned
on the thread in [1]):
* CuSe[10]: stands for "Cursed Sequencer", and like the name suggests,
is Curses based. Interestingly, it was made specifically with visually
impaired users in mind. Some features look interesting, like punch
in/out, a step sequencer, and (the most interesting of all) a "remote
control" like thing, where it's possible to configure key combinations
for controlling the software through the hardware synth's keyboard
(avoiding the need to run back and forth between the synth and the
computer). But it doesn't seem to have some features I want, like
adding/editing lyrics to the notes, MIDI CCs and SysEx events.
Unfortunately, I was never able to get any sound from it (be it
through FluidSynth or my MIDI keyboard).

* Midish[11]: already mentioned on the thread in [1], it supports
recording and playing MIDI, adding events and SysEx messages, and it
has nice filtering features, making it possible to split the keyboard
or transform one event into another. It only supports raw MIDI ports
and it took me a lot of time to figure that to make it work, I needed
to "modprobe snd_virmidi", then (a)connect the virmidi ports to my
synth's ports, then configure the inputs and outputs via the
"~/.midishrc" file. It was the program I got working more
successfully, and I placed my hopes on Midish for a long time, reading
and rereading the manual to see if I could make it do what I wanted. I
was able to record MIDI via my keyboard (though I wish there was a
"remote control" feature or something similar). However, sometimes I
also wanted to insert notes manually, and as already pointed out in
this message from the thread in [1]
http://lists.linuxaudio.org/pipermail/linux-audio-user/2014-February/096120.html
that isn't the most intuitive task, and Midish doesn't seem to let me
add lyrics or edit notes (only cut/copy/paste measures)... Well,
unless I save the song to Midish's format and open it in a text
editor, but I don't want to have to memorize which note numbers are
which :P. Or unless I'm missing something else.

* Midiedit[12]: contrary to Midish, this only edits and doesn't
record. It's contained in only one file and is a Perl script. It has
two interfaces: a Curses based one and a "text editor" based one. With
the Curses interface, it can playback MIDI files (I used Aconnect to
hook its ports to FluidSynth and the sounds played), and as the name
suggests, edit. Events (notes, CCs, program changes etc.) can be
inserted, edited and deleted. They're displayed sequentially in a
list, one per line, with their starting position in ticks, duration
and channel. They can be navigated by using the Up and Down arrows,
then edited (the bottom screen shows the available actions and which
keys do what). One thing I love about it is that it plays back the
notes when the cursor focuses on them. Another thing is that it treats
each note as a single event, by matching the note ons with their
respective note offs, which makes managing notes much easier. So far
so good, but everything always has some "cons". I wish it could mute
or solo tracks, and have a separate view for each track. The search
function can be used to jump straight to the notes from a specific
channel, and cycle through the results with "n" and "N" (similar to
Vi/Vim), but the notes from the other tracks are still there and it
gets confusing easily (at least for me). It doesn't seem to have an
action implemented on the interface to handle lyrics and text events,
but these still can be edited through the "text editor" interface,
which is called "dump mode". In this mode, it opens your text editor
of choice (the one in the "$EDITOR" environment variable) and
represents the MIDI file's contents as readable Perl code (based on
the dump output from the MIDI::Perl[13] module). All events (including
text events, lyrics and SysEx) can be edited this way, (although
obviously it doesn't play back the notes, and we don't have the luxury
of the unification of note ons and offs). After saving and closing the
editor, the Perl code is processed and the content is written back to
the MIDI file.

* From the same author of Midiedit, there are other interesting Curses
based MIDI tools[14], like Midifade[15], which generates "sliders"
for controlling MIDI events through the PC keyboard (on the command
line, we specify how many sliders to generate, the ALSA port, the MIDI
channel and which event each slider will control), and can, for
example, change Ecasound's controller parameters, or the "knob"
parameters from a softsynth.

That's it for the CLI apps I tested. Midish and Midiedit are the ones
I had most "luck" with, but my attempts with them weren't that
productive either.

My search for an accessible MIDI editor or DAW continues. Recently, I
updated Slackware to 14.1. I'm using the stock kernel, no realtime
patches or anything yet. First I want to know if I really need it (I
read that because of changes in recent kernels, RT patches aren't
strictly necessary anymore), and if there's some app that I'm able to
use (if not, there's no point to even try compiling a realtime kernel
:P). I finally got the QT-AT-SPI[16] bridge to compile, meaning I can
access QT GUI apps to some degree now. So, I decided to test some of
the most popular Linux DAWs, those QT based audio apps that taunted me
since I first read about audio production on Linux (basically, the
only options that were available and which I couldn't use).

From the list of features I read, the one that appealed to me the most
was MusE[17]:
* The possibility of mixing MIDI with audio (samples, loops, vocals etc.).
* Instrument definition files for various brands of keyboards that
ease the use of those synthesizers, making it so that we don't need
to memorize patch names/numbers and controller numbers.
* MIDI recording.
* Editing of all types of MIDI events. Besides the visual editor,
there's a "event list editor" (I suppose it's something like in
Midiedit), so maybe it could work for me.
* A few internal instruments (some synths and a FluidSynth frontend).
* DSSI support.
* (apparently) native VST support too (and I assume, a few Windows
VSTs through DSSI-VST).
* And the "remote control" feature :D!

Of course there's the multitude of functions that depend on the mouse
(which I can't use), but it seems to have customizable shortcuts. It's
a graphical application, obviously very far from being text-based, but
I read that it also has some Python bindings, so maybe a few things
can be scripted...?

However, sadly, all isn't as good as it sounds. I installed MusE 2.1.2
and opened the interface. The menus work fine, but the screen reader
has problems reporting the interface elements (for example, it says
"spinbox" or "checkbox" but doesn't specify wish item that control
corresponds to. The "MusE Settings -> Global Settings" screen is full
of these examples and unreadable through the screen reader). I just
took a quick "look" at the interface to have an idea of what the
screen reader could read. I didn't test recording or anything yet,
mainly because of the aforementioned problem with JACK, nor could sort
out the setup part about "/dev/rtc"...

While we're on the subject of MusE, a question for the "MusErs": if I
don't use JACK (or use the "-a" switch), won't MusE ever play anything
through the PC's speakers (imported audio, internal instruments,
FluidSynth soundfont sounds)? Or will it just be unable to communicate
with JACK aware applications?

I'm thinking if I'll insist on MusE for a bit more, and if I do, maybe
I'll contact the developers about the accessibility issues.

Based on my limited knowledge of MIDI editors and DAWs which I
acquired from what I've tested or read about so far, desired features
for my dream DAW would be:
* Curses based interface.
* Both JACK and some form of ALSA Seq support (even if it's only with
raw MIDI ports, or some workaround involving sending commands through
Amidi :P). JACK would be for more complicated setups. ALSA would be
for when I only want to do some quick MIDI editing and don't want to
use a second soundcard just because of JACK (I need my screen reader,
so I can't turn it off to free the first card).
* Playback of MIDI files.
* Support of a metronome click, and a "count in" feature for recording.
* A "remote control" feature, or a way to start recording (or the
count in) only when I press a key on my hardware instrument.
* Each note shown as only one event, by matching the note on with it's
respective note off. This way we don't need to struggle scrolling
through the events trying to match the ons and offs.
* Possibility of inserting/editing/deleting all types of events,
including text events, lyrics and SysEx messages.
* A way to select a track to be "active".
* A separate list of MIDI events for each track (only showing the
contents of the "active" track), where events can be browsed with
arrow keys and be edited or deleted. Also, the possibility of
inserting new events through the interface (without using the MIDI
keyboard).
* Playback of notes when the cursor focuses on them.
* Ability to mute and solo tracks.
* Support for instrument definition files, to ease the use of hardware
synths (maybe some softsynths too). With that, I'd be able to select
the patches and controllers specific to my hardware synth directly
from the computer and with help of my screen reader, reducing the need
of asking for sighted assistance to know the patch names.
* DSSI support, or the possibility of interacting with some command
line based DSSI host.
* The ability to assign the MIDI channel and an instrument
(ALSA/JACK/whatever ports and patch names) to each track (or is this a
work better suited to a "patchbay" like app?).
* Rendering of MIDI tracks to audio with the used synths' sounds.
* Customizable keyboard shortcuts.
* Some form of scripting support, like Perl or Python or Lua bindings,
and the ability to assign user scripted functions to some keybinding
in the editor.

Some more complex features:
* The ability to have both MIDI and audio in the same project. The
audio doesn't need to be editable (just "importable"), only needs to
have its starting position adjustable to be in sync with the MIDI
tracks (triggered via some MIDI event maybe?).
* Recording of MIDI while playing back the audio tracks, keeping them in sync.
* Maybe a pattern editor, where the lists of events become a phrase,
chains of phrases make patterns and chains of patterns make a song
(but maybe it's too complex and is a discussion for another day :P).

And that was my utopic idea for an accessible Linux DAW. I don't know
how hard it is to implement all of that, specially the "MIDI alongside
audio" part. Unfortunately I understand nothing about audio
programming (honestly, I already tried :P) and very little about music
production, so a lot of features I pointed out might be complete
nonsense, or might be accomplished in a much better way (if anyone
knows a way, preferably via command line, please let me know).

As for the "patchbay" (with "jack_connect" etc.), I think even a
simple Shell Script with Dialog based menus would be fine.

As for post processing and mixing (applying LADSPA/LV2 effects etc.),
it can be left to some other app (Ecasound for example) after all
tracks are rendered to audio.

And last but not least, on the thread in [1], I found Teqqer[18]. I
couldn't test it yet because of the JACK problem (I seriously need to
sort that out), but I took a look at the default config variables in
the Python source code, and it looks like scripting support and
customizable shortcuts are mostly taken care of. From what I
understand from the shortcuts, the interface for inserting/editing
MIDI events is a bit similar to Midiedit (Up/Down to cycle through
events, although note ons and offs are separate) and it seems to have
one view for each track (Left/Right to cycle through tracks), and it
even supports patterns! :D So, minus the recording part (from what I
understood, it wasn't made for that), the lack of ALSA (without JACK)
support, I think it has the potential to become something like what
I'm looking for (plus, I always wanted to try a tracker, but never
found a "screen reader friendly" one). I'll definitely be watching
Teqqer to see where it goes!

While my dream DAW doesn't come true, does anyone know a command line
way to record MIDI while simultaneously playing an audio file and
keeping both in sync (even if it involves JACK)? I think Ecasound can
be setup to do the opposite, playing back a MIDI file while recording
audio (but it doesn't support MIDI recording, just playback... Or does
it? :P).

Well, that's it. Sorry for the overly long post. Thanks for your time!

[1]: http://lists.linuxaudio.org/pipermail/linux-audio-user/2014-January/095910.html
[2]: http://www.linux-speakup.org/
[3]: http://espeak.sourceforge.net/
[4]: https://github.com/williamh/espeakup
[5]: https://wiki.gnome.org/Projects/Orca
[6]: http://qt-project.org/doc/qt-4.8/accessible.html
[7]: http://sox.sourceforge.net/
[8]: http://nosignal.fi/ecasound/
[9]: http://www.linuxsampler.org/
[10]: http://pi4.informatik.uni-mannheim.de/~haensel/cuse/index_en.html
[11]: http://www.midish.org/
[12]: http://www.pjb.com.au/midi/midiedit.html
[13]: http://search.cpan.org/perldoc?MIDI
[14]: http://www.pjb.com.au/midi/index.html
[15]: http://www.pjb.com.au/midi/midifade.html
[16]: http://projects.kde.org/qtatspi
[17]: http://www.muse-sequencer.org/
[18]: https://github.com/fps/teqqer

-- 
____________________
Blog: http://aiyumi.warpstar.net/
_______________________________________________
Linux-audio-user mailing list
Linux-audio-user@email-addr-hidden
http://lists.linuxaudio.org/listinfo/linux-audio-user