Re: [LAD] NSM - handling large files

On Thu, Mar 29, 2012 at 4:40 PM, David Robillard <d@drobilla.net> wrote:
> On Thu, 2012-03-29 at 22:23 +0100, Rui Nuno Capela wrote:
>> now back to square one. to make that whole session state, folder or
>> directory, as an archival portable one is, quite frankly and imho again,
>> utopian. nuff said :)
>
> It's indeed utopian to think that some centralized special magic program
> and protocol and file format and whatever else will actually get
> universally adopted.
>
> However, files and directories and links are anything but
> unrealistically utopian.  That's my point.
>
> With all due respect, Emanuel, you can see here what happens when you
> make a big complicated mess out of things and try to ram technology down
> implementer's throats without justification: they simply reject it
> outright as a utopian fantasy.
>
> Achieving archival/etc is *trivial* without any fancy/annoying session
> management crap whatsoever.  If apps want to save in a way that prevents
> it, they will always be able to, however if they do, there is *one*
> simple rule that makes *all* the mentioned functionality possible that
> has nothing to do with any specific session manager API/protocol/file
> formats/etc whatsoever:
>
>  * All references to files outside the session directory must be a
> symlink to that file
>
> That's it.  No APIS, no protocols, no session manager file stores, no
> redundant data that may not match reality, no egregious rules.  There's
> not even a requirement for a session manager to be involved whatsoever,
> if an app saves this way independently you can archive its sessions with
> tar or whatever just the same.  If you did want a fancy GUI archival
> tool that reports what files are used and whatever else, you could write
> one, and it wouldn't even depend on the session manager at all - and I
> don't mean it would loosely depend on it by only using OSC or whatever,
> I mean it would not depend on it *at all*, nor would it depend on any
> file format[1].  All even vaguely common languages have everything
> required in their standard library, right now.  All of that Just
> Works-eyness is because it's a simple idea, and doesn't use anything
> that hasn't been baked in to the OS for literally decades.  It doesn't
> get much less unrealistically utopian than this.
>
> Erecting some fantastic non-existent architecture to achieve, in one
> special circumstance, what this simple rule does, is a folly doomed for
> failure.  To really distill it down, since we're on a "reality" trip:
> you can either try to get people to adopt this convention, or you can
> not have archivable/distributable sessions.  There is no option C.
>
> That said, if I'm overlooking something, do tell (i.e. why, exactly, is
> this simple plan not sufficient?), because from where I'm standing it's
> a real mystery why we are acting like this is so damned complicated.
>
> It's not.  At all.
>
> -dr
>
> [1] Try to sell any file format to this crowd and see how far you get...

I absolutely agree on this point. In fact, referring to external files
in this way is now on my TODO list for Non-DAW.

Currently, when you drag n' drop an external audio file into a Non-DAW
timeline (as opposed to recording it from within Non-DAW), the file
remains external with its path recorded
in the project's journal. Using a symlink for this would be better in
*at least* the following two ways:

1) Allows archiving scripts etc. to discover and import the external
source *without having to understand the Non-DAW journal format*
2) It would allow Non-DAW to import external sources *without having
to update (or break) any existing references*

I cannot imagine any argument that could propose that these are bad things.

If all Linux Audio software dealt with external references in this
way, archiving/export would be much less problematic.

However, I would also like to offer an interesting little statistic...
I, personally, have hundreds of projects representing terabytes of
data, and in all that I don't have a single project which refers to
anything external to its project directory. This is something that
only effects certain users who make extensive use of sound-clip or
sample libraries. Not people who just do plain old
recording/synthesis/mixing. So let's try not to make a mountain out of
a mole hill. What is the actual percentage of users who have
references to external files *and* a strong need to export their
sessions? I suspect that it is in fact a very low number.

Furthermore, in addition to the plain old symlinks, a truly robust
solution might also store e.g. SHA1 hashes of external files, so that
any mismatch is detectable.
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev
Received on Fri Mar 30 04:15:02 2012