Spaces in filenames

[Travis Cobbs]

Michael Heidemann pointed out that the newly created OMR spec requires that filenames contain spaces, while the pre-existing LDraw 1.0.0 spec strongly discourages spaces or tabs in filenames. He (quite rightly, in my opinion) felt that this was inconsistent.

So, the way I see it, we have three choices:

1. Ignore the problem.
   
2. Remove the discouragement text from the LDraw standard.
   
3. Change the OMR standard so that the filenames don't include spaces.
   

My vote is to do go with option 2, at least for spaces. I think tabs should still be discouraged, since they just seem like a bad idea in filenames (assuming they're even valid on all OSes). Please note that the official parts restriction document already disallows spaces in the filenames of files in the library, so this won't affect the parts library. (More specifically, it lists the only allowed characters, and spaces, tab, etc. are not listed.)

[Roland Melkert]

I would also go for option '2' because all major filesystems support spaces these days. Also I don't think people will stop using them anyway.

But you bring up another issue: not all filesystems have the same 'bad character' sets. So are we allowing the full unicode set and let the software/os handle wrong names or do we define the bad ones somewhere in full?

The current 1.0.0 only mentions a couple of 'bad' characters (it doesn't list : and * for example). So maybe add a full list based on e.g. fat32, ext2, ntfs (and other major fs's).

[Willy Tschager]

I'd go for option 2 and allow them for .ldr and .mpd. I would keep them however for official .dat files.

w.

[Allen Smith]

I go with Option 2.

A disclaimer that some older programs may choke on spaces is fine. But if there's still an actively-maintained program that does, I consider that a bug.

To take this even further, imposing any restrictions on user-controlled file names is not realistic. The user gets to enter a filename in a save dialog, or in the filesystem browser. LDraw programs have no control whatsoever over what a user names his files. It's up to the user to discover (the hard way) that some filesystems prohibit certain characters.

Allen

Fun Fact: On Mac OS 9, it was possible to generate filenames which contained '\0', since all strings were stored Pascal-style (length byte followed by chars). Suffice it to say that would not go over well on any system with a C heritage.

[Chris Dee]

I'm happy with option 2.

Chris

[Willy Tschager]

Any more comments on this? If not we may vote on this?

w.

[Roland Melkert]

Besides the listing of 'special chars' (yes / no) everybody seems to agree.

[Willy Tschager]

Roland Melkert Wrote:
-------------------------------------------------------
> So
> are we allowing the full unicode set and let the
> software/os handle wrong names or do we define the
> bad ones somewhere in full?
>
> The current 1.0.0 only mentions a couple of 'bad'
> characters (it doesn't list : and * for example).
> So maybe add a full list based on e.g. fat32,
> ext2, ntfs (and other major fs's).

Since I'm not a programer it is not my sandbox but I'd go with a set that is supported by the largest number of FS.

w.

[Roland Melkert]

The list should (at least) include:

':', ';', '*', '?', '|', '%', '"', '>', '<', "\0", "\t", '\' and '/'

Though '\' or '/' has an exception when it's the dir sep on the current platform. But it still technically can't be part for the file name, it part of the file path.

Also the list should stay a 'guidance', because software could implement escaping to allow just about anything if the fs supports it.

see also:

http://en.wikipedia.org/wiki/Filename

edit: I forgot ';'

[Willy Tschager]

I'm fine with it. Guys?

w.

[Allen Smith]

As I already discussed, I do not feel that it is legitimate to impose any restrictions on user-generated filenames beyond the restrictions of his host filesystem. It is ordinarily the software developer's responsibility to accept any valid input on his host platform. While it is technically feasible to implement arbitrary name restrictions on files my software reads and writes (at least on the platform I develop with), such restrictions are arbitrary, highly abnormal, and would not meet users' expectations.

I understand the desire for maximal compatibility here; it's just I don't consider this to be an issue over which LDraw has any control.

If we limit the spec to discussion of LDraw's technical details, the following technical restrictions emerge:

• Freestanding files unreferenced by any other LDraw file: no technical restrictions
  
• Files referenced by another file: name cannot begin with spaces or tabs.
  

Obviously, files in the official library cannot include any characters which are illegal on any target filesystem. That, however, is a restriction for the part tracker, not the LDraw spec.

Allen

[Roland Melkert]

I agree on not restricting the file name, but I couldn't hurt to keep the list (the standard has many info footnotes).

But when we do include the list, I just think it should be 'complete' the current one isn't.

[Willy Tschager]

How about pack these two:

• Freestanding files unreferenced by any other LDraw file: no technical restrictions
  
• Files referenced by another file: name cannot begin with spaces or tabs.
  

into the specs and add the list of bad characters to the PT reference? I guess this:

http://www.ldraw.org/library/tracker/ref/numberfaq/

would be a suitable place but the last word should be that of Chris.

w.

[Roland Melkert]

Willy Tschager Wrote:
-------------------------------------------------------
> How about pack these two:
>
>

• >
• Freestanding files unreferenced by any other
  > LDraw file: no technical restrictions
  >
• Files referenced by another file: name cannot
  > begin with spaces or tabs.
  >

That would break modularity.

Leading and trailing white spaces must be ignored uniformly, it's kinda defacto already.

>
> into the specs and add the list of bad characters
> to the PT reference? I guess this:
>
> http://www.ldraw.org/library/tracker/ref/numberfaq
> /
>
I could live with that. But I don't see the hurting in a footnote in the main spec ether.

> would be a suitable place but the last word should
> be that of Chris.
>
> w.

[Travis Cobbs]

In an attempt to match MLCAD's behavior, many programs (including LDView) ignore spaces in type 1 lines both before and after the sub-model filename.

[Travis Cobbs]

OK, we seem to be going in two different directions here, so in an effort to at least get something done about the immediate problem ASAP, I propose that we start out by doing the part we all seem to agree on right now. Unless I'm mistaken, all of us agree that the LDraw spec shouldn't discourage the use of spaces in filenames. I understand the point you're trying to make, Roland, and I don't necessarily disagree with it, but I'm hesitant to try to come up with a full list of bad characters, because it seems like doing so will almost certainly result in missed characters.

So I propose that we first make a small update to the spec, then further discuss Roland's point as a separate issue. My proposed change to the spec would be to remove the following sentence:

For reasons of cross-platform and cross-(programming) language portability, the use of white space characters (such as space and tab) in filenames is strongly discouraged.

Furthermore, the sentence following the one above would be changed to the following:

Special characters, such as &, #, |, and ?, should be avoided as they may also cause cross-platform issues and create problems when used in URLs.

Note, the above sentence would be the one that would be updated further based on Roland's proposal. All I did was remove "Other" from the beginning to reflect the deletion of the previous sentence.

---

In addition to the above, I also feel we should codify MLCad's behavior, since it appears to be the de-facto standard. This can be done by adding the following sentence to the end of that paragraph in the spec:

Furthermore, filenames must not contain leading or trailing white space characters.

However, if there's any disagreement about the above at all, then I think we should vote on the first two changes immediately, and temporarily table discussion of leading and trailing whitespace in the same way I'm proposing we temporarily table discussion of Roland's point.

[Roland Melkert]

I agree completely, let's get the main issue resolved asap.

[Allen Smith]

My proposed change to the spec would be to remove the following sentence:

For reasons of cross-platform and cross-(programming) language portability, the use of white space characters (such as space and tab) in filenames is strongly discouraged.

Agree.

Furthermore, the sentence following the one above would be changed to the following:

Special characters, such as &, #, |, and ?, should be avoided as they may also cause cross-platform issues and create problems when used in URLs.

Agree, for the sake of expediency.

In addition to the above, I also feel we should codify MLCad's behavior, since it appears to be the de-facto standard. This can be done by adding the following sentence to the end of that paragraph in the spec:

Furthermore, filenames must not contain leading or trailing white space characters.

However, if there's any disagreement about the above at all, then I think we should vote on the first two changes immediately, and temporarily table discussion of leading and trailing whitespace in the same way I'm proposing we temporarily table discussion of Roland's point.

Recommend temporary tabling this part while we vote on the whitespace thing above, then re-address immediately thereafter in a separate discussion.

Allen