LDraw.org Discussion Forums

Full Version: Compressed LDraw files
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4
Hi all,

I'm thinking about supporting zipped ldraw files in LDCad without the need for users to unzip them first. This idea comes mainly from the fact my new deform functionality tends to create quite large files. These files can be compressed extremely well (1.5MB becomes a couple of K etc).

It would be almost trivial for me to implement an unzip before parsing and a zip before saving files in my editor. But I was thinking maybe other software authors would like to provide the same support.

So a semi serious public discussion about how to name/handle compressed LDRaw files might be in order. That way we could all use this.

I was thinking about using zlib (other authors could use any other zip supporting library) and simply prefixing "gz-" to the file extension, so e.g. "someBigModel.mpd" becomes "someBigModel.gz-mpd". The contents of the mpd itself stay unchanged.

I would recommend against using "gz"ip in the name, simply because most windows users have no idea what it means. I'd say simply appending a 'z' would be better eg. mpdz or ldrz.

But are big files really an issue in this day and age? As far as I know even downloading them from most sites on the internet will be transparently performed compressed (I could be wrong).

It sounds like a good idea, but you need to be careful with your terminology. Used alone, the term "zip" is generally interpreted to mean a PKZIP compatible multi-file archive (with .zip extension), and nothing else.

I personally agree that using zlib to create a gzip-compatible file is a good route to take, and would probably add support for this to a future LDView version if it became at all popular. However, if you go this route, make sure that the files you produce are actual gzip files (with valid gzip header at the beginning, plus checksum and uncompressed size at the end). The zlib library contains sample code for creating .zip files, but I don't feel these would be a good idea.
I agree that simply appending z to the file extension is better (and done in a number of other file formats that support having a gzipped version).
zip/gzip/zlib might be confusing indeed,

I'm using wxWidgets stuff for handing zip files already (the .sf seed files), But the wx library also supplies similar classes for zlib related compression I'm assuming those classes take care of all header and trailer stuff and therefore was hoping to use them in a similar matter.

I agree it should be a proper gzip file so most os/archive managers know how to handle them automatically (most windows archive managers will offer to (right click) extract them even when the extension is non standard). I will have to make sure wxWidgets' class generates such files, otherwise I'm going to use zlib directly.
If z at the ending is more common I'm ok with that, no need to introduce even more confusion Smile

On the drive size issue Tim raised, I agree with you in general but having bought an SSD recently I also hate to see (any) space wasted. And because I prefer quality over smaller files sizes, (optional) compression seemed like a nice all round solution.
Don't get me wrong, I think it's not a bad idea if it takes off and would like to see support for it. Can always uncompress and edit.

However I'd definitely recommend against making it the default setting.

May I suggest to use concatenated extensions?

so the model mymodel.ldr, when compressed, becomes mymodel.ldr.gz or mymodel.ldr.zip or mymodel.ldr.7z etc.

this way, the files can be normally opened using the normal compression programs.
also, e.g. the Windows shell will treat them properly.

your tool might recognize these double-extensions and directly handle them.

the second portion indicates the compression standard that has been used.
The problem with double extensions is that they have essentially zero support in any operating systems. The operating system sees the final extension, and nothing else, so the files become solely compressed files. I think Roland wants the files to be "compressed LDraw" files. That way, when a user double-clicks on one, it opens LDCad automatically. It's not appropriate for LDCad to claim to open all .gz files, because it can't do that; it can only open these specific ones. So if double extensions were used, LDCad could not be reasonably made the proper handler for them.
My 0.02: I like the .mpz idea - a changed suffix to indicate to casual useres that:
1. This isn't the .mpd format you are used to - don't open it in WordPad and
2. Your favorite editor might not understand it.
It's cute for developers that the files are 'secretly' standard compression formats and thus very easy to process, but that's a power user feature.

(With X-Plane we went the opposite route and allowed 7z archives with _no_ change of suffix....when the system handles them transparently it's great, but when an old tool fails to understand why the file is filled with junk, the user becomes confused because the extension doesn't look different.)
Pages: 1 2 3 4