Welcome! Log In Create A New Profile

Advanced
Re: Compressed Library was Re: Compressed LDraw files
January 31, 2013 12:26PM
I looked at compression efficiency when we were selecting a compression format for X-Plane scenery data. When maxing out compression (not always a good idea as it increases encode time significantly) I found that from worst to best compression ratio the list went:

pkzip, bzip2, rar, and 7zip.

Rar is heavily license-entangled, so it's not really an option. We used 7zip because it had the best ratio, but for LDraw, I'd recommend pkzip because of the wide availability of simple, clean, portable compression/decompression utils. (7zip's quite a bit messier.)

I also found that compression ratios were very sensitive to source content. Our file format is a heavily bit-packed binary format, and it has optional RLE encoded in some cases. I found that with 7zip, the RLE encoding significantly worsened 7-zips compression ratio...the moral of the story was: let the experts do the compression.

So...I suspect that the compression ratio differences are coming from iteratively superior compression algorithms (no surprise the oldest tech does the worst) but YMMV based on file contents.

Honestly, the more I read this discussion, the less valuable compression suppot seems, relative to other possible development work. My LDraw file is 138 MB on disk. By comparison, LDD is using 562 MB on disk, itunes 253 MB, Keynote 562 MB, and FireFox 184 MB. My MOC airport is only 6.6 MB on disk for 22,000 bricks and months of my lif

I'm not saying that this is okay or that I'm happy that my web browser is approaching nearly 200 MB in size, but it seems to me that the data sizes the LDraw software suite deals with are quite tame. :-)

Anyway, sorry for the rant...my take-away point is that given multiple options, I would suggest we optimize for developer convenience and not storage efficiency, since that last few % improvement from pkzip to 7zip aren't going to turn into a huge user experience win, but I think we are limited on volunteer dev time.
SubjectAuthorViewsPosted
Compressed LDraw files Roland Melkert561January 29, 2013 12:33PM
Re: Compressed LDraw files Tim Gould267January 29, 2013 12:44PM
Re: Compressed LDraw files Travis Cobbs265January 29, 2013 12:49PM
Re: Compressed LDraw files Roland Melkert261January 29, 2013 02:01PM
Re: Compressed LDraw files Tim Gould264January 29, 2013 02:10PM
Re: Compressed LDraw files Ben Supnik308January 29, 2013 10:36PM
Re: Compressed LDraw files Roland Melkert299January 30, 2013 10:52AM
Re: Compressed LDraw files Ben Supnik312January 30, 2013 12:59PM
Re: Compressed LDraw files Allen Smith294January 30, 2013 09:17AM
Compressed Library was Re: Compressed LDraw files Travis Cobbs316January 30, 2013 09:40AM
Re: Compressed Library was Re: Compressed LDraw files Philippe Hurbain300January 30, 2013 09:49AM
Re: Compressed Library was Re: Compressed LDraw files Allen Smith276January 30, 2013 08:43PM
Re: Compressed Library was Re: Compressed LDraw files Travis Cobbs280January 30, 2013 10:27PM
Re: Compressed Library was Re: Compressed LDraw files Roland Melkert311January 31, 2013 09:55AM
Re: Compressed Library was Re: Compressed LDraw files Travis Cobbs306January 31, 2013 11:01AM
Re: Compressed Library was Re: Compressed LDraw files Roland Melkert293January 31, 2013 11:07AM
Re: Compressed Library was Re: Compressed LDraw files Travis Cobbs297January 31, 2013 11:24AM
Re: Compressed Library was Re: Compressed LDraw files Ben Supnik307January 31, 2013 12:26PM
Re: Compressed Library was Re: Compressed LDraw files Roland Melkert302January 31, 2013 01:09PM
Re: Compressed Library was Re: Compressed LDraw files Steffen393February 02, 2013 05:34PM
Re: Compressed Library was Re: Compressed LDraw files Tim Gould319January 31, 2013 04:02PM
Re: Compressed LDraw files Roland Melkert322January 30, 2013 10:57AM
Re: Compressed LDraw files Ben Supnik312January 30, 2013 01:15PM
Re: Compressed LDraw files Roland Melkert307January 30, 2013 02:07PM
Re: Compressed LDraw files Allen Smith277January 30, 2013 08:39PM
Re: Compressed LDraw files Roland Melkert304January 31, 2013 01:06PM
Re: Compressed LDraw files Travis Cobbs308January 31, 2013 01:59PM
Re: Compressed LDraw files Travis Cobbs279January 29, 2013 12:46PM
Re: Compressed LDraw files Roland Melkert266January 29, 2013 01:57PM
Re: Compressed LDraw files Steffen260January 29, 2013 03:23PM
Re: Compressed LDraw files Travis Cobbs309January 29, 2013 04:17PM
Re: Compressed LDraw files Roland Melkert322January 30, 2013 10:49AM
Re: Compressed LDraw files Steffen298February 02, 2013 05:33PM



Sorry, only registered users may post in this forum.

Click here to login