LDraw.org Discussion Forums

Full Version: Introducing a second HL library?
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4 5 6
Hey,

yes, I also think this is worth a second look. This is not the first time I think about such a thing.

But, I would love to see a new file format: Something like a mixture between standard LDraw and an extension of the .obj file format:
  • At first you have to define all needed vertices (optional with normals, uv-map, color, etc). I can imagine that it would also be possible to add new properties later (e.g. bone-weight, render information, etc).
  • With these vertices you can now define (via indices) lines, triangles, (quads, polygons,) optional lines, etc. Here it might be possible to add new properties later (render information, textures, etc), too.

But (without thinking long about it), I would try to keep the reference lines and primitives. I think this architecture is something that is perfect for a 3D LEGO file format. But you have to be consequent: when there is a cone primitive, you are not allowed to use it in a spherical area ;-). If you need something special you can still ‘inline’ the primitive.

The tools should only interpret that lines they are interested in. All other lines have to be ignored (not only the comments). By this it would be much easier to extent the format in the future.

With something like this it would be possible to create gradient brushes
[Image: opentk_test2.JPG]
and texturing would also be covered.

Just some of my thoughts…

Rolf
Chris Dee Wrote:I like the idea of the new library being programmatically derived each time the current library is updated, if indeed that is feasible.

So it sounds to me like the existing library could be considered to be conventional "source code" and the new, enhanced version a kind of "compiled" version. Since objects in the new library would never be created manually there's no reason for it to be text format, so it could be binary, and thus possibly smaller (4-byte float numbers support 6 significant decimal digits), although if the complilation inlines all subpart and primitive references that may not be the case. In keeping with the LDraw ethic, I think it should be Open Source or Creative Commons. It also wouldn't need to be 8000+ separate files in one folder, which I'm aware has been mentioned as a scaleability concern.

However, it seems pointless (and ambitious) for us to try and define a new binary graphics file format, if there is already one out there that provides all we need. I don't know enough to evaluate that possibilty myself. I think connectibility might be a missing feature of existing formats, but we haven't really incorporated that into the LDraw file format either, apart from a notional acceptence that studs form a connection point.

If we go for an existing format, we should choose one that permits attaching custom data, so that we can put ldraw-related informations in it, such as connection points (as you said) or condlines, etc.
I think we don't have that complicated of a format, we don't have textures, UV, material related stuff, etc. So a custom format could still be a good (and simple enought) solution.

Actually i think the brevity of ldraw format is not as great as it is pictured. Sure, the recursive nature saves space, but the vertices are stored in a very inefficient way: they are repeated in each face instead of being listed beforehand and referenced in faces. This makes "complex" parts very verbose and possibly waste all the gain from primitives.
Maybe an XML wrapper file? We could have a tag the references the raw DAT code and other tags for connections, textures, etc...
Quote:we don't have textures
We do, through texmap extensions.
Quote:Actually i think the brevity of ldraw format is not as great as it is pictured. Sure, the recursive nature saves space, but the vertices are stored in a very inefficient way: they are repeated in each face instead of being listed beforehand and referenced in faces. This makes "complex" parts very verbose and possibly waste all the gain from primitives.
Agreed (though conversion into other 3D formats show that LDraw format is overall very efficient). And it has the big benefit of beeing "local" (a single line carries all information needed to be meaningful).
I too was thinking .obj, I just don't know if it allows for extension without confusing other software capable of importing 'clean' obj. Because it would be very nice to use standard .obj for the main mesh etc but still have the option to (at least) add conditional lines without breaking the format for other tools to use it as is.

Generating the basic .obj would actually be pretty basic as it's almost the same info/data you need to generate in order to draw the part efficiently in OpenGL.
So what?! I love when someone pops in from time to time making a lot of fuss how and what should be change - better WE should change - and then vanishes into thin air right afterwards:

http://forums.ldraw.org/showthread.php?t...6#pid13116

w.
Willy Tschager Wrote:So what?! I love when someone pops in from time to time making a lot of fuss how and what should be change - better WE should change - and then vanishes into thin air right afterwards:

http://forums.ldraw.org/showthread.php?t...6#pid13116

w.

well, if you're talking about me, i'm not vanished, i'm here and i expressed my points of view in these posts and replies. Some people replied but if discussion stops there's no point in talking alone Smile Maybe there's no real interest in changing things, which i'm ok with.

also, i didn't even "popped in". Much to my surprise, my project brigl got picked up by most major lego sites, lately on brickset where i expressed my opinions on ldraw. My comments on brickset were discussed in here. So i just felt like i had to reply.

I already stopped working on brigl years ago, when i found that i was unable to craft the delicate algorithms required to make ldraw work on modern tools (such as Three.js) and lost interest.

So you can think i'm a poor programmer who couldn't get his algorithms together and pretends that everybody else changes theirs, and that's a legit point of view, but i still think that all the extra work needed to deal with the ldraw format is completey superfluous and only stems from some bad/outdated foundations of the architecture. When you find yourself wasting days to fix a problem that you know shouldn't even be there, you lose motivation.

As for helping, as i said i'm not a C/C++ programmer, and onestly i don't have the time to learn a new language. I can work with java and of course javascript, if there's anything that can be done with them i'd love to.

Tomorrow i'll write some more details about my ideas Smile
I think the first step needed before any other is to get rid of all detectable and auto correctable defects in the existing official library.

I'm willing to try and do that using a dev version of my LDCad, which will then generate corrected files for any file currently generating warnings in it's log as they are already corrected in memory. So I just need to make some (small) adjustments in order to write those files to disk.

But afterwards someone else needs to take those (probably many) files and merge them with the pt etc, anyone interested?
Of course our library should be error-free.
It simply isn't yet, because humans make errors and because we lacked time to find all.
However, over time these errors will be gone.

I like it that your post has triggered us to cure these problems.

When 2 years ago I wrote a quick LDRAW parser myself, I also ran it on all official and unofficial files for testing,
and found many BFC syntax errors in official files.
I took the effort to cure all of them and uploaded them to the PT.

However, what I do not like is that you're attacking the file format itself so fundamentally.
Yes, the files have errors like bowtie quads etc., but it still is up to your program to be robust against such.
You don't even have to correct them there.
You can simply skip broken things.
This responsibility cannot be taken from your shoulders. Otherwise your program would require that all its input
is error-free, which does not happen in real life. If you feed a broken JPG image to your painting program,
it should report an error or ignore certain parts of the file, but it certainly shouldn't crash.
Your program needs to be robust against broken input.

Regarding using BFC CCW everywhere, of course, this is trivial to do for all our parts and subparts,
and I like doing that - simply for simplicity.
However, we still need the full BFC syntax like 0 BFC INVERTNEXT so we do not have to duplicate
all our primitives. So changing all parts and subparts to 0 BFC CCW is a nice-to-have, but I would regard
that a low-prio task. We should more focus on fixing the remaining [few!] errors in official files
like bow-tie quads and unknown colors.

The discussion about normals is more complex and already in a different thread.
I'd like to add to that discussion maybe in another post.

I like the simplicity of the LDRAW file format a LOT over other formats, especially binary ones.
The file format has made us very productive, the library has grown HUGE over the past years,
and although there are quirks and problems at some places, many, many, many parts have been created.
There's 2 sides of our library: authors are working a lot with the files and producing parts,
and consuming applications parse them. Bringing both worlds together sometimes needs balancing.
For example, yes, it might be difficult for brigl to fix a bowtie quad - but, hey, then simply ignore it.
It anyway is broken in the input file. On the other hand, curing that quad is easy for a part author,
and it is not the fault of the file format that the bowtie quad is present. Would we choose another file
format, then there would be other errors in it, and your application again would need to be robust against them.
Roland Melkert Wrote:I think the first step needed before any other is to get rid of all detectable and auto correctable defects in the existing official library.

I agree 100%, this is so cheap there's no reason not to procede right away.

Roland Melkert Wrote:I'm willing to try and do that using a dev version of my LDCad, which will then generate corrected files for any file currently generating warnings in it's log as they are already corrected in memory. So I just need to make some (small) adjustments in order to write those files to disk.

But afterwards someone else needs to take those (probably many) files and merge them with the pt etc, anyone interested?

I hope this is not a problem Smile
Pages: 1 2 3 4 5 6