LDraw to Web

[Nick McBride]

Hi- I've been working on a converter to take ldraw dat files and parse them so they can be displayed in 3d in a canvas element on any web page. So far, I've had no problem doing this with any single parts (it works great), but I'm having trouble with assemblies (single dat files, with other referenced dat files). What's tripping me up is the 3d transformations.

I'm trying to understand the calculations that are at: http://www.ldraw.org/Article218.html#lt1


In the calculation:
u' = (a * u) + (b * v) + (c * w) + x;
v' = (d * u) + (e * v) + (f * w) +y;
w' = (g *u) + (h * v) + (i * w) + z;

Is u, v, and w the X, Y, Z coords of the vertices that we're transforming under the part itself, and the x, y, z come from the line type 1, where it gives the origin of the entire part?

My results that I seem to be getting are somewhat of a smeared part in one direction. This tells me that some of the vertices are transforming, but not all. I just need to double check that I'm doing this in the correct order.

[Roland Melkert]

Are you multiplying the reference type 1 line (4x4) matrix recursively with the parent's matrices?

And if so, are you multiplying in the right order? (backwards)

[Nick McBride]

Roland Melkert Wrote:
-------------------------------------------------------
> Are you multiplying the reference type 1 line
> (4x4) matrix recursively with the parent's
> matrices?
>
> And if so, are you multiplying in the right order?
> (backwards)


I'm not sure I entirely understand what you're asking, let me further illustrate what I'm exactly doing...

Here's some quick pseudo code of what I'm trying to do:

//Parent File one-liner
1 0 -20 -36 0 0 1 0 0 0 1 1 0 0 fullbeam.dat
// x y z a b c d e f g h i


//fullbeam.dat sub-file one-liner
3 16 44 0 4 44 2 4 44 0 10
// u v w u v w u v w

foreach VertexPoint in fullbeam.dat{
u' = (a * u) + (b * v) + (c * w) + x;
v' = (d * u) + (e * v) + (f * w) + y;
w' = (g * u) + (h * v) + (i * w) + z;
}

So, with the 3 vertices that are listed in the fullbeam.dat:

Loop 1
u' = (0*44) + (1*0) + (0*4) + -20
v' = (0*44) + (0*0) + (1*4) + -36
w' = (1*44) + (0*0) + (0*4) + 0

Loop 2
u' = (0*44) + (1*2) + (0*4) + -20
v' = (0*44) + (0*2) + (1*4) + -36
w' = (1*44) + (0*2) + (0*4) + 0

Loop 3
u' = (0*44) + (1*0) + (0*10) + -20
v' = (0*44) + (0*0) + (1*10) + -36
w' = (1*44) + (0*0) + (0*10) + 0


So my resulting transformed coords for line 3 in fullbeam.dat would be:
1 0 -20 -32 44 -18 -32 44 -20 -26 44

[Roland Melkert]

Nick McBride Wrote:
> I'm not sure I entirely understand what you're
> asking
>
> For example:
> Say the line reads 1 10 -40 -10 -20 1 0 0 0 1 0 0
> 1 1 x.dat
> x y z
> a b c d e f g h i file
>
> I'm running through x.dat and pulling all the
> vertexes from each line and using this calculation
> on them:
>
> u' = (a * u) + (b * v) + (c * w) + x;
> v' = (d * u) + (e * v) + (f * w) +y;
> w' = (g *u) + (h * v) + (i * w) + z;
> where u, v, and w = the x,y,z coords from each of
> the the vertexes in x.dat. The a,b,c,d,e,f,g,h,
> and i come from the line 1, and so does x,y,z
>
> the u',v',and w' would then replace the vertices
> that were coming from x.dat

I take it you haven't worked with matrices before?

The type 1 line tells you how to transform the vertices of the referenced part in order to place it in the calling document's model space. This is done by a rotation (incl optional scaling) and translation. The rotation/scaling is given by a..i and the translation by x, y and z. But LDraw files are highly recursive so any file within the called file has to ALSO apply the transformation of higher level references.

To do this the 'easy way' you must use the 12 rotation and translation numbers to construct a 4x4 matrix which can be multiplied with the higher level matrices and then applied in one step to the vertices.

The formula you using above is actually a simplified matrix transformation for a 1x4 with a 4x4 matrix, which in it's 'pure' form looks like:

Code:

abs ver.  vertex   ref matrix
/ x'\    / x \   / a b c x \
| y'|    | y |   | d e f y |
| z'| =  | z | * | g h i z |
\ 1 /    \ 1 /   \ 0 0 0 1 /

But it's only the one type 1 line matrix so it will only work for one level deep files. You need to multiply all level matrices with each other in the 4x4 matrix and use a full transformation formula (this one ignores the bottom line, cause it always results in zero)

For example you have the following file structure

Code:

a.dat
AB -> b.dat
AC -> c.dat
         CD -> d.dat
         CE -> e.dat

To render e.g. e.dat absolutely you need to apply:

Code:

abs ver.  vertex    CE Matrix       AC Matrix
/ x'\    / x \    / a b c x \     / a b c x \
| y'|    | y |    | d e f y |     | d e f y |
| z'| =  | z | *  | g h i z |  *  | g h i z |
\ 1 /    \ 1 /    \ 0 0 0 1 /     \ 0 0 0 1 /

So in short you need to do some research on (rotation) matrices and get your hands on a matrix math library for your programming language or write one your self (you actually only need a multiplication and a transformation routine).

Hope this helps you, or at least gets you started. I'm by no means an expert so maybe another forum user can explain it better.

Anyway, I know matrix math can be hard to grasp. When I started with LDraw it was all magic to me too (it still kinda is)

[edit] -> forgot to add the '=' part of the formulas.

[Nick McBride]

Thanks for the good lesson. In the specific way I'm planning on using my converter, the files will only be one level deep, so I don't have to worry about recursion this time around (but always good to know). So I guess I'm going to have to do a little more research on the whole matrix thing. I thought my calculations were correct, due to it only being one level deep. I see your point with having to apply it at the higher levels (from the bottom up), but if we're only talking 1 level deep, my

u' = (a * u) + (b * v) + (c * w) + x;
v' = (d * u) + (e * v) + (f * w) +y;
w' = (g *u) + (h * v) + (i * w) + z;

formula is still correct, right? But then again, I might be a little thick :-)


Just wanted to add a picture of what's happening:
http://www.altoonalights.com/block.gif It's a custom part, but you can see that it would seem that one of the sides is intact, while the rest is "smeared" out of whack.

[Nick McBride]

So I fixed the issue after realizing how thick I was.

I passed the translation variables into a function that was doing the conversion for me. The translation variables were named letters a-i.

Within my function I had a loop, which used "i" as an increment and counter. It simply was over-writing the "i" that came from the transformation matrix. I renamed the variables, and it works like a charm.

Special thanks to Roland though, who helped clarify my math and give me a little more understanding what was going on.

[Jim DeVona]

Cool! A canvas-compatible LDraw renderer would be quite useful.

You might be interested in this very simple LDraw renderer written in Processing (a graphics programming system based on Java). After setting it up, I tried using it to make an LDraw applet you could embed in a web page. You can try it here. I haven't done anything with it since, though, mainly because Java applets are kind of finicky and other technologies (like what you're working on) seem better suited for today's web.

[Steffen]

While reading the above interesting thread,
it just jumped to my mind that we could think of using JavaScript to render a *.dat or *.ldr input
to the browser's canvas.

As this will probably not be a lot of code, we could bear with the ugly JavaScript syntax
and at the same time profit from the recent speedup in JavaScript engines (Google's V8, Firefox's etc.),
and at the same time save users from having to install Java first.

Being a big Iron browser fan (the de-googled version of Chrome,
see http://www.srware.net/en/software_srware_iron.php ),
I start liking that idea more and more:

Have a look at
http://www.chromeexperiments.com
to see what is possible with JavaScript nowadays

(I am told that there are even translator tools which translate nice Java code into JavaScript code,
but I have to find out more about this first)

[Steffen]

using that suggestion would also allow to view LDRAW models on tablet (pad) computers
running either Android _OR_ iOS.

so no special Apps need to be developed for each target platform,
just simple JavaScript painting on a browser canvas - this is something that all current browsers
on all platforms can do
(for example Opera on some mobile devices etc.)

see also here
http://news.lugnet.com/cad/?n=17489&t=i&v=a

[Roland Melkert]

Interesting, but I won't be fast and it probably needs alot of mem for starters.

And where does the script get the needed dat files? I'm still not sure why the original topic starter thinks he doesn't need recursion. Cause that means you can only render primitives which isn't very exiting imho

Using the clients local library raises security issues (most browsers won't allow it by default). Using some remote server might get slow very fast (not too mention needed bandwidth etc).

An alternative would be to put every thing in a mpd so no further dependencies are needed, but that needs additional coding on the site serving the canvas script (a.g. attach a model to a post will automatically generate a (potential gigantic) standalone mpd). But if the server needs to do this, it could just as easily generate a png or something instead using e.g. a custum ldview version.

Just my 2cts

[Jamie B.]

On the parts angle, I don't think it would be that bad. Parts are already HTTP accessible from LDraw.org, although I doubt it's set-up to handle this kind of load.

An app can cache the parts using one of the several HTML5 storage options, as well as using HTTP caches. Additionally, parts average 10k each and subparts average 5k (very compressible). Include procedurally-generated subparts (a la LDView), and I would consider this reasonable downloads.

As for the speed of JavaScript, Chrome and Firefox are both JIT compiling now, so it should be comparable in speed to Java.

[Roland Melkert]

Jamie B. Wrote:
-------------------------------------------------------
> As for the speed of JavaScript, Chrome and Firefox
> are both JIT compiling now, so it should be
> comparable in speed to Java.

I was more thinking about the canvas speed, I haven't done much on this subject lately so I don't know if you can easily get a fully hardware accelerated OpenGL context.

[Steffen]

this is a 3D building application, loading parts on demand.
you can construct a robot, first from basic parts, then add smaller ones,
colorize it, apply stickers etc.

http://www.chromeexperiments.com/detail/...n/?f=webgl


here are more WebGL demos
http://www.chromeexperiments.com/webgl

I think that all major browsers will soon have that technology & performance
http://en.wikipedia.org/wiki/WebGL

[Orion Pobursky]

Jamie B. Wrote:
-------------------------------------------------------
> On the parts angle, I don't think it would be that
> bad. Parts are already HTTP accessible from
> LDraw.org, although I doubt it's set-up to handle
> this kind of load.

I ask that this not be tried for the reason above. Our web space is donated by Peeron and I don't want to increase Dan's traffic.

[Jamie B.]

Roland Melkert Wrote:
-------------------------------------------------------
> I was more thinking about the canvas speed, I
> haven't done much on this subject lately so I
> don't know if you can easily get a fully hardware
> accelerated OpenGL context.

WebGL is exactly that. While function calls won't be as fast as C, the hard work is done by hardware.

For practical tests, try http://lawriecape.co.uk/threejs/webGL.html. On my puny little chromebook, it clocks in at 5-7FPS. (I consider this acceptable for a web viewer.)

[Steffen]

Suggestion: I would like to ask the three threads

http://forums.ldraw.org/showthread.php?tid=1167,1529
http://forums.ldraw.org/showthread.php?tid=1527,1527
http://forums.ldraw.org/showthread.php?tid=856,1022

merged into a single thread "LDraw App / LDraw Web Renderer"