WebGL renderer

[Nicola]

Hello! I'm playing with LDraw a lot lately, and i decided to implement something to display models with WebGL.
Here's the experimental results and some more information:

http://www.lugato.net/ldraw/page.html
new url: http://www.lugato.net/brigl/index.html

I think a WebGL renderer would be great to show off models on the web. No plugin, no downloads, etc.
It's very crude but it already show some models

Ideas and feedbacks are welcome

[Michael Heidemann]

Due to my low level grafik card I am unable to use WebGL at all. I fear there are many people out there with the same problem. But I am sure one day I am able to see your work

[Philippe Hurbain]

Looks rather nice for something "very crude"

[Tim Gould]

That is awesome.

Tim

[Steffen]

AWESOME! AWESOME!

But I ask myself why you need a server-side piece of software to achieve that.
The LDRAW syntax is so simple that JavaScript _itself_ could parse it and produce the data for the WebGL render...

An implementation like that is exactly what I had in mind when writing this:
http://forums.ldraw.org/showthread.php?t...63#pid1963

[Nicola]

Thank you
Just to clarify, there are no server-side software, the "conversion" program runs offline and produces a javascript file with all the model vertices etc. You can see it here. The js is included in the html and then everything run off the client.

The reason i decided not to do even the parsing in javascript is becouse, while the ldraw format is easy to read, it is quite tricky to work with. And also, if you implement the parsing in javascript, you must have some way to load part files ad needed, and that mean having a web server with all the library loaded (i don't know, maybe one exists?). Given that, probably it could work!

Now i want to try to integrate the program with a better 3d library such as Three.js

[Steffen]

Hmm, well, such webserver exists: ldraw.org ;-),
however, it probably would be more practical to keep a .zip with the latest parts update on YOUR server,
to avoid all the traffic to ldraw.org.
I really would suggest to implement parsing etc. in JavaScript, full on client side,
it is very easy to do, and as long as you have a .zip available with the latest parts update,
you can display any model a user likes.
You can download the .zip do the browser cache of the client.
And put an input control onto a page, so a user can browse for a model (*.ldr or *.mpd file)
to display OR specify the URL to it.
Then you would parse it by JavaScript, use the .zip library I just mentioned, and - voila - send it to WebGL.

[Tim Gould]

I disagree that parsing ldraw files is easy. It's actually quite annoying in many ways. Not a difficult problem, but with enough quirks to make it painful.

Tim

[Travis Cobbs]

Given that the current complete.zip is 19.1MB, I'm not sure that's the best way to work things. Every time it gets out of the browser cache, it would have to be reloaded in its entirety, plus any casual user that just looks at one file would have to wait for the 19.1MB download before anything would happen.

Having said that, while ldraw.org does in fact host all of the official (and unofficial) part files on its web server in a fashion that would allow automatic download, ldraw.org is also fairly slow, and getting the parts from there would probably be quite slow. Also, the last time I checked (which was admittedly years ago when implementing automatic download in LDView), gzip compression was wasn't enabled on ldraw.org, so downloading parts actually transfers all the bytes, even though part files compress extremely well, and gzip compression is standard in http 1.1.

[Nicola]

Yeah, there are some really weird stuff, such as the conditional lines, ccw vs cw faces, inversion of subpart, etc .. i spent quite a lot to get that right

[Nicola]

yeah, working with the whole package is not ok. my latest idea is this: upload all parts as single file in some server where you can query single parts. Then in javascript i scan an ldr/mpd and for any reference to a subpart, i query the server to obtain it, and use all the parts to build the full geometry.
This would allow displaying of arbitrary models (ie embeddable in html, from user textarea, etc) without any server side work, with hopefully minimal bandwidth usage. I've tested that my Viper model uses about 80 parts for about 300kB total, which can be compressed to about 30kB with gzip, quite good. With this performances is more easily to overload the video card than to use too much bandwidth
The only problem is that it would generate 80 asynchronous request to the server, which can be quite a lot. But parts file are cacheable so that should help.
I'll try asap

[Steffen]

nice idea. just some more thoughts to share:
you could in fact put all the parts to the server,
then have the client send a "list of needed parts" to it.
the server could put all the needed files into a *.mpd on the fly (which is trivial),
and then that result could get send gzip compressed to the client.

[Tim Gould]

Or store the parts pre-processed somewhere and read that. If you've got them on your own server you're not limited to LDraw File Format.

Tim

[Nicola]

Here the new version!
Now it's all javascript. It asynchronously fetch all part data from a server, build the model and display it. No server logic. The parts on the server are unprocessed, they're just redistribuited in different subfolder as my server was complaining about 7000+ files on a single dir

The main javascript object mantain a cache of already downloaded parts, so if you show multiple models the fetches are limited.

Here's a simple display:

http://www.lugato.net/ldraw/simple.html

And here's a page that lets you paste LDraw model in a textarea to load them:

http://www.lugato.net/ldraw/copypaste.html

And a page with multiple models in a gallery like fashion (caution: use lots of ram ):

http://www.lugato.net/ldraw/multiple.html

Note that my server doesn't GZip.
So there's much room for improvements, for example optimizing part requests, 3d rendering (some better filtering and lighting), zooming, compression of parts, etc.

Let me know if something go wrong while you try it. note that there may be some missing parts on the server.

[Willy Tschager]

http://www.lugato.net/ldraw/simple.html

"Something went wrong loading: 48\....dat"

and there a lots of these messages after loading the page. Afterwards it freezes saying "Initializing..."

w.

[Steffen]

I bow to you.
Seeing the loading progress text messages rush through,
and then seeing the model so beautifully is just amazing.
Great compliments! I am truly amazed! The beauty of this solution is
that it is NOT tied to either iOS or Android, but still can be seen on mobile devices, as long as the browser sports WebGL.
This embodies the MAIN principle of the Internet, offering contents INDEPENDENTLY of OS or browser brand.
Would we have to implement an Android App, an iOS App, an XYZ App separately, this would multiply the needed
effort. THIS way, all is bundled in ONE. YES, it's JavaScript, and JavaScript is not (to my eyes) the most beautiful
of all programming languages, but browsers nowadays are HIGHLY optimized for it, and you can always put something
ELSE to work together with it. However, for a browser, it's the most natural choice, being - in my eyes - MUCH better than
Flash or Silverlight, which again would be vendor-specific.
Good decision, and an impressive prototype. Kudos!

[Steffen]

the improvements which I spontaneously like to collect here, jumping to my eye or mind, are

(a) rotation center:
The center of rotation currently seems to be hardcoded 0/0/0.
I would better like to have it at the model's center, like in LDView.
Using the mouse currently in your web app feels so different than LDView,
that it confuses me. I would love to see a big similarity in mouse handling between the 2 tools.

(b) incremental parts display:
perhaps it is very easy to do, and you could show the model already while it is loading.
the user can watch the downloaded parts appear in the model.
I think it could be quite cheap to do, because you already know all the matrices etc, just the "real" part data is missing.
Thus, you could already build up the necessary data structures, and skip them on rendering, while the model is not fully loaded.
I think this would be an evern more "incremental" load than displaying the progress status message

© mousewheel:
I am missing the use of the mousewheel for zooming...

(d) default color 16:
Nicola, you seem to currently use 0/0/0 always for default color 16.
I would prefer if you would use the color specified explicitly in LDConfig.ldr (some "medium" default grey).
To you already use that file? You can always get the most recent version also from the ldraw.org site.

(e) edges:
you currently are not rendering edges at all.
I would like to have a checkbox or button for toggling that on and off.
It should be nearly trivial to add the drawing of them to your existing implementation.

[Steffen]

Nicola, I am again falling of my chair, impressed.
I just tested the attached file, which uses an LSYNTH synthesized cable,
plus unofficial parts, it loaded quickly and smooth, and looked beautiful!

[Steffen]

Code:

var flip = geometry._inverting ^ (det<0.0) ^ (!ccw); // kungfu

hahahaha, liked that

[Ken Drew]

NICE JOB!

[Philippe Hurbain]

I have the same problem as Willy. Could be an issue with / vs. \ in pathes?

[Nicola]

is most surely is. you're probably using firefox, right? chrome does slash substitution but firefox not.

it's fixed now, can you retry?

[Nicola]

the improvements which I spontaneously like to collect here, jumping to my eye or mind, are

(a) rotation center:
The center of rotation currently seems to be hardcoded 0/0/0.
I would better like to have it at the model's center, like in LDView.
Using the mouse currently in your web app feels so different than LDView,
that it confuses me. I would love to see a big similarity in mouse handling between the 2 tools.

Yes that's hardcoded now. I don't know if ldview calculates the center automatically or uses some metadata in the file. i have to make some tests.
About the mouse, i just thrown in a working system, it surely can be improved, and it needs zooming with the wheel

(b) incremental parts display:
perhaps it is very easy to do, and you could show the model already while it is loading.
the user can watch the downloaded parts appear in the model.
I think it could be quite cheap to do, because you already know all the matrices etc, just the "real" part data is missing.
Thus, you could already build up the necessary data structures, and skip them on rendering, while the model is not fully loaded.
I think this would be an evern more "incremental" load than displaying the progress status message

Yeah i considered it too.. Seeing the model progressively being built would be awesome. It might not be very hard but needs some modifications on the program.

(d) default color 16:
Nicola, you seem to currently use 0/0/0 always for default color 16.
I would prefer if you would use the color specified explicitly in LDConfig.ldr (some "medium" default grey).
To you already use that file? You can always get the most recent version also from the ldraw.org site.

I've yet to understand completely how color works, but i got that 16 is actually the "current color", and i correcly substitute it with what "parent lines" says, it's not hardcoded to 0. What i miss is the whole "complementary" colors and code 24.

(e) edges:
you currently are not rendering edges at all.
I would like to have a checkbox or button for toggling that on and off.
It should be nearly trivial to add the drawing of them to your existing implementation.

Normal edges would be trivial, conditional edges will be not They're one of the most puzzling quirks of LDraw file model. Basically you cannot just send them to OpenGL as a bunch of vectors, you have to manually test one by one and decide if you should draw them with a quite complex algorithm. It looks like some kind of hardcoded, precalculated "toon shading" like algorithm?
What i'd like to do is to use them to see if faces should be smooth or flat (basically what they tell is if the two faces should be "curved" or separated) so that i can render curved part nicely, but that require some work.

Nicola, I am again falling of my chair, impressed.
I just tested the attached file, which uses an LSYNTH synthesized cable,
plus unofficial parts, it loaded quickly and smooth, and looked beautiful!

That's not my merit, LSynth generates standard faces, i didn't do anything special for it

Code:

var flip = geometry._inverting ^ (det<0.0) ^ (!ccw); // kungfu

hahahaha, liked that

Lol, that code required quite a lot of trial and error Another quirk of LDraw is how it mixes CCW and CW faces and how it "inverts" subparts

[Philippe Hurbain]

More details: Works fine with Chrome, but not with Seamonkey (a Firefox variant).
Otherwise, really great!

[Philippe Hurbain]

Crossed posts... but no, it's doesn't work better now

[Philippe Hurbain]

I've yet to understand completely how color works, but i got that 16 is actually the "current color", and i correcly substitute it with what "parent lines" says, it's not hardcoded to 0. What i miss is the whole "complementary" colors and code 24

Indeed, color 16 "inherits" color of parent part. Problem is if there is no parent (eg. if you paste the code of a LDraw part in copypaste.html). This requires the definition of a "default color" to be used in that case.
Color 24 does the same, but for edge lines.

It looks like some kind of hardcoded, precalculated "toon shading" like algorithm?

I think it can be described this way...

What i'd like to do is to use them to see if faces should be smooth or flat (basically what they tell is if the two faces should be "curved" or separated) so that i can render curved part nicely, but that require some work.

That's what LDview does.

[Willy Tschager]

Working now! Just loaded a MPD containing an unofficial. Jaw-dropping!

w.

[Philippe Hurbain]

Still nothing here. Could a cache issue be possible?

[Nicola]

Still nothing here. Could a cache issue be possible?

Maybe yes, did you tried cleaning your browser cache?

[Philippe Hurbain]

No I didn't - and didn't need so! It now works fine with Seamonkey...
Really great work

[Willy Tschager]

This is working out all fine which leads us to the ultimate question:

How do we implement such a renderer into the forum or the CMS once it if fully featured?

w.

[Nicola]

Well it's just importing the .js files and glueing some html and php for the forum.
The problem is making the parts avaiable on the server. They must be on the same server since they're fetched by ajax, or in a server that implement the Access-Control-Allow-Origin directive.

[Steffen]

I've yet to understand completely how color works, but i got that 16 is actually the "current color",
and i correcly substitute >it with what "parent lines" says, it's not hardcoded to 0. What i miss is the whole "complementary" colors and code 24.

As Philo already answered, colors 16 and 24 are for "use parent color".
I just asked to correct the implementation of "what to do when there's no parent".
For example, I put some unofficial part from the parts tracker into your tool.
Then it gets all black, because you map color 16 to "parent color", and when there is no parent,
then use 0/0/0. My only, very simple, request was, as color 16 for this purpose has a R/G/B value
set in ldconfig.ldr, that you use _that_ color from there when there's no parent, instead of 0/0/0

Normal edges would be trivial, conditional edges will be not

I know, I was asking just for the simple, normal edges.

[Steffen]

maybe we CAN allow to host the parts on the ldraw.org server,
BUT only activate your renderer for logged-in forum participants.
this way we can limit bandwidth, as we're not publicly offering the parts download to everybody,
and at the same time get the nice feature of 3D viewing arbitrary LDRAW files in realtime in the forums.

[Steffen]

I was able to run this in Iron/Chrome, but not in Firefox.

Searching for the cause, I found this information:

http://www.sitepoint.com/firefox-enable-...hics-card/

"Your graphics card may be WebGL-compatible, but it’ll be blocked in Firefox if the driver version is 0.0.0.1 behind the approved list."

I applied the 3 changes mentioned there:
- To enable WebGL, set webgl.force-enabled to true
- To enable Layers Acceleration, set layers.acceleration.force-enabled to true
- To enable Direct2D in Windows Vista/7, set gfx.direct2d.force-enabled to true

, and voila, I also had WebGL in Firefox without problems.

Maybe it's the same for SeaMonkey.

(My graphics card is just a normal ATI Radeon HD 4600, which bought because it is passively cooled and thus silent.)

[Nicola]

as color 16 for this purpose has a R/G/B value
set in ldconfig.ldr, that you use _that_ color from there when there's no parent, instead of 0/0/0

Oh ok, didn't noticed this Implemented!

I've uploaded the new version, it includes:
- zooming and panning with mouse wheel
- mesh centering as LDview (and vertices merging for some performaces)
- very basic line handling
- experimental step support

see here for all links

line rendering is very limited becouse: 1) Three.js doesn't support different materials for same line geometry, so only black edges for now, and 2) line width is fixed to one pixel becouse the parameter is ignored (see here, looks like a webgl gap). In linux it should look better.

Let me know of any bug

[Philippe Hurbain]

Wow.
Wow, wow!!!
Amazing!

- experimental step support

One thing missing (but then - it's experimental!): orientation/scale should be kept when changing step.

[Steffen]

Let me know of any bug

I think that the computation of the part center for rotation is not correct.
The topright red shuttle on page http://www.lugato.net/ldraw/multiple.html
does not rotate about its center. It rotates around "some other point".
EDIT: probably I am wrong here

[Steffen]

yes, I agree. when going a step forward or backward, the perspective that the user has established with the mouse
should not be modified.

[Michael Heidemann]

On my Laptop I just needed to update to the latest driver

Really great to see that working.

But I run into an error by processing the attached file. I know it's still beta or alpha, but just at that point I am sure every feedback will help you

[Nicola]

But I run into an error by processing the attached file. I know it's still beta or alpha, but just at that point I am sure every feedback will help you

Uhm it looks like the problem is that there are two submodels referenced by a previous model. My parser assumes MPD contains models sorted by dependency, with the last one being indepentent, the previous one depending on the last and so on. This helps parsing a little, and since all mpd i tryed worked this way i assumed this was standard. Maybe i'm wrong

Saying that, i've tryed to move 8464 - Cylinders1.ldr and 8464 - Cylinders2.ldr to the bottom but it still doesn't work. No error reported but no model too.. Maybe is too big ?

Edit: maybe it's a strange color i can't parse

[Travis Cobbs]

Actually, the MPD spec requires that the very first file in the MPD be the main model, so it is guaranteed that all correctly formatted MPD files will have dependent models show up later in the file. (It could be that you meant all the other models in the file show up in order, but that's not required by the MPD spec.)

[Nicola]

Hi, i've updated the library. See the results here:

http://www.lugato.net/brigl/index.html

News are:
- Better step support (not ideal yet)
- Better handling of non-BFC certified parts
- More optimized vertex merging
- Added smoothing of faces based on Conditional Lines (man if this was hard)
- Added specular light component
- Various fixings

I'm also setting up a public repository (with instructions and sources) so i can stop spamming here.
As always, feedbacks are welcome

[Philippe Hurbain]

Even greater... Smoothing is definitely worth the headhache you got

I'm also setting up a public repository (with instructions and sources) so i can stop spamming here.

That's not spamming! please keep us informed!

[Travis Cobbs]

- Added smoothing of faces based on Conditional Lines (man if this was hard)

I totally agree that is hard (having done it in LDView). Looking at your samples, though, it looks like you did a great job; the results look awesome.

[Nicola]

Hello, i've released a new version (version 4). This release add support for colored lines and a simple form of animations. I've added animations becouse i'd like to showcase moving parts of a model. It is based on code from TWEEN.js
You can find all here as usual:

http://www.lugato.net/brigl/

The repository i set up is here:

https://bitbucket.org/msx80/brigl

That's all

[Philippe Hurbain]

The animations are really cute, especially the bounce at the end!
But "lines" example does no longer work here (hangs up at "generating geometry") (browser=Seamonkey 2.14.1, tell me if you need other details on my machine).

[Nicola]

lol i tested stuff for ages, i don't know how i missed that one.. It's correct now

[Philippe Hurbain]

It's correct now

Confirmed!

[Steffen]

this is so ULTIMATELY cool...