OMR Model Render

[Orion Pobursky]

Now that I've implemented Lasse's WebGL renderer on the PT, what is the feeling of expanding it to the OMR? Is anyone particularly attached to brigl? Why?

[Lasse Deleuran]

There is one huge advantages with brigl: It is able to render with light sources and smooth surfaces. This does wonders for realism, as opposed to the building instructions-focused cell shading. I personally find this important for proper presentation of LEGO models. This is in the pipeline for buildinginstructions.js. However. I do not know when this feature will be added as I only have 2-3 hours each evening to work on projects.

While there have been a lot of performance improvements in buildinginstructions.js, I still have to measure it against the renderer on OMR (and to be fair, also Mecabricks): Both loading time, rendering time and storage should be compared.

The optional features buildinginstructions.js brings to the table are:

• Logos on studs
  
• Optional lines (line type "5": Those around the cylinders of studs and other rounded surfaces)
  
• Use IndexedDB to cache parts for quicker loading
  
• In-browser Generated simple primitives for even quicker loading (stud.dat, 4-4edge.dat, etc.)
  

New features can of course be added - both pull requests, GitHub issues and direct messages are accepted

[Travis Cobbs]

For what it's worth, I agree the lit rendering is better on the OMR.

[Lasse Deleuran]

I am currently working on 3 improvements for the buildinginstructions.js project. One of these is more realistic rendering. 


As you can see, I have a lot of work ahead of me. Currently there are 3 tasks which have to be finished:

1) Proper computation of normals. In the picture above I have used the built-in normal calculation. There is another post on this forum where we discuss normals, and I will be implementing a custom normal calculation which ensures hard edges where we have lines of type 2.

2) Rendering of special materials. I am currently pretty happy with how rubber is rendered. However. Having proper rendering of chrome and metal will require both a proper environment map, and other textures. My hope is that I will end up with some really neat looking parts, but this is unfortunately a new area for me.

3) Reflections, shadows and floor should be added. There are tutorials showing how to do this with three.js and I just have to find the time to do this right... unless someone else want to take it on and create some pull requests 

[Orion Pobursky]

Awesome,

Thanks for all the work you have done and continue to do. I look forward to the results.

[Lasse Deleuran]

I'm the one who thanks for all the great parts and tools! This is simply my way of giving a little back.

I think I have the edges under control now:


The next step will be surfaces - they have to be roughed up.

[Lasse Deleuran]

An update!



You can see some artifacts on the flat yellow parts. It is by modifying these artifacts and adding "damage" to the parts that I expect to get the rendering to become more realistic. Also. I will add a pattern to the ground, different lights, and other improvements. Slow and steady


EDIT. A closeup showing the level of detail up close:

[Roland Melkert]

An update!

Starting to look very nice.

Did you make those bevels using a shader or did you manipulate the meshes before rendering?

[Lasse Deleuran]

Starting to look very nice.

Did you make those bevels using a shader or did you manipulate the meshes before rendering?

Thanks! 

With these tenderers I am using out-of-the-box shaders as I am not too worried about performance (yet)

The smooth shading around studs, curved slopes and such is created by some heuristics I have come up with for splitting normals. It is not yet perfect, as can be seen on the macaroni brick in the lower front corner. 

The smoothened corners is done by generating textures for normal maps on the fly and again using some less-than-perfect heuristics for UV calculation. 

I will will post a write up with illustrations in the “adventures of creating a web renderer” topic once I have some more progress on the environment mapping, ground texture and lights. 

But I have to say. This is quite fun. And much more technically than I feared.

[Lasse Deleuran]

I have now added materials for the various types of colors, and it is time to test it out on brickhub.org to find errors before suggesting to use it in OMR.

It works fine for some models:



(Hint: The keys "QWEASD" can be used to move the lights - I am still working on proper scene controls)

While other models show that there are issues with some parts:



I believe the issue here is due to my custom UV generation. 

Furthermore. I would like to use normals to generate stud logos in order to preserve loading and rendering performance. As you can see, I have turned stud logos off for now since it has a major impact (hundreds of triangles per stud).

[Lasse Deleuran]

That should do it. I have made a new UV calculation heuristic, and it seems to compute good surface coordinates for all of the test elements I have tested it on.

The code has been pushed to the master branch of buildinginstructions.js and the file "sample_physical.php" shows how to load a model using this renderer.

What do you think? Is it alright for the OMR page?

[Orion Pobursky]

That should do it. I have made a new UV calculation heuristic, and it seems to compute good surface coordinates for all of the test elements I have tested it on.

The code has been pushed to the master branch of buildinginstructions.js and the file "sample_physical.php" shows how to load a model using this renderer.

What do you think? Is it alright for the OMR page

Here's a link for others:
sample_physical.htm

The rendering looks good but the lighting seems a bit dark for my taste. Someone else want to chime in?

[Lasse Deleuran]

You can press "Right arrow" to go to light control, move it around with "WADSQE" and add an additional light using "P" (then click right to select it - the red diamond will indicate it is selected) and move it around. The console will show the position of the lights. Once a good position is found, you can hardcode it.

I am currently looking into adding a "sun"-like light instead of just point lights.

[Orion Pobursky]

You can press "Right arrow" to go to light control, move it around with "WADSQE" and add an additional light using "P" (then click right to select it - the red diamond will indicate it is selected) and move it around. The console will show the position of the lights. Once a good position is found, you can hardcode it.

I am currently looking into adding a "sun"-like light instead of just point lights.

I'd definitely like something a little more diffuse.

[Lasse Deleuran]

I'd definitely like something a little more diffuse.

Yeah, and softer shadows, right? I think they are way too hard right now.

BTW. The code for the scene set up is found in Scene.js: Scene.js



There is also ambient light.

Right now the ambient light is only set to 0.45 out of 1: this.setHemisphereLight(0xF4F4FB, 0x30302B, 0.45);

It can just be increased.

[Orion Pobursky]

I'd definitely like something a little more diffuse.

Also, the rendering is good in my view. At least good enough for our purposes. That said, I'm still getting weird lighting effects on curved surfaces as seen on the top of the test model:
   

[Lasse Deleuran]

That's interesting. I see this artefact on both 2x4 and 1x3 curved slopes, but not on the 1x2 ones.

That will be next on my list after the light and shadow setup.

[Orion Pobursky]

That should do it. I have made a new UV calculation heuristic, and it seems to compute good surface coordinates for all of the test elements I have tested it on.

The code has been pushed to the master branch of buildinginstructions.js and the file "sample_physical.php" shows how to load a model using this renderer.

What do you think? Is it alright for the OMR page?

Here's some more weird reflections:
   

I'd like to reemphasize that my comments are just nits and not complaints. I'm definitely going to be looking into replacing brigl with your code.

[Lasse Deleuran]

Here's some more weird reflections:


I'd like to reemphasize that my comments are just nits and not complaints. I'm definitely going to be looking into replacing brigl with your code.

Nitpicking is exactly what I need to make this good, so keep it up!

I have just pushed a commit to master where ambient light has been increased, an additional point light has been added and the scene is nice and light now.

Also. I have decreased the imperfections shown on "ABS material" colors, such as yellow. This decreased the amount of waves you would see in those reflections. The imperfections have been added on purpose - they allow for the light to be reflected irregularly which looks more realistic at some angles. However. I would also love to see other and even better ways of generating these material surfaces. Right now they are simple randomly generated cosine curves with some additional randomness and imperfections thrown in.

[Lasse Deleuran]

Also, the rendering is good in my view. At least good enough for our purposes. That said, I'm still getting weird lighting effects on curved surfaces as seen on the top of the test model:

I found the issue. I am making the edges of generated textures curved (so that normal bricks have rounded corners when you zoom in), but the UV generation on curved surfaces was hitting these curved corners, giving the artefacts. A new version has been pushed without this issue.

[Lasse Deleuran]

Soft shadows and directional lights have been added to the source. The default scene is now nicely lit.

There are still some complex parts which cause issues with UV generation causing black spots. Those I will look into next.

[Orion Pobursky]

Soft shadows and directional lights have been added to the source. The default scene is now nicely lit.

There are still some complex parts which cause issues with UV generation causing black spots. Those I will look into next.

Much better in my view. Black still looks too black but not disturbingly so.

[Lasse Deleuran]

Much better in my view. Black still looks too black but not disturbingly so.

It might not just be a darkness issue, but rather an issue of clearly showing outlines.

I have added the following piece of code (almost similar to changing ldconfig.ldr):

Code:

if(colorID === 0) {
            params.color = 0x1B1B1B;
            params.roughness = 0;
        }

Giving the following result:


I have previously moved the lights around to achieve proper reflections and nuances in the black surfaces, but we cannot expect users to do so.
I had hoped the directional light would help here, but is works poorly on curved surfaces.

This is what I can achieve by adding lights to highlight the black parts:

[Orion Pobursky]

Don't get too obsessed with this. Black is hard.

[Lasse Deleuran]

Don't get too obsessed with this. Black is hard.

Apparently so!
Transparent parts, and illuminating parts are also a bit tricky. I have done my best with them for now.

That said. I also have the UV issue. Right now I see one of the quads on 3820.dat fails. I have downloaded the subb files and will see if I can debug it in the bus to work tomorrow.

Meanwhile. I have uploaded the new renderer to Brickhub.org, so it runs when you click "view 3D" on any of the models. I hope that will also lead to some bug reports so everything can be smoothened out and be made really good. We have a fierce and beloved competitor when it comes to online rendering.

[Travis Cobbs]

Here's a link for others:
sample_physical.htm

The rendering looks good but the lighting seems a bit dark for my taste. Someone else want to chime in?

It's not a big deal, but given that the surfaces are parallel, I would expect the reflection to line up here:


It kind of looks like both parts have the same amount of travel in one dimension, but the 1x3 has much less travel in the other dimension, leading to both the discontinuity in the specular between the two parts, and the much more pronounced slope of the reflection in the 1x3 part.

[Lasse Deleuran]

The latest push of code fixes all the issues I have observed so far with UV generation (and resulting black spots). The code is not yet pushed to brickhub.org, but it is in the github.org master branch.


On the current TODO list:

- The issue with black (No good solution yet)
- Illuminating materials (Can be done by adding rendering passes, but performance will be affected)
- Transparent materials using ray tracing for volume detection so they appear more correctly (Can be done by adding ray-tracing pass, but performance will be affected)
- Logos on studs created using normals so the rendering performance stays acceptable
- General performance improvements by using less advanced materials
- Distance blur to recreate DSLR lens properties (Can be done by adding rendering passes, but performance will be affected)

Any other improvement ideas?

Which of these do you feel are required before the code base can be used for OMR?

[Orion Pobursky]

Which of these do you feel are required before the code base can be used for OMR?

I’m already working on code for the OMR. Didn’t have much time last night to work on it.

[Orion Pobursky]

I’m already working on code for the OMR. Didn’t have much time last night to work on it.

Super slow right now but it is pulling data correctly. Now to work on integration with the OMR software and customizing the scene setting.

https://omr.ldraw.org/test.htm

[Orion Pobursky]

Super slow right now but it is pulling data correctly. Now to work on integration with the OMR software and customizing the scene setting.

https://omr.ldraw.org/test.htm

And I have it successfully integrated but had to revert due to a variable scoping problem. Hopefully will have everything up tonight.

[Lasse Deleuran]

That was quick!

It loads fairly quickly on my 10 year old dev box. The skyline (heh) can be made brightened by setting the color of the renderer:

scene.background = new THREE.Color(0x6D6D6D);

Testing on models with stickers show that debth testing can fail, so the camera parameters (far, near, FOV, etc.) might have to be tuned so they work on all models.

[Orion Pobursky]

That was quick!

It loads fairly quickly on my 10 year old dev box. The skyline (heh) can be made brightened by setting the color of the renderer:

scene.background = new THREE.Color(0x6D6D6D);

Testing on models with stickers show that debth testing can fail, so the camera parameters (far, near, FOV, etc.) might have to be tuned so they work on all models.

Once I iron out the kinks, we'll be able to test on the entire omr catalog and tweak as needed.

[Lasse Deleuran]

It's not a big deal, but given that the surfaces are parallel, I would expect the reflection to line up here:


It kind of looks like both parts have the same amount of travel in one dimension, but the 1x3 has much less travel in the other dimension, leading to both the discontinuity in the specular between the two parts, and the much more pronounced slope of the reflection in the 1x3 part.

Is this a reflection from a point light or a directional light? A directional light can explain it since it does not pay attention to the parts in comparison to the light source.

On the other hand. You can also see similar effects with point lights since I reuse textures on the parts (it is expensive to generate textures, so reusing them is good for performance).

My best guess is that you are seeing a directional light here. You can press LEFT/RIGHT to toggle between the lights and use DELETE to remove them. Button controls are on their way, but I am focusing on setting up the renderer first as Orion is progressing much faster than I dared hope.

[Orion Pobursky]

We're in business.

Here's a sample model page (with brigl):
http://omr.ldraw.org/files/1182

To see a model rendered with LDBI simply add "_ldbi" after "files":
http://omr.ldraw.org/files_ldbi/1182

Note: this a preview. There are quite a few tweaks to be made

[Magnus Forsberg]

Note: this a preview. There are quite a few tweaks to be made

Doesn't handle texmap images.
http://omr.ldraw.org/files_ldbi/401

Or is it that one of the tweaks?

[Orion Pobursky]

Doesn't handle texmap images.
http://omr.ldraw.org/files_ldbi/401

Or is it that one of the tweaks?

No. Texmap support is a known limitation.
Support TEXMAP language extension #13

But neither does brigl.

[Orion Pobursky]

No. Texmap support is a known limitation.
Support TEXMAP language extension #13

But neither does brigl.

I did, however find a bug that prevented the display of that model. This is now fixed.

[Orion Pobursky]

We're in business.

Here's a sample model page (with brigl):
http://omr.ldraw.org/files/1182

To see a model rendered with LDBI simply add "_ldbi" after "files":
http://omr.ldraw.org/files_ldbi/1182

Note: this a preview. There are quite a few tweaks to be made

So one clear advantage of LDBI is that I can load huge models on my iPhone. Brigl crashes when I load the UCS Imperial Star Destroyer and LDBI does not.

[Orion Pobursky]

So one clear advantage of LDBI is that I can load huge models on my iPhone. Brigl crashes when I load the UCS Imperial Star Destroyer and LDBI does not.

The UCS Falcon crashes the tab on my desktop when brigl is used. With LDBI, time to render is reasonable and it doesn't crash the tab.

http://omr.ldraw.org/files_ldbi/347

[Philippe Hurbain]

The UCS Falcon crashes the tab on my desktop when brigl is used. With LDBI, time to render is reasonable and it doesn't crash the tab.

http://omr.ldraw.org/files_ldbi/347

Yes, works great, especially considering the size of the model. As it nonetheless takes some time to load (about 30 seconds here), would it be possible to get some form of progress bar?
Otherwise, got a display problem, the feet of the Falcon sometimes disappear:
   

[Lasse Deleuran]

Yes, works great, especially considering the size of the model. As it nonetheless takes some time to load (about 30 seconds here), would it be possible to get some form of progress bar?
Otherwise, got a display problem, the feet of the Falcon sometimes disappear:

A progress bar would be a great idea, but my attempts to create one have failed as it has decreased the loading speed significantly. I will have to check StackOverflow for some pointers.

The 'Feet' issue might be due to the debth buffer issue mentioned earlier. I have also observed it with other large models which are not on wheels.

How is the general feeling of prioritization? Right now my list is:

1) Fix rendering issues (this includes what you mention here, but also the UV calculation as you can see there are still many warnings generated in the log)
2) Stud logos (while studs can easily be enabled by setting a parameter, my experience is that the file size doubles and rendering time doubles as well. I am working on a light-weight alternative)
3) Control buttons to set up camera, lights, etc.
4) Progress bar
5) Better support of transparency.
6) Support of illumination parts (such as glow-in-the-dark ghosts, etc.)
7) Distance-based blur for a more realistic rendering
8) Textures
9) Geometry culling (such as removing studs that cannot be seen)


And of course, everyone is welcome to take up a task and create pull requests - it is all open source / license free

[Magnus Forsberg]

I did, however find a bug that prevented the display of that model. This is now fixed.

Yes, sorry about that. I was unclear.
brigl is loading the model, and showing the fallback code.
ldbi refused to even begin working. Nothing happened when I clicked on the button.

Now, your bugfix did change something. It looks like something is starting, but the model is not loading. Not on Chrome, IE 11 or Firefox.

[Lasse Deleuran]

Yes, sorry about that. I was unclear.
brigl is loading the model, and showing the fallback code.
ldbi refused to even begin working. Nothing happened when I clicked on the button.

Now, your bugfix did change something. It looks like something is starting, but the model is not loading. Not on Chrome, IE 11 or Firefox.

From the debugger I can see that the files are failing to load:

http://omr.ldraw.org/files_ldbi/textures/cube/px.jpg etc.

are environment map textures which are in the github project. They are in public domain, so can be copied without issues.

http://omr.ldraw.org/media/LDraw%20model...0-%20Azari&

Seems to be the model file having some hyphen or other special character issues resulting in the file not being loaded.

[Orion Pobursky]

Yes, sorry about that. I was unclear.
brigl is loading the model, and showing the fallback code.
ldbi refused to even begin working. Nothing happened when I clicked on the button.

Now, your bugfix did change something. It looks like something is starting, but the model is not loading. Not on Chrome, IE 11 or Firefox.

Turns out between the time I posted about the bugfix and the time you accessed the page I introduced another bug. This is now fixed.

[Orion Pobursky]

From the debugger I can see that the files are failing to load:

http://omr.ldraw.org/files_ldbi/textures/cube/px.jpg etc.

are environment map textures which are in the github project. They are in public domain, so can be copied without issues.

This is a different problem that I'll PM you about.

[Orion Pobursky]

This is a different problem that I'll PM you about.

Scratch that. This appears to be a problem on the server end since this doesn't happen in my test page on the cms.

[Magnus Forsberg]

I find the Metallic colours very strange.

   

This is a Metallic Gold print.  http://omr.ldraw.org/files_ldbi/1182



And the Chrome colours is also off.


  Image 2.png (Size: 83.4 KB / Downloads: 169)

This is Chrome Silver, looking very black. http://omr.ldraw.org/files_ldbi/640

[Lasse Deleuran]

I find the Metallic colours very strange.



This is a Metallic Gold print.  http://omr.ldraw.org/files_ldbi/1182



And the Chrome colours is also off.



This is Chrome Silver, looking very black. http://omr.ldraw.org/files_ldbi/640

That is due to the loading issue with environment map.  

See the model on Brickhub here: https://brickhub.org/i/386

And it looks like this:

[Orion Pobursky]

I find the Metallic colours very strange.



This is a Metallic Gold print.  http://omr.ldraw.org/files_ldbi/1182



And the Chrome colours is also off.



This is Chrome Silver, looking very black. http://omr.ldraw.org/files_ldbi/640

This is fixed.

[Lasse Deleuran]

This is fixed.

I must say. Metal surfaces look amazing with this version of three.js and WebGL!

Imagine getting transparent parts to look just as real. We might end up with a viable real-time renderer!