Automatic image creation for training AI

[Murray]

Hello all.

I have been wondering if there is a way to create a collection of images of a part from different angles automatically.
The only comparison I can think of is to use a Photoshop action to rotate the image, save a copy, repeat that a few times, close the part and onto the next.

I want to train a neutral network to identify Lego bricks on a conveyor, and sort them.  The standard way is to take photos from a few angles and label them. 

image borrowed from here

That could take forever with the dataset I want to create, even using a handful of camera taking photos at the same time.

Does anyone know if this is possible with Ldraw?
I saw that you can go to a predetermined viewpoint and take a screenshot, seems like a good start?

I know a virtual image set won't be as accurate as taking sample photos of real bricks in situ, but I can add real photos to the dataset latter for bad matchups.

Thanks for any recommendations ?

[Gerald Lasser]

Here: https://forums.ldraw.org/thread-23780.html

is a link to Brothers Brick where someone did use the LDraw parts Library to generate images to train the AI.


A recommendation could be to generate images with LDView from the command line having random lon/lat entered and rendered.

gerald

[Murray]

Here: https://forums.ldraw.org/thread-23780.html

is a link to Brothers Brick where someone did use the LDraw parts Library to generate images to train the AI.


A recommendation could be to generate images with LDView from the command line having random lon/lat entered and rendered.

gerald

Thanks Gerald.

Unfortunately, he doesn't seem to describe how he used the Ldraw parts library.
It seems he may have used it to create an AI cross reference to classify his own images manually?

Great reading though.  

I will look at the command line option as well,. Thanks

[Jose Alfonso]

I will look at the command line option as well,. Thanks

Look at this example of command line with LDView:

LDVIEW_PATH/LDView.exe LDRAWBRICK.dat -SaveWidth=160 -SaveHeight=120 -DefaultLatLong=45,30 -SaveSnapshot=IMAGE.png -AutoCrop=1 -SaveZoomToFit=1 -SaveAlpha=1 -LineSmoothing=1 -DefaultColor3=0xBFBFBF -FOV=0.1 -LightVector=-0.125,0.125,1

For example, "SaveZoonToFit=1" zooms until the brick touches one of the borders of the image. I think this is good for AI training, at least when labelling for object detection. "AutoCrop=1" crops the image to the brick, so the result can be 130x120 pixels or 160x100 instead of the maximum 160x120. Maybe you should remove that option. With DefaultLatLong you change the angle, and with DefaultColor3 you set the color of the brick.

Other parameters are read from the configuration of LDView. For your task you should unmark "Edge lines" in Geometry tab to avoid artificial edges.

[Murray]

Look at this example of command line with LDView:

LDVIEW_PATH/LDView.exe LDRAWBRICK.dat -SaveWidth=160 -SaveHeight=120 -DefaultLatLong=45,30 -SaveSnapshot=IMAGE.png -AutoCrop=1 -SaveZoomToFit=1 -SaveAlpha=1 -LineSmoothing=1 -DefaultColor3=0xBFBFBF -FOV=0.1 -LightVector=-0.125,0.125,1

For example, "SaveZoonToFit=1" zooms until the brick touches one of the borders of the image. I think this is good for AI training, at least when labelling for object detection. "AutoCrop=1" crops the image to the brick, so the result can be 130x120 pixels or 160x100 instead of the maximum 160x120. Maybe you should remove that option. With DefaultLatLong you change the angle, and with DefaultColor3 you set the color of the brick.

Other parameters are read from the configuration of LDView. For your task you should unmark "Edge lines" in Geometry tab to avoid artificial edges.

Thanks Jose, that's exactly what I'm looking for ?

[Murray]

Look at this example of command line with LDView:

LDVIEW_PATH/LDView.exe LDRAWBRICK.dat -SaveWidth=160 -SaveHeight=120 -DefaultLatLong=45,30 -SaveSnapshot=IMAGE.png -AutoCrop=1 -SaveZoomToFit=1 -SaveAlpha=1 -LineSmoothing=1 -DefaultColor3=0xBFBFBF -FOV=0.1 -LightVector=-0.125,0.125,1

For example, "SaveZoonToFit=1" zooms until the brick touches one of the borders of the image. I think this is good for AI training, at least when labelling for object detection. "AutoCrop=1" crops the image to the brick, so the result can be 130x120 pixels or 160x100 instead of the maximum 160x120. Maybe you should remove that option. With DefaultLatLong you change the angle, and with DefaultColor3 you set the color of the brick.

Other parameters are read from the configuration of LDView. For your task you should unmark "Edge lines" in Geometry tab to avoid artificial edges.

I had a quick look at getting this working (which I couldn't quite figure out yet,  command line as in command prompt?)  but I noticed that the parts don't have names like 1x1 plate,  is that in another database somewhere for the names?
I was using the complete parts library that I downloaded not a local one installed with LDraw\Lecad\MLcad mostly because I couldn't find them.

[Willy Tschager]

I had a quick look at getting this working (which I couldn't quite figure out yet,  command line as in command prompt?)

Have a look at this: http://www.holly-wood.it/ldview-en.html

w.

[Johannes Ahlmann]

I am currently doing this using Blender and Cycles path tracing renderer.

Happy to help you out if you are still looking for some input/ thoughts.

You can see some (pathologic) examples in this thread:
https://forums.ldraw.org/thread-24780.html

Regards,
Johannes

[Murray]

I am currently doing this using Blender and Cycles path tracing renderer.

Happy to help you out if you are still looking for some input/ thoughts.

You can see some (pathologic) examples in this thread:
https://forums.ldraw.org/thread-24780.html

Regards,
Johannes

Wow, those renders look real!
I'm trying to get my camera setup working for this project, but once I get that sorted I will definitely ask how you got did that.

Thanks Johannes

[Orion Pobursky]

I am currently doing this using Blender and Cycles

If you're Blender/Cycles savvy, Roland is currently looking for a collaborator with the LDCad Blender export.