Automatic image creation for training AI

Automatic image creation for training AI - Printable Version

+- LDraw.org Discussion Forums (https://forums.ldraw.org)
+-- Forum: General (https://forums.ldraw.org/forum-12.html)
+--- Forum: General LDraw.org Discussion (https://forums.ldraw.org/forum-6.html)
+--- Thread: Automatic image creation for training AI (/thread-24721.html)

Automatic image creation for training AI - Murray - 2021-07-05

Hello all.

I have been wondering if there is a way to create a collection of images of a part from different angles automatically.
The only comparison I can think of is to use a Photoshop action to rotate the image, save a copy, repeat that a few times, close the part and onto the next.

I want to train a neutral network to identify Lego bricks on a conveyor, and sort them. The standard way is to take photos from a few angles and label them.
[Image: lego-sorter-fe.jpg?w=800]

image borrowed from here

That could take forever with the dataset I want to create, even using a handful of camera taking photos at the same time.

Does anyone know if this is possible with Ldraw?
I saw that you can go to a predetermined viewpoint and take a screenshot, seems like a good start?

I know a virtual image set won't be as accurate as taking sample photos of real bricks in situ, but I can add real photos to the dataset latter for bad matchups.

Thanks for any recommendations ?

RE: Automatic image creation for training AI - Gerald Lasser - 2021-07-05

Here: https://forums.ldraw.org/thread-23780.html

is a link to Brothers Brick where someone did use the LDraw parts Library to generate images to train the AI.

A recommendation could be to generate images with LDView from the command line having random lon/lat entered and rendered.

gerald

RE: Automatic image creation for training AI - Murray - 2021-07-05

(2021-07-05, 13:50)Gerald Lasser Wrote: Here: https://forums.ldraw.org/thread-23780.html

is a link to Brothers Brick where someone did use the LDraw parts Library to generate images to train the AI.

A recommendation could be to generate images with LDView from the command line having random lon/lat entered and rendered.

gerald

Thanks Gerald.

Unfortunately, he doesn't seem to describe how he used the Ldraw parts library.
It seems he may have used it to create an AI cross reference to classify his own images manually?

Great reading though.

I will look at the command line option as well,. Thanks

RE: Automatic image creation for training AI - Jose Alfonso - 2021-07-05

(2021-07-05, 14:47)Murray Wrote: I will look at the command line option as well,. Thanks

Look at this example of command line with LDView:

LDVIEW_PATH/LDView.exe LDRAWBRICK.dat -SaveWidth=160 -SaveHeight=120 -DefaultLatLong=45,30 -SaveSnapshot=IMAGE.png -AutoCrop=1 -SaveZoomToFit=1 -SaveAlpha=1 -LineSmoothing=1 -DefaultColor3=0xBFBFBF -FOV=0.1 -LightVector=-0.125,0.125,1

For example, "SaveZoonToFit=1" zooms until the brick touches one of the borders of the image. I think this is good for AI training, at least when labelling for object detection. "AutoCrop=1" crops the image to the brick, so the result can be 130x120 pixels or 160x100 instead of the maximum 160x120. Maybe you should remove that option. With DefaultLatLong you change the angle, and with DefaultColor3 you set the color of the brick.

Other parameters are read from the configuration of LDView. For your task you should unmark "Edge lines" in Geometry tab to avoid artificial edges.

RE: Automatic image creation for training AI - Murray - 2021-07-06

(2021-07-05, 17:01)Jose Alfonso Wrote: Look at this example of command line with LDView:

LDVIEW_PATH/LDView.exe LDRAWBRICK.dat -SaveWidth=160 -SaveHeight=120 -DefaultLatLong=45,30 -SaveSnapshot=IMAGE.png -AutoCrop=1 -SaveZoomToFit=1 -SaveAlpha=1 -LineSmoothing=1 -DefaultColor3=0xBFBFBF -FOV=0.1 -LightVector=-0.125,0.125,1

For example, "SaveZoonToFit=1" zooms until the brick touches one of the borders of the image. I think this is good for AI training, at least when labelling for object detection. "AutoCrop=1" crops the image to the brick, so the result can be 130x120 pixels or 160x100 instead of the maximum 160x120. Maybe you should remove that option. With DefaultLatLong you change the angle, and with DefaultColor3 you set the color of the brick.

Other parameters are read from the configuration of LDView. For your task you should unmark "Edge lines" in Geometry tab to avoid artificial edges.

Thanks Jose, that's exactly what I'm looking for ?

RE: Automatic image creation for training AI - Murray - 2021-07-06

(2021-07-05, 17:01)Jose Alfonso Wrote: Look at this example of command line with LDView:

LDVIEW_PATH/LDView.exe LDRAWBRICK.dat -SaveWidth=160 -SaveHeight=120 -DefaultLatLong=45,30 -SaveSnapshot=IMAGE.png -AutoCrop=1 -SaveZoomToFit=1 -SaveAlpha=1 -LineSmoothing=1 -DefaultColor3=0xBFBFBF -FOV=0.1 -LightVector=-0.125,0.125,1

For example, "SaveZoonToFit=1" zooms until the brick touches one of the borders of the image. I think this is good for AI training, at least when labelling for object detection. "AutoCrop=1" crops the image to the brick, so the result can be 130x120 pixels or 160x100 instead of the maximum 160x120. Maybe you should remove that option. With DefaultLatLong you change the angle, and with DefaultColor3 you set the color of the brick.

Other parameters are read from the configuration of LDView. For your task you should unmark "Edge lines" in Geometry tab to avoid artificial edges.

I had a quick look at getting this working (which I couldn't quite figure out yet, command line as in command prompt?) but I noticed that the parts don't have names like 1x1 plate, is that in another database somewhere for the names?
I was using the complete parts library that I downloaded not a local one installed with LDraw\Lecad\MLcad mostly because I couldn't find them.

RE: Automatic image creation for training AI - Willy Tschager - 2021-07-06

(2021-07-06, 15:03)Murray Wrote: I had a quick look at getting this working (which I couldn't quite figure out yet, command line as in command prompt?)

Have a look at this: http://www.holly-wood.it/ldview-en.html

w.

RE: Automatic image creation for training AI - Johannes Ahlmann - 2021-08-03

I am currently doing this using Blender and Cycles path tracing renderer.

Happy to help you out if you are still looking for some input/ thoughts.

You can see some (pathologic) examples in this thread:
https://forums.ldraw.org/thread-24780.html

Regards,
Johannes

RE: Automatic image creation for training AI - Murray - 2021-08-03

(2021-08-03, 21:44)Johannes Ahlmann Wrote: I am currently doing this using Blender and Cycles path tracing renderer.

Happy to help you out if you are still looking for some input/ thoughts.

You can see some (pathologic) examples in this thread:
https://forums.ldraw.org/thread-24780.html

Regards,
Johannes

Wow, those renders look real!
I'm trying to get my camera setup working for this project, but once I get that sorted I will definitely ask how you got did that.

Thanks Johannes

RE: Automatic image creation for training AI - Orion Pobursky - 2021-08-03

(2021-08-03, 21:44)Johannes Ahlmann Wrote: I am currently doing this using Blender and Cycles

If you're Blender/Cycles savvy, Roland is currently looking for a collaborator with the LDCad Blender export.