I used LDraw files and LDCad to train an AI program to recognize bricks

I used LDraw files and LDCad to train an AI program to recognize bricks - Printable Version

+- LDraw.org Discussion Forums (https://forums.ldraw.org)
+-- Forum: General (https://forums.ldraw.org/forum-12.html)
+--- Forum: General LDraw.org Discussion (https://forums.ldraw.org/forum-6.html)
+--- Thread: I used LDraw files and LDCad to train an AI program to recognize bricks (/thread-23906.html)

I used LDraw files and LDCad to train an AI program to recognize bricks - Jacob Sullivan - 2020-02-21

Big shout out to the LDRAW community and Roland Melkert for LDCad! And Rebrickable!
I trained a machine learning algorithm to recognize pictures of bricks, using an animation I made in LDCad. With a parts list from Rebrickable. And parts from LDraw of course.
My ideal scenario is to grab a handful of bricks, toss them on the table, take a picture, and have the program catalog the pieces.

I put all the how-to details in an article on medium. https://towardsdatascience.com/machine-learning-lego-image-recognition-using-virtual-data-and-yolov3-f12e0544012
I am working on some other ideas for machine learning brick projects. And it would be cool if people have feedback or wanted to collaborate.

RE: I used LDraw files and LDCad to train an AI program to recognize bricks - Gerald Lasser - 2020-02-21

(2020-02-21, 22:13)Jacob Sullivan Wrote: Big shout out to the LDRAW community and Roland Melkert for LDCad! And Rebrickable!
I trained a machine learning algorithm to recognize pictures of bricks, using an animation I made in LDCad. With a parts list from Rebrickable. And parts from LDraw of course.
My ideal scenario is to grab a handful of bricks, toss them on the table, take a picture, and have the program catalog the pieces.

I put all the how-to details in an article on medium. https://towardsdatascience.com/machine-learning-lego-image-recognition-using-virtual-data-and-yolov3-f12e0544012
I am working on some other ideas for machine learning brick projects. And it would be cool if people have feedback or wanted to collaborate.

Hello Jacob,

Welcome to the LDraw Forum!
Thanks for sharing this post of your project here! I definitely need to spend more time going through your blog-post , very interesting!

Gerald

RE: I used LDraw files and LDCad to train an AI program to recognize bricks - Philippe Hurbain - 2020-02-22

Very cool, thanks for sharing! I stumbled on Daniel West video recently, both project shows that sorting a heap of LEGO parts may soon no longer be a nightmare Wink

RE: I used LDraw files and LDCad to train an AI program to recognize bricks - Jacob Sullivan - 2020-02-22

(2020-02-22, 9:15)Philippe Hurbain Wrote: Very cool, thanks for sharing! I stumbled on Daniel West video recently, both project shows that sorting a heap of LEGO parts may soon no longer be a nightmare

Thanks!
I thought Daniel West's video was very cool! I have not yet decided to tackle the full on robot sorter ... yet. :-)
But I might put my project into a mobile app so it could be used from a phone. I would need to do some work to let the user identify the parts that the image recognition gets wrong.

RE: I used LDraw files and LDCad to train an AI program to recognize bricks - Philippe Hurbain - 2020-02-22

(2020-02-22, 15:39)Jacob Sullivan Wrote: But I might put my project into a mobile app so it could be used from a phone.

A collaborative system could make it awesome (but I fear that the cost of infrastructure would make it irrealistic!). I use this plant identification app (https://plantnet.org/en/), it works amazingly well...