Online tool for OMR compliance

[Lasse Deleuran]

It has been some time since I floated the idea in here, but I have finally added functionality to buildinginstructions.js and by extension, the website BrickHub.org which checks for OMR compliance, and in most cases, can fix errors directly on the site. here is a video showing how to use it

I do not expect that I have found all OMR checks, and some of my checks might even be erroneous. This is why I am creating this thread. Let's discuss OMR checks and fixes  

The checks and fixes are currently implemented in this Javascript file: LDROMR.js

Here are the checks that I have implemented:

• Check and fix individual file headers read "0 FILE <set number> - <name>.ldr" then "0 <name>", followed by "0 Name:<set number> - <name>.ldr".
  
• Ensure the file name is "<set number> - <name>.mpd"
  
• Check and fix moved parts: If a part is named "~Moved <name>" then replace its usages with "<name>.dat"
  
• Check and fix placements: If a part is placed with more than three decimals in any coordinate, then round the placement to three decimals. Example: "1 16 0.0001 0 0 ..." becomes "1 16 0 0 0 ...".
  
• Check and fix all author lines: Ensure all authors are the same and read "!Author First name Last name [username]"
  
• Check and fix all license lines to read "Redistributable under CCAL version 2.0 : see CAreadme.txt"
  
• Set all "0 LDRAW_ORG" lines to read either "Model" or "Unofficial_Model".
  
• Ensure all unofficial parts are copied into the MPD file and named with the set number prefix in the "0 FILE" and "0 Name:" lines.
  
• If the set is from 2007 or after and contains old colors (6, 7, or 8), then replace them with the newer colored parts.
  
• If the set is from before 2007 and contains new colors (70, 71, or 72), then replace them with the old colored parts.
  

These are the checks I have not yet implemented:

• Verify that "Theme" lines correctly identify a theme.
  
• Verify that "History" lines are well formed.
  
• Verify that "History" lines are in the correct order.
  

The purpose of this work is to make it even easier to submit OMR-compliant model file - especially for people who are not using Windows, so they can't easily use MPDCenter.

[Orion Pobursky]

Is this .js file something we can use on LDraw.org or is it tightly coupled to all your other code?

[Lasse Deleuran]

Is this .js file something we can use on LDraw.org or is it tightly coupled to all your other code?

I have done my best to make it reusable for others, and focused on basic tree traversal. The Javascript file basically consists of pairs of functions, and converting it to any other object oriented or functional language should be a breeze. 

The potentially difficult part would be to match the data model. The LDraw (or MPD) file structure is parsed into a tree structure: Each 'file' in an MPD file is represented by one type of nodes. Each step within it as a second type of node, and finally each line is a third type of node.

Other programs might not model steps as separate nodes, but as long as you model MPD files as tree structures, then translation to another program should be fairly easy.


I have added documentation to the LDROMR.js file to help potential developers both to add their own checks and to translate everything to another language.

[Orion Pobursky]

I have done my best to make it reusable for others, and focused on basic tree traversal. The Javascript file basically consists of pairs of functions, and converting it to any other object oriented or functional language should be a breeze. 

The potentially difficult part would be to match the data model. The LDraw (or MPD) file structure is parsed into a tree structure: Each 'file' in an MPD file is represented by one type of nodes. Each step within it as a second type of node, and finally each line is a third type of node.

Other programs might not model steps as separate nodes, but as long as you model MPD files as tree structures, then translation to another program should be fairly easy.


I have added documentation to the LDROMR.js file to help potential developers both to add their own checks and to translate everything to another language.

Awesome. I’ll look into it at some point. 

Santeri is working on a part validator but it’s written in python. I don’t want to step on his toes.

[Philippe Hurbain]

Great tool and tutorial video! I'll check that in more details when I do my next model...

[Lasse Deleuran]

Great tool and tutorial video! I'll check that in more details when I do my next model...

That would be great. And please tell me how it goes.

[Eugen]

The purpose of this work is to make it even easier to submit OMR-compliant model file - especially for people who are not using Windows, so they can't easily use MPDCenter.

Is it possible port it to C++? It would be cool merge such OMR checker in open-source cross platform software such as LeoCAD, (that work not only on Windows, but on Linux MacOS and other OS too)

• https://github.com/leozide/leocad/issues/273

[Lasse Deleuran]

Is it possible port it to C++? It would be cool merge such OMR checker in open-source cross platform software such as LeoCAD, (that work not only on Windows, but on Linux MacOS and other OS too)

• https://github.com/leozide/leocad/issues/273
  

Porting to C++ is easy.
Fitting it into the internal structure might be tricky. I do not know enough of this to give an estimation.

If it fails, then my "sample_functions.htm" file can be used as a starting point for an OMR conversion service. Let's see what people will find useful.

[Willy Tschager]

Lasse,

I tried your tool today with the 926 - Command Centre as for some strange reasons it is in the AIOI but not the OMR. I deliberately added some errors just to see how the site copes with them. Here is the source and the result I downloaded from Brickhub.


  926 - Command Centre.mpd (Size: 15.65 KB / Downloads: 1)

  926 - 926.mpd (Size: 14.71 KB / Downloads: 1)

It picked the {} in the Usename, it fixed the MLCad metas:

0 ROTATION CENTER 0 0 0 1 "Custom"
0 ROTATION CONFIG 0 0

BUT

(please ignore my tone if you might find it harsh ;-)

• It didn't load the MPD name - just the set number and as you can see it returned a 926 - 926.mpd.
  
• I find the page overwhelming at first.
  
• I don't like the fact that the thumbnail generator/instructions/parts is in between the section where I actually edit the header and the suggestions what has to be changed in order to get it OMRized.
  
• I cannot actually see the code itself or at least the header
  
• All line breaks get deleted and you cannot longer view the file in notepad for a quick check.
  
• The site continuous to switch between Unofficial_Model and Model. When I set it to Model it still suggests to set to Unofficial model
  
• I would love to get an indication in the header section itself if for example:
  Author: Willy Tschager [Holly-Wood] ✎
  is right or wrong OMR wise (green tick/red cross)
  
• There is a Download button and a Delete button, where is the upload button when I'm done? I know there is the button which says Upload once you have picked the file from the disk but my understanding is that I LOAD, correct and upload.
  

w.

[Lasse Deleuran]

Lasse,

I tried your tool today with the 926 - Command Centre as for some strange reasons it is in the AIOI but not the OMR. I deliberately added some errors just to see how the site copes with them. Here is the source and the result I downloaded from Brickhub.




It picked the {} in the Usename, it fixed the MLCad metas:

0 ROTATION CENTER 0 0 0 1 "Custom"
0 ROTATION CONFIG 0 0

BUT

(please ignore my tone if you might find it harsh ;-)

• It didn't load the MPD name - just the set number and as you can see it returned a 926 - 926.mpd.
  
• I find the page overwhelming at first.
  
• I don't like the fact that the thumbnail generator/instructions/parts is in between the section where I actually edit the header and the suggestions what has to be changed in order to get it OMRized.
  
• I cannot actually see the code itself or at least the header
  
• All line breaks get deleted and you cannot longer view the file in notepad for a quick check.
  
• The site continuous to switch between Unofficial_Model and Model. When I set it to Model it still suggests to set to Unofficial model
  
• I would love to get an indication in the header section itself if for example:
  Author: Willy Tschager [Holly-Wood] ✎
  is right or wrong OMR wise (green tick/red cross)
  
• There is a Download button and a Delete button, where is the upload button when I'm done? I know there is the button which says Upload once you have picked the file from the disk but my understanding is that I LOAD, correct and upload.
  

w.

First of all. Thanks for the feedback and I'm sorry I could not come back to this earlier. I'm used to a much harsher tone in my everyday work, so it is no problem that you write it all up in brief form. I can also understand the frustration, so let's dive into the points:

• File name being "926 - 926.mpd": I found the bug in the uploader. This has been fixed now. Uploading your file results in the name being read correctly: "Command Centre - Main File". Thanks for reporting it!
  
• Page being overwhelming: I have reduced the amount of text and done some restructuring. I hope it is less overwhelming now, but I would still like to take suggestions.
  
• Suggestions below image: Good point. The suggestions have now been moved up above the image.
  
• View the file source code in the page: I am working on a solution for this. The parts list has a pop-up for Bricklink wanted list code, so this feature should be easy to add for the file content.
  
• Line breaks: The code was saving with Linux line ends. I have changed it to Windows style, which is also more correct according to the standard.
  
• Unofficial_Model vs Official Model: It was not easy to make a good decision here. I want people to be able to change to either of these values, but this also causes you to be able to change back and forth all the time. I have come up with a new solution. When you set it to 'Model' it is unlikely that you want to change it back to unofficial, so I have removed the option - unless you just pressed 'Model' - then you are still able to press 'Unofficial' in case you pressed the other button by mistake.
  
• Having red/green ticks next to the header fields: This makes sense. I will look into a way where this will not be too cluttered while still showing the OMR messages - this is currently a challenge for me to get right for users on touch-based devices, hence the more blunt OMR line blocks below.
  
• Download vs Upload: I am not sure I understand this correctly. 'Upload' always means computer->website, while 'Download' means website->computer.
  

I will return once I have a solution for the being able to view the file content.

And again. Please bring in more suggestions.

[Willy Tschager]

File name being "926 - 926.mpd": I found the bug in the uploader. This has been fixed now. Uploading your file results in the name being read correctly: "Command Centre - Main File". Thanks for reporting it!

Now it loads:

Surface Explorer - Main Model

and saves:

6880 - Surface Explorer - Main Model.mpd

while my file was:

6880 - Surface Explorer.mpd

So this is still messy. I expected the mpd name 6880 - Surface Explorer to show up and save the same name if untouched.

Download vs Upload: I am not sure I understand this correctly. 'Upload' always means computer->website, while 'Download' means website->computer.

What I expected was that the file doesn't end up immediately at my Brickhub page. I see the OMRizer more as a test/edit page before your model gets stored on Brickhub. So I log in, LOAD my file, do all the needed editing and then DOWNLOAD/UPLOAD/DELETE the model. I'm somewhat confused that More models uploaded by Holly Wood is on the same page. The OMRized should be more of a subset where I edit a particular file I pick from My page.

w.

[Lasse Deleuran]

As it is right now, OMR compliance checks is something that are triggered when an official model is detected. I would prefer if all official models on brickhub.org also became OMR compliant. You are right that it seems a bit strange to see your other models in the bottom of the page, so I have removed that feature. To "publish" a file, the toggle "Show on front page and in searches" is used. I feel this formulation is less technical than "SUBMIT"/"Publish" or similar.



I have updated the page to give a more intuitive representation of the connection between set number, set name and the file which is downloaded. When selecting "Official LEGO set", the title becomes the file name and by clicking on either the set number or the set name, these become editable (the dash and ".mpd" are fixed). This was the best I could come up with tonight. I am open to better suggestions.

The OMR function for updating header lines has also been improved. It now tries to insert the correct set number according to the one in the title (previously it simply appended, resulting in non-optimal file names).

Another fix has been in detection of suffixes for files. Assume the set number is "1234". Now when it detects:

1234 - Model.mpd
1234 - Model
Model.ldr
1234-Model.ldr
43849 - Model.ldr

it will correct to:

1234 - Model.ldr

A lot of the files in the AIOI have "1234 - Model.mpd" instead og "1234 - Model.ldr". Is this technically OMR compliant?

[Johann Eisner]

In my opinion, .ldr is wrong and .mpd is correct.

Please refer LDraw.org Standards: Official Model Repository (OMR) Specification Version 1.0.3

Has been discussed here before: https://forums.ldraw.org/thread-23070.html

Excerpt from the OMR Specification:

Base File Naming
Each model in the OMR will consist of several files that are packaged together into a single MPD file. For sets that include instructions for multiple models, each model will have its own MPD file. Each MPD for the set will be named in the following manner:

Code:

<Set Number>[-<Optional Qualifier>] - <Set Name>[ - <Sub Model Name>]

Where:

Code:

<Set Number>

: The number assigned on the container of the set.

Code:

<Optional Qualifier>

: Is a sequential number, starting with 2.

Code:

<Set Name>

: The name of the set printed on the container in English.

Code:

<Sub Model Name>

: This is Optional in most cases. This is required for alternate models that are detailed in instructions (e.g. the Creator theme). In this case the naming is left to the discretion of the author but should be descriptive of the model contained in the MPD.
The

Code:

<Optional Qualifier>

is not mandatory and gets added only if there is more than one set that could be assigned the same

Code:

<Set Number>

. The first set using a given number would be understood to never contain the qualifier however numbering should start with the oldest set and some investigation should be done in existing set databases.
Example:

Code:

6901 - Mobile Lab.mpd (Produced in 1980)
6901-2 - Space Plane.mpd (Produced in 1998)

For playsets or other sets where there are multiple models that are part of an integral whole, all of the submodels will be contained in one MPD.
Example:
The creator set 4896 - Roaring Roadsters has 3 models in the instructions:

Code:

4896 - Roaring Roadsters - Roadster.mpd
4896 - Roaring Roadsters - Dragster.mpd
4896 - Roaring Roadsters - SUV.mpd

[Lasse Deleuran]

The thing is. The example in the Specification has it the other way around with only ".ldr" sub files within a ".mpd" enclosing file:

Code:

0 FILE 6712 - Main.ldr
0 Main
0 Name: 6712 - Main.ldr
0 Author: Willy Tschager [Holly-Wood]
0 !LDRAW_ORG Model
0 !LICENSE Redistributable under CCAL version 2.0 : see CAreadme.txt

1 0 0 0 0 1 0 0 0 1 0 0 0 1 6712 - Fireplace.ldr
1 0 -140 -49 -32 0.5 0 -0.866 0 1 0 0.866 0 0.5 6712 - Horse.ldr
1 0 -140.866 -129 -31.5 0.5 0 -0.866 0 1 0 0.866 0 0.5 6712 - Sheriff.ldr
1 0 83 -99 64 1 0 0 0 1 0 0 0 1 6712 - Bandit.ldr
1 6 81 3 76 1 0 0 0 1 0 0 0 1 30139.dat
1 2 -51 8 86 1 0 0 0 1 0 0 0 1 6064.dat
1 7 80 0 -14 0.707 0 0.707 0 1 0 -0.707 0 0.707 3069bp03.dat

0 FILE 6712 - Horse.ldr
0 Horse
0 Name: 6712 - Horse.ldr
0 Author: Willy Tschager [Holly-Wood]

[Orion Pobursky]

Nowhere in the spec does it require a file to have a mpd extension. Since a MPD document is defined by the 0 FILE meta and not the file extension it is acceptable for the base file to have either a ldr or mpd extension.

[Johann Eisner]

Sorry, I misinterpreted the text. 😔
I thought it was about the main file, not the submodel files.
Have a nice sunday.

[Lasse Deleuran]

Sorry, I misinterpreted the text. 😔
I thought it was about the main file, not the submodel files.
Have a nice sunday.

No worries. I will try to let the page be as is and test it for helping with OMR compliance for some models that I have planned.