LDraw.org Discussion Forums

Full Version: Auto correcting (some) defects in the official library
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4 5 6
Michael Heidemann Wrote:How should a corrected "bowtie quad" be controlled to have the correct winding???
A part with proper BFC wouldn't have bow ties, so any bow tie corrected file will be ether non bfc or have a false bfc status to begin with.
I've ran the last version of my tweaks on a clean 1401 library (re downloaded/installed complete.zip to be sure) and put all changed files in a

7MB 7zip file

The archive contains two lib root folders, one containing files whom only needed cap corrections and one with all the other auto fixed stuff.

I've also included the LDCad logfiles of the batch run for them both so you can check what was changed in detail, although I recommend using something like winmerge or svn to detect / inspect the changes.

Finally there is also a log file of a batch run on the already corrected library which shows the 'fatal' errors which remain in a couple of dozen files.

I'm not in anyway suggesting these files should be bluntly put into the part tracker as they might need human intervention adjustments in some cases.

Anyway here you go Smile
Thanks for undertaking this work.

To ease the processing of these files through the Parts Tracker, would it be possible to update the !LDRAW_ORG line to Unofficial and add a new !HISTORY line? Otherwise I need to admin edit every file.

If the basis for this was the latest official library, how do you propose merging your corrections with changes in files already on the parts tracker. I've not counted, but there is at least one case of this (2599.dat).
You could probably use a simple search and replace for the !LDRAW_ORG line (I like using grepwin for such things)

But the history lines might be a problem. Current version of LDCad does not really manage those lines, I could add a simple 'insert after last one (no matter where it might be)' function though.

Or maybe someone here could demonstrate some regular expression search and replace skills in order to add those lines Smile

As for files already on the trackar, I would ignore those as datheader can do (and probably already has done) the same corrections on those files as part of their review process??.
No, Please don't do that.

Updated part now present at the PT should not be overwritten.
We've spent hour correcting them !
I re-iterate, it is great that you are taking this on. Many people will appreciate a cleaner library, but please don't under-estimate the effort involved.
Roland Melkert Wrote:You could probably use a simple search and replace for the !LDRAW_ORG line (I like using grepwin for such things)
Yes, there are many ways that I could do this, but if you want to help get these submitted to the Parts Tracker, I think you should make as much effort as possible to fit in with the change control process.
The current official LDRAW_ORG line should be updated something like:

0 !LDRAW_ORG Part UPDATE 2003-03
to
0 !LDRAW_ORG Unofficial_Part

Roland Melkert Wrote:But the history lines might be a problem. Current version of LDCad does not really manage those lines, I could add a simple 'insert after last one (no matter where it might be)' function though.
They will only be in the header so inserting something like this after the last one would be fine.
0 !HISTORY 2014-11-18 [roland] Auto-corrected with LDCad Vxx.x

The parts release script uses the following algorithm (but it's written in Perl, so I guess not much use for you).
- Read and re-write header to "0 !LICENSE" license line (in fact in official library files this should always be exactly five lines)
- Read and re-write rest of header until one of the following REGEXs is satisfied:
Code:
/^\s*[1-5]\s/
/^\s*0\s*BFC\s*INVERTNEXT\s*/
/^\s*0\s*\/\/\s*/
- write new !HISTORY line (don't forget CRLF DOS-style line ending)

You will also need to request Part Author rights for me to be able to proxy-submit these files under your name, viz:
Part Tracker Wrote:... send an email to the Parts Library Admin (Chris Dee) with your true first and last names and your LDraw.org username. You must also include the statement "I accept the LDraw.org Contributor Agreement with regards to all past and future contributions I make to LDraw.org".

If possible please separate files which already exist as updated versions on the Parts Tracker - these will need special treatment to avoid losing any changes (which may be nothing to do with the corrections you are making).
let's leave out the uppercase lowercase changes for now

the remaining 300 files can be manually reviewed, that's doable.

the uppercase/lowercase change should happen as a separate batch, for example
done by Chris with the next parts release IMHO
Initially I only intended to process these files to help kick start the cleaning process no need to credit me etc. Hence the request (in the first post) for some one else to do something with them in regards of the pt etc.

But if it helps things get along more quickly I'll happily make those header changes during the export, it will just take me a couple of days to find the time to make the necessary changes to LDCad and process the lib again.

@Magnus: Concerning the files already on the pt, like I wrote before those should indeed be ignored/excluded.

Is there a list of all parts currently on the tracker whom are also part of 1401?, if so I can exclude them in code to make 100% sure they won't cause problems.
Roland Melkert Wrote:I've ran the last version of my tweaks on a clean 1401 library (re downloaded/installed complete.zip to be sure) and put all changed files in a

7MB 7zip file

The archive contains two lib root folders, one containing files whom only needed cap corrections and one with all the other auto fixed stuff.

I've also included the LDCad logfiles of the batch run for them both so you can check what was changed in detail, although I recommend using something like winmerge or svn to detect / inspect the changes.

Finally there is also a log file of a batch run on the already corrected library which shows the 'fatal' errors which remain in a couple of dozen files.

I'm not in anyway suggesting these files should be bluntly put into the part tracker as they might need human intervention adjustments in some cases.

Anyway here you go Smile

Awesome work! Thanks!
Supposing we decide to use these parts, what kind of test should we made? A visual inspection on LDView would be enought? We could divide them among us and verify them rather quickly
Quote:Is there a list of all parts currently on the tracker whom are also part of 1401?, if so I can exclude them in code to make 100% sure they won't cause problems.
I just made one, for each folder of library (attached).
Pages: 1 2 3 4 5 6