Part Snapping Language Extension

[Orion Pobursky]

The proposal for part snapping extension is now ready for discussion. I basically copy, pasted, and wiki formatted Roland's LDCad spec.

https://wiki.ldraw.org/wiki/Part_Snappin..._Extension

Here are some thoughts/notes for discussion:
- The wiki has a talk page for discussion. Please do not use that and have all discussions here.
- I removed the LDCAD prefix, should we add something else?

[Roland Melkert]

The proposal for part snapping extension is now ready for discussion. I basically copy, pasted, and wiki formatted Roland's LDCad spec.

https://wiki.ldraw.org/wiki/Part_Snappin..._Extension

I think you can leave out the shadow library references.

Also the grid property is a bit of a mess in LDCad as I started it out as only supporting x and z, but the newest version also supports y. I just realized this is undocumented.

In order to have backwards compatibility both formats are accepted, e.g.

C 4 C 8 20 20
for x, z and
C 4 C 2 C 8 20 24 20
for x, y, z

I think for the official one we should only use the x y z one.

[Orion Pobursky]

The proposal for part snapping extension is now ready for discussion. I basically copy, pasted, and wiki formatted Roland's LDCad spec.

https://wiki.ldraw.org/wiki/Part_Snappin..._Extension

I think you can leave out the shadow library references.

Also the grid property is a bit of a mess in LDCad as I started it out as only supporting x and z, but the newest version also supports y. I just realized this is undocumented.

In order to have backwards compatibility both formats are accepted, e.g.

C 4 C 8 20 20
for x, z and
C 4 C 2 C 8 20 24 20
for x, y, z

I think for the official one we should only use the x y z one.

Ok. The wiki should be editable to anyone who registerd so feel free to make changes. You are the subject matter expert.

[Roland Melkert]

Ok. The wiki should be editable to anyone who registerd so feel free to make changes. You are the subject matter expert.

Did not realize that.

I plan to update my website in my upcoming vacation time along side releasing a new LDCad version.

I'll transfer anything related to the wiki too.

[Orion Pobursky]

Ok. The wiki should be editable to anyone who registerd so feel free to make changes. You are the subject matter expert.

Did not realize that.

I plan to update my website in my upcoming vacation time along side releasing a new LDCad version.

I'll transfer anything related to the wiki too.

This is primarily the reason I put both this and the flex part extension on the wiki.

[Merlijn Wissink]

An official snapping specification sounds pretty nice!
I was wondering; if this goes through and everything is agreed upon, will that mean that all* parts wil have to go through the tracker again to include snapping metadata? That's going to be a boatload of parts.

*With all I mean all parts that currently have snapping metadata in LDCad, not all in the whole LDraw library.

[Steffen]

> will that mean that all* parts wil have to go through the tracker again to include snapping metadata?

I think that's the wrong wording.
It just means that if someone wants to add that info to a part, it can be done,
and it is allowed on the PT, and, if present, will/should be included in the part review.

Pumping all parts onto the PT is not the way to go.
As we are all volunteers, people will have their favorite parts to work on, and it will depend on time and dedication
if and whether a part gets snapping info officially attached.

It will take a long time until all official parts carry that information.
In a far future, when that is the case, we then (in 10 years or so) can make connectivity info mandatory (like nowadays we require BFC).

Everybody remember the times when
- parts were not even syntactically correct
- parts were not BFC correct

These times have long gone now. All official parts
- have undergone a syntax check
- carry proper BFC info now

So, there is hope

[Philippe Hurbain]

> will that mean that all* parts wil have to go through the tracker again to include snapping metadata?

I think that's the wrong wording.
It just means that if someone wants to add that info to a part, it can be done,
and it is allowed on the PT, and, if present, will/should be included in the part review.

Pumping all parts onto the PT is not the way to go.
As we are all volunteers, people will have their favorite parts to work on, and it will depend on time and dedication
if and whether a part gets snapping info officially attached.

If we have a snapping structure close enough to LDCad one (and if Roland agree!!!) we could make a small utility to convert LDCad snapping info and add that to parts. Of course it would still need to be validated through PT, but a huge proportion of parts could be rapidly covered this way.

[Steffen]

I would love to have a button in LDView which renders that snapping information in some visual way...

[Travis Cobbs]

It seems like all parts that LDCad already has snapping info for need to have snapping info added in some kind of automated way. And I don't think forcing all of those parts to go through the standard review process would be the best way to go in this case.

[Travis Cobbs]

I would love to have a button in LDView which renders that snapping information in some visual way...

I won't make any promises, but if we make snapping info legal in official parts, it could happen.

[Orion Pobursky]

I don’t think that part snapping should be held to the same standard as regular parts. I also like the idea of a separate, parallel library. That said, this discussion is for library policy, not the actual extension itself. If and when this extension is adopted, we can have a separate discussion regarding how it fits into the Official Library.

[Chris Dee]

> will that mean that all* parts wil have to go through the tracker again to include snapping metadata?

I think that's the wrong wording.
It just means that if someone wants to add that info to a part, it can be done,
and it is allowed on the PT, and, if present, will/should be included in the part review.

Pumping all parts onto the PT is not the way to go.
As we are all volunteers, people will have their favorite parts to work on, and it will depend on time and dedication
if and whether a part gets snapping info officially attached.

It will take a long time until all official parts carry that information.
In a far future, when that is the case, we then (in 10 years or so) can make connectivity info mandatory (like nowadays we require BFC).

Everybody remember the times when
- parts were not even syntactically correct
- parts were not BFC correct

These times have long gone now. All official parts
- have undergone a syntax check
- carry proper BFC info now

So, there is hope

Does the snapping data need to be in the .dat file for the part? This information will also be common to all patterned versions of a part and it does not make sense to repeat it in every file. Could the data live in a file for each part (design ID) in a sister folder to parts (and p)? That could be maintained without having to update the .dat file?

[Orion Pobursky]

This is kind of Roland already does and what I envisioned we’d do if this extension is adopted

[Chris Dee]

This is kind of Roland already does and what I envisioned we’d do if this extension is adopted

So why all the angst about recycling every part through the Parts Tracker?

[Orion Pobursky]

This is kind of Roland already does and what I envisioned we’d do if this extension is adopted

So why all the angst about recycling every part through the Parts Tracker?

I have no idea. Since it doesn't affect geometry, I don't even think that the standard of acceptance should be the same. Maybe just a pull request with automated checks and admin approval.

[Chris Dee]

So why all the angst about recycling every part through the Parts Tracker?

I have no idea. Since it doesn't affect geometry, I don't even think that the standard of acceptance should be the same. Maybe just a pull request with automated checks and admin approval.

But if the snapping information is in a separate file in a sister folder, we don't even need to touch the existing .dat file.

[Orion Pobursky]

I have no idea. Since it doesn't affect geometry, I don't even think that the standard of acceptance should be the same. Maybe just a pull request with automated checks and admin approval.

But if the snapping information is in a separate file in a sister folder, we don't even need to touch the existing .dat file.

Exactly. That's why I think that a) the snapping info should be in a sister library and b) the current PT approval process is not appropriate for approving snap info.

[Roland Melkert]

But if the snapping information is in a separate file in a sister folder, we don't even need to touch the existing .dat file.

Exactly. That's why I think that a) the snapping info should be in a sister library and b) the current PT approval process is not appropriate for approving snap info.

Speaking from experience: It is not easy to maintain a sister library, especially with all the moved to stuff etc.

In LDCad I use a shadow library only in means to extend the official dat files without polluting the originals, the program doesn't really care if snap info is present in official files or the shadow content. Both sources will be merged in memory before further processing.

[Orion Pobursky]

Exactly. That's why I think that a) the snapping info should be in a sister library and b) the current PT approval process is not appropriate for approving snap info.

Speaking from experience: It is not easy to maintain a sister library, especially with all the moved to stuff etc.

In LDCad I use a shadow library only in means to extend the official dat files without polluting the originals, the program doesn't really care if snap info is present in official files or the shadow content. Both sources will be merged in memory before further processing.

I think the moved to process could be largely automated if we have everything on git. I really am loathe to have non-geometry info in the actual part file. I do not want to send the entire part through the PT process just to change the snapping info.

[Travis Cobbs]

I don’t think that part snapping should be held to the same standard as regular parts. I also like the idea of a separate, parallel library. That said, this discussion is for library policy, not the actual extension itself. If and when this extension is adopted, we can have a separate discussion regarding how it fits into the Official Library.

That's reasonable. My comments on the actual spec:

1. SNAP_INCL really needs the snap info to be in a separate file from the geometry, unless we have dat files that only contain snap info and nothing else. The spec itself, being based on LDCad, mentions "shadow library" in three places.
   
2. I'm not sure I like the fact that many (but not all) of the meta-statements and parameters are heavily abbreviated. For example: SNAP_FGR instead of SNAP_FINGERS, an ori instead of orientation. The fact that "origin" (which is instead called position in this spec) also starts with ori further confuses things.
   

[Merlijn Wissink]

I would love to have a button in LDView which renders that snapping information in some visual way...

In LDCad this is already possible. Open up a model or a part and press F11. It'll highlight all connection points.

[Merlijn Wissink]

Does the snapping data need to be in the .dat file for the part? This information will also be common to all patterned versions of a part and it does not make sense to repeat it in every file. Could the data live in a file for each part (design ID) in a sister folder to parts (and p)? That could be maintained without having to update the .dat file?

My personal opinion is that the snapping data should be in the .dat file itself. It's data that belongs to that part. Adding it in separate library would create so many unnecessary small files.

The patterned parts (reusing snapping metas) problem is not as big as it sounds. You can add snapping info to subpart, which will mean the same snapping info will carry over to all child parts. Many patterned parts have a shared parent submodel in which the snapping info can be defined. You can already see this working in LDCad's current snapping metadata library. Another example would be the stud: adding snapping meta to the few stud submodels and in one swoop you already have a huge part of snapping covered. Of course, there are edge cases. Parts that have non-standard studs that don't use a stud-submodel need manual correction and such, but that'll always be the case no matter the solution.

Plus I think that maintaining a shadow library has its own slew of headaches (you have to keep everything perfectly in sync).

[Chris Dee]

Does the snapping data need to be in the .dat file for the part? This information will also be common to all patterned versions of a part and it does not make sense to repeat it in every file. Could the data live in a file for each part (design ID) in a sister folder to parts (and p)? That could be maintained without having to update the .dat file?

My personal opinion is that the snapping data should be in the .dat file itself. It's data that belongs to that part. Adding it in separate library would create so many unnecessary small files.

The patterned parts (reusing snapping metas) problem is not as big as it sounds. You can add snapping info to subpart, which will mean the same snapping info will carry over to all child parts. Many patterned parts have a shared parent submodel in which the snapping info can be defined. You can already see this working in LDCad's current snapping metadata library. Another example would be the stud: adding snapping meta to the few stud submodels and in one swoop you already have a huge part of snapping covered. Of course, there are edge cases. Parts that have non-standard studs that don't use a stud-submodel need manual correction and such, but that'll always be the case no matter the solution.

Plus I think that maintaining a shadow library has its own slew of headaches (you have to keep everything perfectly in sync).

You make some good arguments there - thanks. But at Orion said elsewhere "this discussion is for library policy, not the actual extension itself.". The proposed SNAP_INCL allows for shared connection metadata and the use of a common subpart for patterned parts does indeed save a lot of the potential duplication.

I am prepared alter my opinion, but I'd also like to see more active Reviewers, otherwise the Parts Tracker throughput could be a barrier.

[Roland Melkert]

[*]SNAP_INCL really needs the snap info to be in a separate file from the geometry, unless we have dat files that only contain snap info and nothing else. The spec itself, being based on LDCad, mentions "shadow library" in three places.

Include will copy the (non recursive) snap info of the given file as currently merged in memory. But I was planning to extend it to (also) allow for generic include files in LDCad 2 so I'm open to do the same here.

[*]I'm not sure I like the fact that many (but not all) of the meta-statements and parameters are heavily abbreviated. For example: SNAP_FGR instead of SNAP_FINGERS, an ori instead of orientation. The fact that "origin" (which is instead called position in this spec) also starts with ori further confuses things.

This was mostly to prevent "code bloat". The same reason for the defaults, as only properties with non default values need to be given. Hence the split of position and orientation.

[Travis Cobbs]

SNAP_INCL really needs the snap info to be in a separate file from the geometry, unless we have dat files that only contain snap info and nothing else. The spec itself, being based on LDCad, mentions "shadow library" in three places.

Include will copy the (non recursive) snap info of the given file as currently merged in memory. But I was planning to extend it to (also) allow for generic include files in LDCad 2 so I'm open to do the same here.

What I meant is that if the snap data goes inside the dat files, then in order to include snap data from another file, the loader also has to load the entire other file's geometry, even if that geometry is never rendered. I understand that using SNAP_INCL wouldn't act like a #include for the whole file, just its snap data, but it still seems bad. You of course wouldn't have this problem in LDCad with the shadow library, because the shadow library files don't contain any geometry. But if this is used as an official spec for files in the official library, then I think that SNAP_INCL as it currently stands is very messy if the snap data is inside the existing dat files.

I'm not sure I like the fact that many (but not all) of the meta-statements and parameters are heavily abbreviated. For example: SNAP_FGR instead of SNAP_FINGERS, an ori instead of orientation. The fact that "origin" (which is instead called position in this spec) also starts with ori further confuses things.

This was mostly to prevent "code bloat". The same reason for the defaults, as only properties with non default values need to be given. Hence the split of position and orientation.

To be clear, I just used that as an example of how the abbreviations were confusing (to me, at least). I definitely don't have a problem with the position and orientation being specified separately, and I also don't have a problem with parameters have good default values.