LDraw.org Discussion Forums
Search path discussion - Printable Version

+- LDraw.org Discussion Forums (https://forums.ldraw.org)
+-- Forum: General (https://forums.ldraw.org/forum-12.html)
+--- Forum: Official File Specifications/Standards (https://forums.ldraw.org/forum-32.html)
+--- Thread: Search path discussion (/thread-24495.html)

Pages: 1 2 3 4


Search path discussion - Orion Pobursky - 2021-03-16

Since non-LSB members can't comment on the LSB forum, this thread is for discussion of the search path proposal there.

To kick this off I'd like to comment on something Roland posted:
Quote:I disagree on this, a stud.dat should not replace all studs in the whole scene if it's located in the mpd itself or its folder.

This is precisely the point of embedding such a file in the MPD and why we do it for the OMR. Local files should always override library parts. Otherwise the OMR breaks and parts authoring become morse inconvenient since you'd have to overwrite the official file if you're using or creating a fix.


RE: Search path discussion - Philippe Hurbain - 2021-03-16

(2021-03-16, 13:26)Orion Pobursky Wrote: This is precisely the point of embedding such a file in the MPD and why we do it for the OMR. Local files should always override library parts. Otherwise the OMR breaks and parts authoring become morse inconvenient since you'd have to overwrite the official file if you're using or creating a fix.
Fully agree. Embedded parts should have precedence so a model using unofficial parts won't be broken inadvertantly. I remember a time when embedding parts was little used and it was a struggle to find again the unofficial parts used by author.


RE: Search path discussion - Roland Melkert - 2021-03-16

(2021-03-16, 14:48)Philippe Hurbain Wrote: Fully agree. Embedded parts should have precedence so a model using unofficial parts won't be broken inadvertantly. I remember a time when embedding parts was little used and it was a struggle to find again the unofficial parts used by author.

I don't understand. If the parts are embedded in the MPD they will be loaded instead of the library ones.

But only withing the context of the model.

So if you put a stud.dat inside the mpd with e.g. a yellow 4-4disc.dat only the embedded files actually referencing stud.dat will use that stud variant.

All the other non embedded files will use the normal stud.dat as found in <lib>\p

Like so:
   

If I understand you correctly you would want all studs to be yellow in that image ?

LDView seems to ignore it and use the library one for everything.
   


mpd code:
Code:
0 FILE main.ldr
1 7 0 0 0 -1 0 0 0 1 0 0 0 -1 819.dat
1 10 40 -24 -30 -1 0 0 0 1 0 0 0 -1 3065.dat
1 10 0 -24 -30 -1 0 0 0 1 0 0 0 -1 3065.dat
1 10 -30 -24 -20 0 0 -1 0 1 0 1 0 0 3065.dat
1 10 -40 -48 20 -1 0 0 0 1 0 0 0 -1 brick.dat

0 FILE brick.dat
0 UNOFFICIAL PART
0 BFC CERTIFY CCW
1 16 -10 0 0 1 0 0 0 1 0 0 0 1 stud.dat
1 16 10 0 0 1 0 0 0 1 0 0 0 1 stud.dat
1 16 0 24 0 20 0 0 0 24 0 0 0 20 box.dat

0 FILE stud.dat
0 UNOFFICIAL PART
0 BFC CERTIFY CCW
1 16 0 0 0 6 0 0 0 1 0 0 0 6 4-4edge.dat
1 16 0 -4 0 6 0 0 0 1 0 0 0 6 4-4edge.dat
1 16 0 0 0 6 0 0 0 -4 0 0 0 6 4-4cyli.dat
1 14 0 -4 0 6 0 0 0 1 0 0 0 6 4-4disc.dat
0 1 16 0 -4 0 1 0 0 0 1 0 0 0 1 logo3.dat



RE: Search path discussion - Orion Pobursky - 2021-03-16

That's not the way it's supposed to work. If stud.dat is embedded in the mpd or located in the same directory as the model file then it should replace all instances of stud.dat for the entire model. This allows, say, a part author to test out a fix on a bunch of different parts simultaneously or a model author to include a custom version of some commonly file (e.g. a stud with a custom logo).


RE: Search path discussion - Roland Melkert - 2021-03-16

(2021-03-16, 19:35)Orion Pobursky Wrote: That's not the way it's supposed to work. If stud.dat is embedded in the mpd or located in the same directory as the model file then it should replace all instances of stud.dat for the entire model. This allows, say, a part author to test out a fix on a bunch of different parts simultaneously or a model author to include a custom version of some commonly file (e.g. a stud with a custom logo).
You could do that by adding your custom/development library paths in front of the official ones.

The main thing I'm disliking about the global scope thing is the fact an embeded file which happens to have the same name as some official part would break all dependencies.

The 'context' method will only affect its local scope so there will never be problems with duplicate names, it's comparable with local variables in a programming language.


RE: Search path discussion - Travis Cobbs - 2021-03-16

The only reason you saw what you did in LDView is because you had primitive substitution enabled (which is the only way to see stud logos in LDView). I didn't mention it in the search path discussion, but LDView's primitive substitution trumps the search path when it is enabled. If LDView recognizes a file to be a primitive when primitive substitution is enabled, it never even bothers to look at the file system.


RE: Search path discussion - Travis Cobbs - 2021-03-16

(2021-03-16, 19:43)Roland Melkert Wrote: You could do that by adding your custom/development library paths in front of the official ones.

The main thing I'm disliking about the global scope thing is the fact an embeded file which happens to have the same name as some official part would break all dependencies.

The 'context' method will only affect its local scope so there will never be problems with duplicate names, it's comparable with local variables in a programming language.

At this point in time, I think it's safe to say that if a user embeds a *.dat file in their MPD, they intend it to replace the *.dat file of that same name in the LDraw parts library. Nobody should be using *.dat for personal files anymore. So I still feel that this global replacement is correct.

Note that putting files alongside the main model to do the same thing is a leftover from the days before MPDs. However, I still think that it is consistent to continue with this behavior.


RE: Search path discussion - Roland Melkert - 2021-03-16

(2021-03-16, 22:11)Travis Cobbs Wrote: At this point in time, I think it's safe to say that if a user embeds a *.dat file in their MPD, they intend it to replace the *.dat file of that same name in the LDraw parts library. Nobody should be using *.dat for personal files anymore. So I still feel that this global replacement is correct.

So to be clear, the way you expect it to work is like so:

given a mpd containing main.ldr, brick.dat and stud.dat  with a reference tree like so:

d:\model.mpd
  main.ldr
    3001.dat
       s\3001s01.dat
         stud.dat
   brick.dat

  brick.dat
    box.dat
    stud.dat

  stud.dat


The main.ldr reference will look in these
<mpd>    <--hit
d:\
<lib>\p
<lib>\parts

The 3001.dat reference will look in these:
<mpd>
d:\
<lib>\p
<lib>\parts    <-- hit

The s\3001s01.dat reference in 3001.dat will look in these:
<lib>\parts  (because its the parent folder of 3001.dat, just like d:\ is for the mpd)    <-- hit
<mpd>
d:\
<lib>\p 
<lib>\parts 

The stud.dat reference in s\3001s01.dat will look in these:
<lib>\parts\s  (parent folder of 3001s01.dat)
<mpd>     <-- hit
d:\
<lib>\p
<lib>\parts 


So in short the mpd content and parent folder of the file being loaded will be prefixed to ALL following search requests?

And every recursively loaded file will add to that by also prefixing their folder/mpd location?

Do note I make no distinction between .dat, .ldr, .mpd or if its located in parts / p etc, they are all just ldraw files being processed using a generic method.

Or are we suggesting that models have different kind of rules for resolving references then parts (which is even more messy imho).

so what would happen if you got a second mpd also containing a stud.dat which references the first mpd? 

e.g.
Code:
a.mpd
  main.ldr
     3001.dat
     b.mpd
  stud.dat

b.mpd
  main.ldr
     3001.dat
  stud.dat


If I understand the proposed system correctly it should use b.mpd's stud.dat for every brick used  in b.mpd's model's and it should use a.mpd's stud.dat for everything else.

You will end up with 2 different versions of <lib>parts\3001.dat this way.

The context approach wouldn't have this problem because every file on disk lives in it's own private container, which is very easy to cache as they never change, even when you load additional ldraw files.

In the context approach you would only end up with two different 3001.dat's if they have different ldraw sources much easier to maintain (especially in editors).

Hope this is somewhat understandable as I'm mainly trying to show we really should look beyond the single mpd impact of this 'global' system.


RE: Search path discussion - Trevor Sandy - 2021-03-17

(2021-03-15, 20:05)Roland Melkert Wrote: The real question here is do we WANT a local stud.dat to affect the global scope or should it only affect its local context (like I assumed).

imho the local context approach is much more logical, global stuff should be handled by prefixing extra search locations to the global order.

Since LDraw.org does not author any applications why not define specifications in a manner that support both scenarios as they are both valid? This way, application developers can decide if to support either or both approaches and LDraw.org could focus on qualifying how unofficial file load/parse under each scope should be implemented, pros, cons etc...

Cheers,


RE: Search path discussion - Travis Cobbs - 2021-03-17

(2021-03-16, 23:43)Roland Melkert Wrote: So to be clear, the way you expect it to work is like so:

given a mpd containing main.ldr, brick.dat and stud.dat  with a reference tree like so:

d:\model.mpd
  main.ldr
    3001.dat
       s\3001s01.dat
         stud.dat
   brick.dat

  brick.dat
    box.dat
    stud.dat

  stud.dat


The main.ldr reference will look in these
<mpd>    <--hit
d:\
<lib>\p
<lib>\parts

The 3001.dat reference will look in these:
<mpd>
d:\
<lib>\p
<lib>\parts    <-- hit

The s\3001s01.dat reference in 3001.dat will look in these:
<lib>\parts  (because its the parent folder of 3001.dat, just like d:\ is for the mpd)    <-- hit
<mpd>
d:\
<lib>\p 
<lib>\parts 

This last part is wrong. While logical, "current subfile directory" is never supposed to be in the part search path. In other words, 3001.dat will only be able to find s\3001s01.dat because <lib>\parts is in the standard search path. In the same way, s\3001s01.dat would not find a hypothetical s\3001s02.dat if it simply referred to it as 3001s02.dat. It is required to use s\3001s02.dat.


(2021-03-16, 23:43)Roland Melkert Wrote: The stud.dat reference in s\3001s01.dat will look in these:
<lib>\parts\s  (parent folder of 3001s01.dat)
<mpd>     <-- hit
d:\
<lib>\p
<lib>\parts 

Edited.

(2021-03-16, 23:43)Roland Melkert Wrote: So in short the mpd content and parent folder of the file being loaded will be prefixed to ALL following search requests?

No: the mpd content and the parent folder of the top-level file that the user opened will be prefixed to all search requests.


(2021-03-16, 23:43)Roland Melkert Wrote: And every recursively loaded file will add to that by also prefixing their folder/mpd location?

As stated above, only the top-level folder is in the path. We should perhaps decide what rules should be followed if an mpd loads another mpd. I think a better solution to that may be to state that the behavior is undefined, and consider mpds loading other mpds to be invalid. (The whole point of mpds is to encapsulate a bunch of files into one, so having one load another defeats their very purpose, IMO.


(2021-03-16, 23:43)Roland Melkert Wrote: Do note I make no distinction between .dat, .ldr, .mpd or if its located in parts / p etc, they are all just ldraw files being processed using a generic method.

Or are we suggesting that models have different kind of rules for resolving references then parts (which is even more messy imho).

From the standpoint of loading the files, all should be treated equal, no matter what they are or where they come from. Once the file is loaded, programs are free (and even encouraged) to detect parts and treat them differently. LDView does this (for seams), and I'm pretty sure LDCad does also.


(2021-03-16, 23:43)Roland Melkert Wrote: so what would happen if you got a second mpd also containing a stud.dat which references the first mpd? 

e.g.
Code:
a.mpd
  main.ldr
     3001.dat
     b.mpd
  stud.dat

b.mpd
  main.ldr
     3001.dat
  stud.dat


If I understand the proposed system correctly it should use b.mpd's stud.dat for every brick used  in b.mpd's model's and it should use a.mpd's stud.dat for everything else.

You will end up with 2 different versions of <lib>parts\3001.dat this way.

The context approach wouldn't have this problem because every file on disk lives in it's own private container, which is very easy to cache as they never change, even when you load additional ldraw files.

In the context approach you would only end up with two different 3001.dat's if they have different ldraw sources much easier to maintain (especially in editors).

Hope this is somewhat understandable as I'm mainly trying to show we really should look beyond the single mpd impact of this 'global' system.

We can define what happens if one mpd reference another, but as I said above, I would prefer to state that it should be avoided at all costs and explicitly make the behavior undefined. I'm not entirely sure what LDView does in this situation right now. Either stud.dat from a.mpd will be used for the entire model, or stud.dat from b.mpd will be used for the entire model. (Probably a.mpd, but I'm not sure.) I'm pretty sure that it will never have two different files with the same "reference filename" in memory at one time, since my "loaded model" cache is keyed off the lower case version of that "reference filename". (Note: by "reference filename", I mean the filename listed in the type 1 line that caused the file to be loaded.)

What I'm not sure of is what it does when it encounters the second stud.dat. I think it just ignores it, but if it doesn't, it might in fact be broken.