Standardize official search path


Standardize official search path
#1
It has been requested that we standardize the official search path. So I'm creating this thread to do that. Right now, I think that most programs follow in LDView's footsteps, but I'm open for the official search path being different. LDView by default looks for files in the following order:

MPD subfiles
Model Directory (the directory that the top-level model resides in)
<LDraw>\p
<LDraw>\parts
<LDraw>\models
<LDraw>\Unofficial\p
<LDraw>\Unofficial\parts

If we agree that the above order is correct, we can then discuss where to put it in the specification. If we want a different default search path, we can discuss that.

Note: If your name isn't Travis Cobbs, Chris Dee, or Roland Melkert, please don't respond to this thread. Read the rules for this board. Other than the three of us, only admins and mods can post here, but they aren't supposed to do so. If you want to bring up concerns, do so in the Parts Authoring board.
Reply
RE: Standardize official search path
#2
(2021-03-14, 1:50)Travis Cobbs Wrote: It has been requested that we standardize the official search path. So I'm creating this thread to do that. Right now, I think that most programs follow in LDView's footsteps, but I'm open for the official search path being different. LDView by default looks for files in the following order:

MPD subfiles
Model Directory (the directory that the top-level model resides in)
<LDraw>\p
<LDraw>\parts
<LDraw>\models
<LDraw>\Unofficial\p
<LDraw>\Unofficial\parts

If we agree that the above order is correct, we can then discuss where to put it in the specification. If we want a different default search path, we can discuss that.

Note: If your name isn't Travis Cobbs, Chris Dee, or Roland Melkert, please don't respond to this thread. Read the rules for this board. Other than the three of us, only admins and mods can post here, but they aren't supposed to do so. If you want to bring up concerns, do so in the Parts Authoring board.

That sequence seems logical to me.
Chris (LDraw Parts Library Admin)
Reply
RE: Standardize official search path
#3
(2021-03-14, 1:50)Travis Cobbs Wrote: MPD subfiles
Model Directory (the directory that the top-level model resides in)
<LDraw>\p
<LDraw>\parts
<LDraw>\models
I have no problem with this part (it basically just adds models to the spec.)

But I never really liked the following
Quote:<LDraw>\Unofficial\p
<LDraw>\Unofficial\parts

Because it breaks the 'package', also imho you should look in the unofficial location before the official one because part and primitives can be updated.
With the current order the old ones will still load.


I also would like to make clear this order is to be followed within the context of the currently begin processed ldraw file.

Meaning a stud.dat in the mpd's subfolder (or as a submodel) will not be used for each part in the scene.

It will only be used if it's actually referenced inside the mpd itself trough a "1 ..... stud.dat" line.

This because all references inside the mpd will be tested against the parent folder.

But once e.g. 3001.dat start loading it will look inside it's own folder (parts) first before going to the library p folder etc.

This will also ensure you can cache meshes.
Reply
RE: Standardize official search path
#4
(2021-03-15, 20:05)Roland Melkert Wrote:
(2021-03-14, 1:50)Travis Cobbs Wrote: MPD subfiles
Model Directory (the directory that the top-level model resides in)
<LDraw>\p
<LDraw>\parts
<LDraw>\models
I have no problem with this part (it basically just adds models to the spec.)

But I never really liked the following
Quote:<LDraw>\Unofficial\p
<LDraw>\Unofficial\parts

Because it breaks the 'package', also imho you should look in the unofficial location before the official one because part and primitives can be updated.
With the current order the old ones will still load.

I'm OK if Unofficial is left out of the official spec, but I am not OK with putting it somewhere else in the spec. (Also, I'd prefer if it were documented in the spec as an optional pair of paths that can be checked after everything else.) The Unofficial directory was something that I created for LDView. If LDView cannot find a part, it checks ldraw.org for an unofficial part. If it finds it, it downloads it and puts it into Unofficial. Note that this was not designed to allow users to have the latest possible unofficial version of existing parts. It was designed purely to find (and auto-download) missing parts. LDView never deletes things it has downloaded into Unofficial. However, since Unofficial comes at the end of the search path, that doesn't really matter, since as soon as a part becomes official, the official version will be loaded instead of the unofficial version. (LDView also keeps track of when it downloaded unofficial files so that after a configurable time has passed, it can check to see if the file has updated. I only does this if it loads an unofficial file due to not finding it anywhere else in the search path.)


(2021-03-15, 20:05)Roland Melkert Wrote: I also would like to make clear this order is to be followed within the context of the currently begin processed ldraw file.

Meaning a stud.dat in the mpd's subfolder (or as a submodel) will not be used for each part in the scene.

It will only be used if it's actually referenced inside the mpd itself trough a "1 ..... stud.dat" line.

This because all references inside the mpd will be tested against the parent folder.

But once e.g. 3001.dat start loading it will look inside it's own folder (parts) first before going to the library p folder etc.

This will also ensure you can cache meshes.

I'm not sure I completely understand what you are saying. My expectation (based on the proposed search path) is that if the user puts a custom stud.dat in the directory alongside the main model (or inside the mpd), the proposed path order (and what I feel is correct) would use that stud.dat for the entire model. LDView doesn't support having two different versions of the same filename at once. Once LDView loads a file (from disk, or while processing an MPD), that loaded copy goes into its in-memory cache, and all future references to that filename come from the cache. The cache isn't hierarchical either (other than that if the type 1 line includes any slashes, those are part of the filename key in the cache).

Just to be clear, I'm not saying that we have to define the spec based on how LDView currently behaves. I am saying that I think it behaves fine, though. If we define the spec to be different, we should do so because we don't think the way I proposed is right.

Your last sentence seems to say that you are arguing in favor of what LDView already does. However, the rest of that section is arguing for something else, which is why I said that I don't completely understand what you are saying.

In case I'm not clear here, LDView does the following:

  1. Starts at the model the user opens (the main model). Records the directory that this model resides in for use in search path lookups.
  2. Reads the model file.
  3. If the model file is an MPD, parses it enough to know what MPD submodels are inside. Each of these goes into its global "loaded models" cache keyed off of the filename in the 0 FILE line.
  4. Second pass full parse of the file (as well as parsing the MPD submodels). For each type 1 line, it first checks to see if there is an entry in its global "loaded models" cache matching the filename portion of the type 1 line. If not, it uses the search path to find the specified file and jumps back to step 2 for that file, then loads it and inserts it into the global "loaded models" cache based on the type 1 filename, then references the result. If the model was already in the global "loaded models" cache, it instead references the cache entry.
Reply
RE: Standardize official search path
#5
Travis Cobbs Wrote:It was designed purely to find (and auto-download) missing parts. LDView never deletes things it has downloaded into Unofficial. However, since Unofficial comes at the end of the search path, that doesn't really matter
If that is the intend of those folders I'm ok with it.

Travis Cobbs Wrote:My expectation (based on the proposed search path) is that if the user puts a custom stud.dat in the directory alongside the main model (or inside the mpd), the proposed path order (and what I feel is correct) would use that stud.dat for the entire model.
I disagree on this, a stud.dat should not replace all studs in the whole scene if it's located in the mpd itself or its folder.

For example d:\test contains only these two files.
d:\test\awsomeModel.mpd
d:\test\stud.dat

The mpd contains:

Code:
0 FILE main.ldr
1 4 0 0 0 1 0 0 0 1 0 0 0 1 3001.dat
1 4 0 0 0 1 0 0 0 1 0 0 0 1 myPart.dat

0 FILE myPart.dat
1 16 0 0 0 1 0 0 0 1 0 0 0 1 stud.dat

In this case the references should translate like so:

For main.ldr
1 4 0 0 0 1 0 0 0 1 0 0 0 1 3001.dat  --> <lib>\parts\3001.dat
1 4 0 0 0 1 0 0 0 1 0 0 0 1 myPart.dat  -->  <mpd>\myPart.dat

Then loading for 3001.dat starts within its own context, meaning it does nothing with the mpd content nor files inside that mpd's folder.
So any stud.dat will be loaded as <lib>\p\stud.dat  unless 3001.dat has an embeded stud.dat (should never happen but could have been edited.) or another stud.dat is also present in the <lib>\parts folder.

Then for myPart.dat it will translate like
1 16 0 0 0 1 0 0 0 1 0 0 0 1 stud.dat  --> d:\test\stud.dat  (because its in the current context's parent folder 'd:\test'.)

Hope this make it more clear on how I always assumed search resolving should go.

The real question here is do we WANT a local stud.dat to affect the global scope or should it only affect its local context (like I assumed).

imho the local context approach is much more logical, global stuff should be handled by prefixing extra search locations to the global order.

edit: I assumed 3001.dat has no further dependencies. Otherwise it would recursively keep applying the context approach)
Reply
RE: Standardize official search path
#6
As mentioned in the thread that's open to public content, I still feel that my proposed behavior is in fact the intended behavior.
Reply
RE: Standardize official search path
#9
(2021-03-15, 20:05)Roland Melkert Wrote: Because it breaks the 'package', also imho you should look in the unofficial location before the official one because part and primitives can be updated.
With the current order the old ones will still load.

It would be wrong, and dangerous, to look in the Unofficial library before the official one. A updated primitive could be added to the unofficial library without corresponding changes to the other files in which it is used. The Parts Update build process should ensure that the Official lbrary has a coherent set of dependent files - updated primtives and subfiles are only released along with the necessary concommitant changes to their parents.
Chris (LDraw Parts Library Admin)
Reply
RE: Standardize official search path
#7
[SteeCo hat, Admin hat] ON

Please bring this to an end and ratify.

[SteeCo hat, Admin hat] OFF

w.
LEGO ergo sum
Reply
RE: Standardize official search path
#8
(2021-03-25, 20:15)Willy Tschager Wrote: [SteeCo hat, Admin hat] ON

Please bring this to an end and ratify.

[SteeCo hat, Admin hat] OFF

I haven't had the time to research the 'defacto' scoping behavior in the wild some more.

So maybe we should only vote on the folder order, as the scoping stuff really shouldn't be rushed.

I'm ok with:

<LDraw>\p
<LDraw>\parts
<LDraw>\models
<LDraw>\Unofficial\p
<LDraw>\Unofficial\parts

Given <LDraw> is a 'library', meaning you could have multiples of those.
Reply
« Next Oldest | Next Newest »



Forum Jump:


Users browsing this thread: 1 Guest(s)