[split] Buffer Exchange cleanup script

[N. W. Perry]

(background: Trying to write an LDCad script that will comment out all type 1 lines that are hidden by buffer exchange, so the model can be viewed in e.g. LDView…)

Hmm, so the logic of buffer exchange is a little trickier than I thought. It's easy enough when there's just one block of store/retrieve statements—just hide everything between the first store and the last retrieve. But when there are multiple hidden blocks with some visible lines in between them, it's harder.

I might need a hint. :-)

[Roland Melkert]

Hmm, so the logic of buffer exchange is a little trickier than I thought. It's easy enough when there's just one block of store/retrieve statements—just hide everything between the first store and the last retrieve. But when there are multiple hidden blocks with some visible lines in between them, it's harder.

I'm not completely sure without testing myself (been awhile since I wrote the relevant code) but I think you can use ref:getVisible to test if the line is currently visible inside LDCad's editor.

Downside is this function is only available on reference lines, and as you are looping all lines you will probably be working with the generic srcLine object.

You can determine a line to be a reference line by using srcLine:isRef but this still doesn't give you access to a refLine object. (you just gave me the idea to add an 'asRefLine' function to srcLine )

A possible workaround for this could be to also load all references and setup a mapping table to find the reference with the same getLineIndex value the current srcLine has.

Alternative to all this would be to implement the buffer exchange rules while looping the lines.

[N. W. Perry]

Alternative to all this would be to implement the buffer exchange rules while looping the lines.

Well, that's where I'm at already. But I didn't think about trying getVisible—because obviously LDCad already knows the rules of buffer exchange, so it would be nice not to have to figure them out myself!

Downside is this function is only available on reference lines, and as you are looping all lines you will probably be working with the generic srcLine object.

I don't have to handle every line, or do I? The only lines I need to manipulate would be the ref lines.

[N. W. Perry]

I'm not completely sure without testing myself (been awhile since I wrote the relevant code) but I think you can use ref:getVisible to test if the line is currently visible inside LDCad's editor.

So, either this doesn't work or I did it wrong (the latter is most likely!). Using the test function below, which should print only the currently visible ref lines, I instead get all of the ref lines:

Code:

function printVis()

  local sf=ldc.subfile()
  if not sf:isLinked() then
    ldc.dialog.runMessage('No active session')
    return
  end

  local cnt=sf:getRefCount()

  for i=1, cnt do
    local ref=sf:getRef(i)
    if ref:getVisible() then
      print(ref)
    end
  end

end

On your API guide, it does say that getVisible only pertains to animations, so perhaps it doesn't reflect buffer exchange status after all?

---

Attached here is a test file I'm using. If my script is successful, the finished model will display only the white pyramid, not any of the red or blue bricks, when opened in any LDraw viewer/editor.

  bufExTest.dat (Size: 2.21 KB / Downloads: 1)

[N. W. Perry]

Well, now that my son (and his Christmas Minecraft™ LEGO) has gone off to bed, I think I have worked out the basic logic for this problem.

The key, I think, is to loop the lines in reverse. And since this script is only intended to display the model in its final state, it doesn't have to apply the full rules of buffer exchange and account for all the different visibility changes as you step through the model.

On the first loop, we would look for the first instance of a BUFEXCHG RETRIEVE statement (being the last one in the file). When we find one, we would return its buffer ID (A, B, C, etc.) since only the lines stored in that buffer will display at the final step of the model.

Then we loop the lines a second time, checking whether each line is a BUFEXCHG STORE or RETRIEVE statement—and if so, whether it matches the active buffer ID—or a type 1 reference line. Assuming a starting condition of "true", we would store any type 1 lines in an array. If we encounter a RETRIEVE statement with the active ID, we would toggle the condition to "false". The next ref lines we encounter would then not be stored to the array, since they would be hidden in the final model, until we reach a STORE statement with the active ID. This toggles the condition back to "true", and so on until the beginning of the file is reached.

Finally, we would apply a function to any type 1 lines that are not stored in the array, to comment them out. (Or maybe it makes more sense to change the starting condition to "false" and apply the function to the lines that are in the array.)

This, I think, should do the trick—at least for the main subfile. (But it should be no problem to simply loop through each subfile separately, right?—perhaps on the condition that the subfile isn't called by a ref line that's being commented out…)

[N. W. Perry]

On the first loop, we would look for the first instance of a BUFEXCHG RETRIEVE statement (being the last one in the file). When we find one, we would return its buffer ID (A, B, C, etc.) since only the lines stored in that buffer will display at the final step of the model.

Progress…I realized that only taking the final buffer ID won't work, because the active "final" buffer can change within a file (as I found in one of my models, where I retrieved A but stored to B…).

Turns out, this actually makes things easier. Now, given the initial visibility state of true, we look for a RETRIEVE statement, and when we find one we simply take its ID, and switch the visibility to false. This causes the script to ignore any further RETRIEVE statements, and now looks for a STORE statement with a matching ID. This turns the visibility true again, and so when we next find a RETRIEVE statement, we take its ID again, in case it is different, and so on…

I've tested this on a few models with complex buffer exchange statements, and the results seem to be correct. The resulting script is below—right now, it only prints the visible lines (or their index) in the console; the next step is to actually toggle them as comments.

If anyone has any models that use a lot of buffer exchange, I'd love to have some test files!

Code:

genTools=require('genTools')

function printVis()
  local ses=ldc.session()
 
  if not ses:isLinked() then
    ldc.dialog.runMessage('No active session')
    return
  end
 
  local sf=ldc.subfile()
  local cnt=sf:getLineCount()
  local lines={}
  local isVis=true
 
  for i=cnt, 1, -1 do
    local line=sf:getLine(i)
    local index=line:getLineIndex()
    local strBeg=line:getString():sub(1,11):upper()
    local strEnd=line:getString():sub(13):upper()
    local strInit=line:getString():sub(12,12):upper()
    local isBufEx=strBeg=='0 BUFEXCHG '
    local isStor=isBufEx and strEnd==' STORE'
    local isRetr=isBufEx and strEnd==' RETRIEVE'
   
    if isRetr then
      if isVis then
        id=strInit
      end
      isVis=false
    elseif isStor and strInit==id then
      isVis=true
    elseif line:isRef() and isVis then
      table.insert(lines, index)
    end
  end
 
  for k, v in ipairs(lines) do
    print(v)
  end
end

function onRun()
  printVis()
end

function register()
  local macro=ldc.macro('Print visible')
  macro:setEvent('run', 'onRun')
end

register()

[Roland Melkert]

So, either this doesn't work or I did it wrong (the latter is most likely!).

This is a bug.

I checked the code and getVisible should indeed return a value which takes bufferexchange into account.

It seems to use outdated functions to do so though.

I will change this so it will only returns true when the part is currently visible (depending on current step, bufexchg, ghosting, etc).

Same goes when running in animation mode.

On your API guide, it does say that getVisible only pertains to animations, so perhaps it doesn't reflect buffer exchange status after all?

The api guide is very old, The function was changed for use with macros somewhere down the road.

[N. W. Perry]

This is a bug.

I checked the code and getVisible should indeed return a value which takes bufferexchange into account.

It seems to use outdated functions to do so though.

I will change this so it will only returns true when the part is currently visible (depending on current step, bufexchg, ghosting, etc).

Same goes when running in animation mode.


The api guide is very old, The function was changed for use with macros somewhere down the road.

That should allow the script to work at any step in the model, then. So you could, for example, export to Studio and do photoreal renderings by step.

Nevertheless, I'm glad I went through the exercise of figuring out how to do it logically. The solution turned out to be surprisingly elegant. :-)

Is there a way to loop through all subfiles of a model, so that the script can work recursively?

[Roland Melkert]

Is there a way to loop through all subfiles of a model, so that the script can work recursively?

You can get submodels trough the ref obect's getSubfile function.

But in the current alpha there is no (easy) way to know if it is a part or submodel.

The upcoming 1.7 Beta 1 has a whole bunch of type information functions though.

Quick copy/paste from the current code:

Code:

     
static int luaCB_isPart(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 0, true); }
static int luaCB_isRealPart(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 1, true); }
static int luaCB_isColourPart(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 2, true); }
static int luaCB_isShortCut(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 3, true); }
static int luaCB_isModel(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 4, true); }
static int luaCB_isGenerated(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 5, true); }
static int luaCB_isOfficialPart(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 6, true); }
static int luaCB_isMoved(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 7, true); }
static int luaCB_isAliasPart(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 8, true); }
static int luaCB_isMissing(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 9, true); }
static int luaCB_isRedirectedPart(lua_State *luaState) { return singleBoolParamIO(luaState, luaTypeName, 10, true); }

[N. W. Perry]

You can get submodels trough the ref obect's getSubfile function.

But in the current alpha there is no (easy) way to know if it is a part or submodel.

The upcoming 1.7 Beta 1 has a whole bunch of type information functions though.

Hmm, that might be worth waiting for then…sounds like I'd have to, while I'm already looping the lines, also check whether each ref is a model and, if so, whether it has its own buffer exchange applied.

Alternate would be if I could just work at the file level—that is, apply the script to every line of the .ldr or .mpd file, and not be concerned with whether something is a submodel or not. Maybe I can already do that using io. scope? Or maybe it doesn't work that way…