LDraw.org Discussion Forums
Proposal: New !DATA meta-command for embedded textures - Printable Version

+- LDraw.org Discussion Forums (https://forums.ldraw.org)
+-- Forum: Administrative (https://forums.ldraw.org/forum-4.html)
+--- Forum: Standards Board (https://forums.ldraw.org/forum-5.html)
+--- Thread: Proposal: New !DATA meta-command for embedded textures (/thread-23683.html)

Pages: 1 2 3


Proposal: New !DATA meta-command for embedded textures - Travis Cobbs - 2019-09-23

Note: this is in response to this thread.

This is a proposal to add a new !DATA meta-command that can be used to embed texture data into model files (for use with unofficial parts in an OMR model). The proposed specification for the new meta-statement follows. Please discuss.

Purpose

This document describes the !DATA meta-statement, which contains a set of commands for embedding binary data into LDraw files.

New meta-statement token

!DATA

This is the main data-embedding meta-command.

Reuse of existing meta-statement token

!:

This will be used to specify a BASE-64 encoded data stream. This token was introduced with the !TEXMAP meta-statement, and will be reused here for three reasons: its reuse will not interfere with its use in !TEXMAP; it is useful for the same reasons that it was useful in !TEXMAP; and !DATA is only useful right now for embedding texture data for use with !TEXMAP, so the two meta-statements are related to each other.

Syntax

0 !DATA START
0 !: <data1>
0 !: <data2>
...
0 !: <datan>
0 !DATA END

Description

This meta-statement allows for BASE-64 encoded data to be used inside a multi-part-dat (MPD) file. The decoded binary data replaces of the embedded MPD sub-file.
  1. This meta-statement must come after the 0 FILE meta-statement signifying a sub-file in a multi-part-dat (MPD) file.
  2. The FILE that the DATA belongs to must not contain any geometry.
  3. The FILE must contain one and only one DATA section.
  4. The FILE may contain header information.
  5. All characters after 0 !: that are not valid BASE-64 characters must be ignored. (This is intended to allow white-space characters to be placed in the line, but still applies to all other characters that are not valid BASE-64 characters.)
  6. Each 0 !: line except for the last must contain exactly 76 encoded characters.
  7. The last 0 !: line must contain no more than 76 encoded characters.
Example

0 FILE main.ldr
0 Author: Roland Melkert
1 15  -40 -20 0  1 0 0  0 0 1  0 1 0 plane.dat

0 FILE plane.dat
0 plane texmap test
0 UNOFFICIAL PART
0 BFC NOCERTIFY

0 !TEXMAP START PLANAR  0 0 0  40 0 0  0 0 40  checker123yyz.png
4 16  0 0 0  40 0 0  40 0 40  0 0 40
0 !TEXMAP END

0 FILE checker123yyz.png
0 !DATA START
0 !:iVBORw0KGgoAAAANSUhEUgAAAQAAAAEACAYAAABccqhmAAAABmJLR0QA/wD/AP+gvaeTAAAACXBI
0 !:WXMAAC4jAAAuIwF4pT92AAAAB3RJTUUH3gcbDhUzUyNxeQAAArVJREFUeNrt3cENwzAMBMFjoM7d
0 !:p6vIX2lBeRECZzo4GFjIL9ZOdhpVeu3dOj9VZb/9bT4BxhIAEABAAAABAAQAEABAAAABAAQAEABA
0 !:AAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEADgX5Wk9UC6+/T22+8F
0 !:AAgAIACAAAACAAgAIACAAAACAAgAIACAAAACAAgAIACAAAACAAgACAAgAIAAAAIACAAgAIAAAAIA
0 !:CAAgAIAAAAIACABwgeU+u/32z93vBQB+AQABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQA
0 !:EABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAI4t99l79z9p3v/6/pP3ewGAXwBAAAAB
0 !:AAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQA
0 !:EABAAIBjy3324fu/8f0H7/cCAL8AgAAAAgAIACAAgAAAAgAIACAAgAAAAgAIACAAgAAAAgAIACAA
0 !:gAAAAgAIACAAgAAAAgAIACAAgAAAAgAIACAAgAAAxypJ64H06ffZ7bffCwAQAEAAAAEABAAQAEAA
0 !:AAEABAAQAEAAAAEABAAQAEAAAAEABAAQABAAQAAAAQAEABAAQAAAAQAEABAAQAAAAQAEABAA4ArL
0 !:fXb77Z+73wsA/AIAAgAIACAAgAAAAgAIACAAgAAAAgAIACAAgAAAAgAIACAAgAAAAgAIACAAgAAA
0 !:AgAIACAAgAAAAgAIACAAgAAAAgAcqyStB9Ldp7fffi8AQAAAAQAEABAAQAAAAQAEABAAQAAAAQAE
0 !:ABAAQAAAAQAEABAAQABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAIAL/AB/0mJQmotO+gAA
0 !:AABJRU5ErkJggg==
0 !DATA END


RE: Proposal: New !DATA meta-command for embedded textures - Roland Melkert - 2019-09-25

So basically this is a encoded sub-subFile?

If so why limited to one block per mpd subfile?


I would propose to ether:....


EDIT: sorry based my comment on the wrong version.


RE: Proposal: New !DATA meta-command for embedded textures - Roland Melkert - 2019-09-25

(2019-09-23, 23:10)Travis Cobbs Wrote: Note: this is in response to this thread.

I like this proposal, only thing I would like to add is to maybe use the DATA meta itself as an alternative to the FILE meta.

That way you won't end up with empty models in non supported programs (they would just ignore the unknown 0 lines, as if they were part of the current FILE meta's content).

Also you wont need the "0 !DATA END" line as normal FILE rules would apply (meaning the next FILE or !DATA meta will split subfiles)


Code:
0 FILE main.ldr
0 Author: Roland Melkert
1 15  -40 -20 0  1 0 0  0 0 1  0 1 0 plane.dat

0 FILE plane.dat
0 plane texmap test
0 UNOFFICIAL PART
0 BFC NOCERTIFY

0 !TEXMAP START PLANAR  0 0 0  40 0 0  0 0 40  checker123yyz.png
4 16  0 0 0  40 0 0  40 0 40  0 0 40
0 !TEXMAP END

0 !DATA checker123yyz.png
0 !:iVBORw0KGgoAAAANSUhEUgAAAQAAAAEACAYAAABccqhmAAAABmJLR0QA/wD/AP+gvaeTAAAACXBI
0 !:WXMAAC4jAAAuIwF4pT92AAAAB3RJTUUH3gcbDhUzUyNxeQAAArVJREFUeNrt3cENwzAMBMFjoM7d
0 !:p6vIX2lBeRECZzo4GFjIL9ZOdhpVeu3dOj9VZb/9bT4BxhIAEABAAAABAAQAEABAAAABAAQAEABA
0 !:AAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEADgX5Wk9UC6+/T22+8F
0 !:AAgAIACAAAACAAgAIACAAAACAAgAIACAAAACAAgAIACAAAACAAgACAAgAIAAAAIACAAgAIAAAAIA
0 !:CAAgAIAAAAIACABwgeU+u/32z93vBQB+AQABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQA
0 !:EABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAI4t99l79z9p3v/6/pP3ewGAXwBAAAAB
0 !:AAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAAABAAQA
0 !:EABAAIBjy3324fu/8f0H7/cCAL8AgAAAAgAIACAAgAAAAgAIACAAgAAAAgAIACAAgAAAAgAIACAA
0 !:gAAAAgAIACAAgAAAAgAIACAAgAAAAgAIACAAgAAAxypJ64H06ffZ7bffCwAQAEAAAAEABAAQAEAA
0 !:AAEABAAQAEAAAAEABAAQAEAAAAEABAAQABAAQAAAAQAEABAAQAAAAQAEABAAQAAAAQAEABAA4ArL
0 !:fXb77Z+73wsA/AIAAgAIACAAgAAAAgAIACAAgAAAAgAIACAAgAAAAgAIACAAgAAAAgAIACAAgAAA
0 !:AgAIACAAgAAAAgAIACAAgAAAAgAcqyStB9Ldp7fffi8AQAAAAQAEABAAQAAAAQAEABAAQAAAAQAE
0 !:ABAAQAAAAQAEABAAQABAAAABAAQAEABAAAABAAQAEABAAAABAAQAEABAAIAL/AB/0mJQmotO+gAA
0 !:AABJRU5ErkJggg==



RE: Proposal: New !DATA meta-command for embedded textures - Travis Cobbs - 2019-09-25

(2019-09-25, 0:51)Roland Melkert Wrote: I like this proposal, only thing I would like to add is to maybe use the DATA meta itself as an alternative to the FILE meta.

That way you won't end up with empty models in non supported programs (they would just ignore the unknown 0 lines, as if they were part of the current FILE meta's content).

Also you wont need the "0 !DATA END" line as normal FILE rules would apply (meaning the next FILE or !DATA meta will split subfiles)

Would having a sub-file in an MPD with no geometry really hurt anything in programs that don't support !DATA?

I can see plus and minus points both ways. Having !DATA behave just like FILE is arguably cleaner, since it doesn't involve replacing a file that is being loaded with completely different contents. But putting it inside a FILE is much less of a departure from established practice. Putting it inside a FILE allows for header information to go in. However, I'm not sure if this is a plus or a minus, since these are intended as a drop-in for actual texture files, and such files obviously cannot contain LDraw header information.

My general leaning is toward having !DATA go inside a FILE (as indicated in the proposal). However, I'm not strongly in favor of that, so if you and Chris both feel that !DATA should be used instead of FILE, I can go with that.


RE: Proposal: New !DATA meta-command for embedded textures - Roland Melkert - 2019-09-25

(2019-09-25, 2:07)Travis Cobbs Wrote: Would having a sub-file in an MPD with no geometry really hurt anything in programs that don't support !DATA?
Not during viewing.

But most editors have a list of submodels somewhere.

Navigating to them will present an empty model which you can then 'mess up' if not careful.

And doing so would break the mpd for software which does understand the new meta.


RE: Proposal: New !DATA meta-command for embedded textures - Willy Tschager - 2019-11-19

Could you please bring this to an end?

Thx, w.


RE: Proposal: New !DATA meta-command for embedded textures - Travis Cobbs - 2019-11-19

(2019-09-25, 19:13)Roland Melkert Wrote: Not during viewing.

But most editors have a list of submodels somewhere.

Navigating to them will present an empty model which you can then 'mess up' if not careful.

And doing so would break the mpd for software which does understand the new meta.

I don't understand your final point, but I'll bow to your superior wisdom with respect to editors. (Note, since it's often not obvious in posts, I am not being sarcastic.) Please write up a proposal (like mine above) for your proposed version of !DATA, and we can discuss it, and then vote on it.


RE: Proposal: New !DATA meta-command for embedded textures - Travis Cobbs - 2020-01-15

(2019-11-19, 18:25)Travis Cobbs Wrote: I don't understand your final point, but I'll bow to your superior wisdom with respect to editors. (Note, since it's often not obvious in posts, I am not being sarcastic.) Please write up a proposal (like mine above) for your proposed version of !DATA, and we can discuss it, and then vote on it.

Unless I'm misunderstanding, I think your are suggesting we go with my original proposal. Since you were the one to suggest using MPD submodels in the first place I want to make sure you haven't forgotten your original reasoning.


RE: Proposal: New !DATA meta-command for embedded textures - Roland Melkert - 2020-01-15

(2020-01-15, 2:09)Travis Cobbs Wrote: Unless I'm misunderstanding, I think your are suggesting we go with my original proposal. Since you were the one to suggest using MPD submodels in the first place I want to make sure you haven't forgotten your original reasoning.

I was trying to make it an hybrid.

With !DATA acting like FILE non supporting editors will ignore it all, and supporting ones could hide those subfiles and/or give give them special treatment while reusing the search mechanism already in place for the FILE meta.

Side note, should we include the mime type of the encoded data? "image/png" being the only one officially supported from the get go? 

I can imagine adding more stuff to mpd's (e.g. LDCad scripts Blush  )


RE: Proposal: New !DATA meta-command for embedded textures - Orion Pobursky - 2020-01-15

(2020-01-15, 19:11)Roland Melkert Wrote: Side note, should we include the mime type of the encoded data? "image/png" being the only one officially supported from the get go?

This actually worries me a bit. LDraw file could, potentially, be a security risk making me want to scrap the idea completely.