Datsville

[Roland Melkert]

Hi all, I've been playing around with datsville.

But somehow I thought it was bigger, I'm using the version from datsville.googlecode.com.

Anyone has a newer / bigger version?

Also why is everything stuffed in a gigantic MPD (33000 lines!) Not very efficient for use in svn.

[Tore Eriksson]

Roland Melkert Wrote:
-------------------------------------------------------
> Hi all, I've been playing around with Datsville.
>
> But somehow I thought it was bigger, I'm using the
> version from datsville.googlecode.com.
>
> Anyone has a newer / bigger version?
>
> Also why is everything stuffed in a gigantic MPD
> (33000 lines!) Not very efficient for use in svn.


I'm really glad you brought this subject up. It's still my favourite LDraw subject, even though I once again have taken a long break from the work. This week for example, I've shown LDraw, Datsville, and my 7 seconds animation for four of my friends. It sort of inspired me a little, almost enough to pick up the work again. Let's say I'm very close to a relapse now.

This is my unofficial version of Datsville, from last Christmas. http://dl.dropbox.com/u/15954981/rev038.zip
I think it is the latest, but I haven't checked if there are some half-finished things I was into when I got caught up in other stuff again... What you need to know is that I've replaced lots of parts with boxed versions of dat files (for example B\3001.dat instead of 3001.dat) where it shouldn't make any difference. You'll have to create the folder 'B' in your LDraw\Parts folder, and unpack the content of Boxed_dats...zip into LDraw\Parts\B to make this version of Datsville renderable.

I would really need some helping hands, as I felt I'm losing my inspiration and along with that my creativity. There has been a lot of grunt work, just filling areas with grass and sidewalks. Anyone who wants to join this project will get a lot of power and quite free hands to add models and make changes in the layout if needed. I still wish to have the final veto, but I may hand that over too if somebody with the right energy and visions wishes to take over.

http://dl.dropbox.com/u/15954981/rev038.zip

/Tore

[Roland Melkert]

Wow thanks for this version.

Thanks for the offer but I'm not really a modeller, but always admired datsville for it's bigness

I also like to use it to test my LDraw software, this version unveiled some problems with my current renderer in regards to 'boxed' parts (it doesn't recognize them as being parts, and messes up lighting some how.).

Those boxed parts are they hand generated? or do you use some software for them. Ether way I think general support for them would improve if you added at least "0 !LDRAW_ORG Unofficial_Part" as a first line.

Anyhow it's a very clever idea to eliminate unneeded geometry, I'm getting ~1.5 fps out of town.ldr

Maybe I could help the project by adding special support for e.g. boxed parts to my editor, any thoughts on that?

[Tore Eriksson]

Roland Melkert Wrote:
-------------------------------------------------------
> Wow thanks for this version.

Thank you for showing some interest in it. However, when I checked it after zipping it and posting the link here, I noticed that there is a couple of references to non-existing parts or models. I have to check that up tonight.

> I also like to use it to test my LDraw software,
> this version unveiled some problems with my
> current renderer in regards to 'boxed' parts (it
> doesn't recognize them as being parts, and messes
> up lighting some how.).

What lighting? The parts made by DatShine, or?

> Those boxed parts are they hand generated?

Some are, some are not. It depends on the part's complexity. Most Basic bricks, plates, and tiles are made by a very simply DOS program I rushed together.

> or do you use some software for them.

Of course, I use LDraw Boxer to automatically determine where parts can be replaced by boxes. But I guess that wasn't the question.

> Ether way I think general support for them would improve if you
> added at least "0 !LDRAW_ORG Unofficial_Part" as
> a first line.

Hmm, I think you're right. As I have experienced almost no general support whatsoever for my ideas and my objections, and that those standards constantly change, I haven't even cared about what currently is required as long as it works for the softwares I use.

> Anyhow it's a very clever idea to eliminate
> unneeded geometry, I'm getting ~1.5 fps out of
> town.ldr

Thank you!

> Maybe I could help the project by adding special
> support for e.g. boxed parts to my editor, any
> thoughts on that?

That would be really great of course! Normally, it shouldn't be necessary to add any special support as I tried to find a system that just works without any manual hands-on.

If you mean your editor may have the option to for example right-click on a part or a group of parts, and you may manually select their state as "Standard", "Boxed", "No top studs", "No bottom details" - that would be really awesome!

When it comes to your editor demanding the exact line "0 !LDRAW_ORG Unofficial_Part" to treat an object as a part, I find it a very vulnerable approach. It may be better to have a look at how Lars solved the issue in L3P 1.4B, as L3P somehow handles boxed parts as parts. I don't know if L3P regards all files with .DAT extension as parts (I don't think so, I'm not sure?) or any file placed in the LDraw\Parts path (except sub-parts of course!) as parts. That may be a less fragile and more future safe approach since the header standards constantly change.

/Tore

[Roland Melkert]

Tore Eriksson Wrote:
-------------------------------------------------------
> What lighting? The parts made by DatShine, or?

e.g. b\3001.dat has only a reference to box.dat, this isn't bad it will be drawn etc. But because it's threaded as a model instead of a part it won't correct for scaling etc in regards to normals. This results in all those bricks being gray instead of e.g. white in the whole city's rendering.

I'm rewriting the file id-er to circumvent this by promoting any files with non (-)1.0 determinants references, but because of rounding errors it decided to promote whole town0-2.ldr to a single part But with a fuzzy compare it seems to work without adjusting headers.

>
snip
>
> If you mean your editor may have the option to for
> example right-click on a part or a group of parts,
> and you may manually select their state as
> "Standard", "Boxed", "No top studs", "No bottom
> details" - that would be really awesome!
>

I was thinking about special part bin groups and replacement actions on the selection.

> When it comes to your editor demanding the exact
> line "0 !LDRAW_ORG Unofficial_Part" to treat an
> object as a part, I find it a very vulnerable
> approach. It may be better to have a look at how
> Lars solved the issue in L3P 1.4B, as L3P somehow
> handles boxed parts as parts. I don't know if L3P
> regards all files with .DAT extension as parts (I
> don't think so, I'm not sure?) or any file placed
> in the LDraw\Parts path (except sub-parts of
> course!) as parts. That may be a less fragile and
> more future safe approach since the header
> standards constantly change.
>
> /Tore

It all depends what the software in question wants to do with it. l3p and povray wouldn't care because they don't use normals the way e.g. OpenGL needs them, so it's a non issue for those apps.

I need to know what's a part and what not in order to optimize the 3D data speed and memory wise. For example loading the town.ldr in my editor eats 'only' about 400MB of memory but still renders at less then a second.

So depending on the rendering method knowing if something is a part or model is very valuable information (real-time) rendering speed and quality wise.

[Tore Eriksson]

Roland Melkert Wrote:
-------------------------------------------------------
>
> It all depends what the software in question wants
> to do with it. l3p and povray wouldn't care
> because they don't use normals the way e.g. OpenGL
> needs them, so it's a non issue for those apps.

This is way beyond my skills, so I just have to trust you on that. But I know that L3P needs to know whether it deals with a part or a model, because it produces seams between parts but not between models. And it's capable of identifying a part even without the information from the header.

> I need to know what's a part and what not in order
> to optimize the 3D data speed and memory wise. For
> example loading the town.ldr in my editor eats
> 'only' about 400MB of memory but still renders at
> less then a second.

That's fantastic! I really want to keep my library of boxed DATs as small as possible, but this makes it so interesting that I might let it explode in size (and thus, the POV output files generated by L3P...) by adding those so far not needed extra lines.

Besides the file size factor, I've been hesitating to use the Name !LDRAW_ORG in vain for my own private, unofiffcial files. My understanding of "0 !LDRAW_ORG Unofficial_Part" is:
* Property of LDraw.org. You may use it under this and that Licence Agreement if you agree with this and that.
* This file meets header standards and all other silly specs the LSC have come up with. At least it did when released.
* This part is in the Parts Tracker, waiting to be part of the Official LDraw Parts Library.
* In other words: Don't use this line at home! It is reserved for a few chosen Admins trusted with it.
No, that's not really my understanding of it. But it's sure what it's signaling.

And I'm prepared to suppress those feeling if needed for the good sake, if necessary to make your editor support boxed parts.

> So depending on the rendering method knowing if
> something is a part or model is very valuable
> information (real-time) rendering speed and
> quality wise.

And, like I said, if the main file resides somewhere in LDraw\Parts, that information itself should be sufficient(?) to determine that you're dealing with a part.

/Tore

[Roland Melkert]

Tore Eriksson Wrote:
-------------------------------------------------------
> I really want to keep my library
> of boxed DATs as small as possible, but this makes
> it so interesting that I might let it explode in
> size (and thus, the POV output files generated by
> L3P...) by adding those so far not needed extra
> lines.

I've got it working now without the need for header information, but it's still guessing and so it can guess wrong, like for example the town0-1.ldr has this line:

1 7 540 0 -800 0 0 0.50 0 0.50 0 -0.50 0 0 baihous1.ldr

The whole house is scaled down (to datsville scale I guess?). This will cause my 'improved' guessing code to handle the whole town0-1.ldr as a single part. But it can be corrected by fixing the header of town0-1.ldr only instead of all the boxed parts.

>
> Besides the file size factor, I've been hesitating
> to use the Name !LDRAW_ORG in vain for my own
> private, unofiffcial files. My understanding of "0
> !LDRAW_ORG Unofficial_Part" is:
> * Property of LDraw.org. You may use it under this
> and that Licence Agreement if you agree with this
> and that.
> * This file meets header standards and all other
> silly specs the LSC have come up with. At least it
> did when released.
> * This part is in the Parts Tracker, waiting to be
> part of the Official LDraw Parts Library.
> * In other words: Don't use this line at home! It
> is reserved for a few chosen Admins trusted with
> it.
> No, that's not really my understanding of it.
> But it's sure what it's signaling.
>

You right about the !LDRAW_ORG it implies it's part of the library official or not. But as far I can find there is no other way of marking the nature of a ldraw file. I think I'm going to bring this up in the LSC.

> And, like I said, if the main file resides
> somewhere in LDraw\Parts, that information itself
> should be sufficient(?) to determine that you're
> dealing with a part.
>

You are right but there are exceptions for example shortcuts threading them as parts will double used resources for the referenced 'real' part, and don't get me started about the fixed color parts, they would plunder VGA resources like locus.
> /Tore

[Tore Eriksson]

The new revision of my unofficial Datsville is at:
http://cid-7183b586c107b55b.skydrive.liv.../Datsville

* Fixed minor problems with minifig arms and a misplaced BFC statement
* Updated MovedTo parts with parts from LDraw.org 2011-01 Parts Update

/Tore

[Roland Melkert]

Tore Eriksson Wrote:
-------------------------------------------------------
> The new revision of my unofficial Datsville is
> at:
> http://cid-7183b586c107b55b.skydrive.live.com/brow
> se.aspx/.Public/Datsville
>
> * Fixed minor problems with minifig arms and a
> misplaced BFC statement
> * Updated MovedTo parts with parts from LDraw.org
> 2011-01 Parts Update
>
> /Tore
Thanks I'll play with it tomorrow, in the mean time here are some screen caps from the current output in my editor.

Full town
Zoom in on shell station

I'm still playing with things like 'when not to render edges anymore (cause it becomes a gray mass in the distance that way). But I'm pretty content with it. Thanks for helping me improve my editor by supplying this insane model

[Tore Eriksson]

Roland Melkert Wrote:
>
> I've got it working now without the need for
> header information, but it's still guessing and so
> it can guess wrong, like for example the
> town0-1.ldr has this line:
>
> 1 7 540 0 -800 0 0 0.50 0 0.50 0 -0.50 0 0
> baihous1.ldr
>
> The whole house is scaled down (to datsville scale
> I guess?). This will cause my 'improved' guessing
> code to handle the whole town0-1.ldr as a single
> part. But it can be corrected by fixing the header
> of town0-1.ldr only instead of all the boxed
> parts.
>

For a human, the line:
0 Unofficial Datsville Town Sub-Model
says it all, but I understand it's a bit harder to make software with logic fuzzy enough to understand it.

If a header line contains the string "official" (disregard upper/lowercase), that line probably a part/model/primitive identifier.
- If one header line contains both "official" and "model" (disregard upper/lowercase), it's a model.
- If one header line contains both "official" and "part" - but not "sub-part"! (disregard upper/lowercase), it's a part.
If a file contains type 1 references to .LDR files, it's a model.
If a file contains references to parts, it's most likely a model.
If a file contains several line type 2 - 5 and references to primitives, it's likely a part.

But I also understand that if a program has to evaluate a lot of part files with this fuzzy logic, it takes more processing than it should have had to. I think I'll soon get back with a suggestion of a better alternative to that !LDraw_org meta-statement...

/Tore
(I've changed my mind about "If a file is located in the Parts folder...", since there are lots of shortcuts, like mc with rider, that have to be treated as models, not parts.)

[Chris Dee]

Most of those assembly Parts folder files, like the motorcycle+rider are anachronistic and probably would not be included if they were suggested today. We did reject the inclusion of the minifig-scale destroyer droid, IIRC. Backward compatibilty, and all that, means we can't really delete them. I suppose they could be moved to Models, as "usage examples", like car.dat and pyramid.dat.

So, I'm not sure you should define any algorithm that allows models to exist in the Parts folder.

Personally, I'm very worried about the proposed fuzzy logic. It's that kind of vague header formatting that convinced us (well me at least) to formalise the Parts file header as you see it today. This means that the format can be programmatically checked - and other tools can rely on the defined semantics.

Something like "0 !PRIVATE Unofficial_Set", would allow us, in the future to start issuing LDraw.org versions of models, using "0 !LDRAW_ORG Unofficial_Set" during review and "0 !LDRAW_ORG Official_Set" upon release.

[Tore Eriksson]

Chris Dee Wrote:
-------------------------------------------------------
> Most of those assembly Parts folder files, like
> the motorcycle+rider are anachronistic and
> probably would not be included if they were
> suggested today.

A great fortune they did make it there in time then!

> We did reject the inclusion of
> the minifig-scale destroyer droid, IIRC.

That's really bad. It would be very handy to have in the Parts folder where it belongs..

> Backward compatibilty, and all that, means we
> can't really delete them.

That is in deed very good news!

> I suppose they could be moved to
> Models, as "usage examples", like car.dat and
> pyramid.dat.

I suppose not!

> So, I'm not sure you should define any algorithm
> that allows models to exist in the Parts folder.

Shortcuts could techically be called "models", but from my POV they are much closer to be treated as parts than models, even though they really are neither.

> Personally, I'm very worried about the proposed
> fuzzy logic. It's that kind of vague header
> formatting that convinced us (well me at least) to
> formalise the Parts file header as you see it
> today. This means that the format can be
> programmatically checked - and other tools can
> rely on the defined semantics.

I think you worry too much. Let's keep those two things separate:
A. Shortcuts in the Parts folder are already formally defined
0 !LDRAW_ORG Shortcut UPDATE 2010-02 (30187c03.dat)
From the software's POV, it should be treated as a model.
But as an LDraw user, I DON'T want it on my own private workspace, ie the Models folder. No way!
B. I made the mistake to have no header at all in some of my boxed parts. This has been no problems with the software I've used so far, but Roland has made me aware that it may cause problems in some cases, like his new editor.

How do we address issue B? (Issue A is a non-issue IMO) The bad, really bad way would be a new, over-formalized Standard that you force us poor users to obey, or else we will encounter endless warnings and errors. Of course it will render all previously made models illegal, like DAT-Header now signals 'Hold' for all parts ending with the previouly compulsary "0" line. (Oh, stupidivity of stupidivities!) Now, where was I? I don't mind a new syntax, like "0 !PRIVATE Part", although I find it too easy to misunderstand for those who happen to have that kind of humour.

But I get very much more worried about the opposite ditch. Leaning too much on defined semantics, blaming the ignorant users for all syntax errors. Backwards compability should IMO also mean that software recognize parts and models made 5 or 10 years ago, without spitting out needless and annoying warnings and errors. They should accept and understand statements like:
0 Un-official Part
0 Un-official LDraw Primitive
0 Unofficial Shortcut
0 Unofficial Classic Space Model
0 Unofficial Moonbase Vehicle model
, regardless what standard may be set - or later on altered again.

Instead of "0 !PRIVATE" as a complement to "0 !LDRAW_ORG", I would suggest either "0 !LDRAW" or maybe "0 !INFO". But maybe there's an even better name?

> Something like "0 !PRIVATE Unofficial_Set", would
> allow us, in the future to start issuing LDraw.org
> versions of models, using "0 !LDRAW_ORG
> Unofficial_Set" during review and "0 !LDRAW_ORG
> Official_Set" upon release.

1. What motivates that unnatural underscore in Official_Set? In LDRAW_ORG, I can understand it, but not to separate normal words from each_other. That underscore is a heavily potential source of errors. It makes it (slightly) harder to write and much harder to remember to write as it's certainly not the normal way to write. And also, it's a little harder for the programmer to extract relevant information like is it a model, part, primitive, assembly or a shortcut.

2. Isn't a set still a model, or is there a difference I have overlooked?

(Hmm, let me see if I got it right about to start issuing LDraw.org versions of models. First, it's an LDraw.org unofficial official TLG set model that after after reviewing becomes an LDraw.org official official TLG set model. Right? Oh, where are my Aspirine pills...?)

/Tore

[Roland Melkert]

Tore Eriksson Wrote:
-------------------------------------------------------
> But as an LDraw user, I DON'T want it on my own
> private workspace, ie the Models folder. No way!
> B. I made the mistake to have no header at all in
> some of my boxed parts. This has been no problems
> with the software I've used so far, but Roland has
> made me aware that it may cause problems in some
> cases, like his new editor.
>
Current version handles them fine without a header, only thing is there's a chance on 'glitches' cause it's guessing (aka applying fuzzy logic).

I'm not expecting (or demanding) users to change stuff in order to use my software, it might only hurt render quality or prevent them from (easily) editing a particular model.

> How do we address issue B? (Issue A is a non-issue
> IMO) The bad, really bad way would be a new,
> over-formalized Standard that you force us poor
> users to obey, or else we will encounter endless
> warnings and errors. Of course it will render all
> previously made models illegal, like DAT-Header
> now signals 'Hold' for all parts ending with the
> previouly compulsary "0" line. (Oh, stupidivity of
> stupidivities!) Now, where was I? I don't mind a
> new syntax, like "0 !PRIVATE Part", although I
> find it too easy to misunderstand for those who
> happen to have that kind of humour.
>

Personally I use all information in the headers as 'nice to have', if missing my software won't add stuff or make changes unasked. But like said it will hurt rendering and or functionality when it can't be parsed and identified correctly. that's where the power of a good standard comes in.


> But I get very much more worried about the
> opposite ditch. Leaning too much on defined
> semantics, blaming the ignorant users for all
> syntax errors. Backwards compability should IMO
> also mean that software recognize parts and models
> made 5 or 10 years ago, without spitting out
> needless and annoying warnings and errors. They
> should accept and understand statements like:
> 0 Un-official Part
> 0 Un-official LDraw Primitive
> 0 Unofficial Shortcut
> 0 Unofficial Classic Space Model
> 0 Unofficial Moonbase Vehicle model
> , regardless what standard may be set - or later
> on altered again.
>
You are right there should always be a 'fallback' but users can't expect all software to handle all documents flawlessly, especially if the documents are that old and don't 'try to help' the parser.

> Instead of "0 !PRIVATE" as a complement to "0
> !LDRAW_ORG", I would suggest either "0 !LDRAW" or
> maybe "0 !INFO". But maybe there's an even better
> name?
>

I will take this up in the LSC soon, maybe a generic 'what's the nature of this document' meta would help which programs can scan for the term "model" or "part". Or a simular meta with a set of static keywords to choose from.

> > Something like "0 !PRIVATE Unofficial_Set",
> would
> > allow us, in the future to start issuing
> LDraw.org
> > versions of models, using "0 !LDRAW_ORG
> > Unofficial_Set" during review and "0 !LDRAW_ORG
> > Official_Set" upon release.
>
> 1. What motivates that unnatural underscore in
> Official_Set? In LDRAW_ORG, I can understand it,
> but not to separate normal words from each_other.
> That underscore is a heavily potential source of
> errors. It makes it (slightly) harder to write and
> much harder to remember to write as it's certainly
> not the normal way to write. And also, it's a
> little harder for the programmer to extract
> relevant information like is it a model, part,
> primitive, assembly or a shortcut.
>
It's a parser thing, it so the grammar can be dissected without worrying about multiple words in a single 'token'. After it's read the program can split it on the _ and handle it from there (sub parsing the intended goal). An alternative would be using " " around the goal e.g. 0 !PRIVATE "datsville model" or by making it a single variable meta (everything after !PRIVATE is a single value no further dissection needed in order to get the 'goal text' part of the string.)


> 2. Isn't a set still a model, or is there a
> difference I have overlooked?
>
> (Hmm, let me see if I got it right about to start
> issuing LDraw.org versions of models. First, it's
> an LDraw.org unofficial official TLG set model
> that after after reviewing becomes an LDraw.org
> official official TLG set model. Right? Oh, where
> are my Aspirine pills...?)
>

I think the 'official' 'unofficial' keywords only apply to it being in the related library version.

[Chris Dee]

I only used the "Unofficial_" for alignment with existing Part/Subpart/Primitive/Shortcut terminology, under the (maybe false) premise that LDraw.org might like to issue "Official" versions of sets - that have gone through some form of review - as part of the library. I switched from "Model" to "Set" deliberately, as the distributed package already has a Models folder which I suspect many users use as a kind of working area. By using "Set" we could have a "Set" folder alongside Parts.

I think my personal conceptual defintion of "Shortcut" is much more narrow. It is merely a representation of something as delivered already assembled, by LEGO, rather than something that needs to be built be a user.

[Tore Eriksson]

Roland Melkert Wrote:
-------------------------------------------------------

> I've got it working now without the need for
> header information, but it's still guessing and so
> it can guess wrong, like for example the
> town0-1.ldr has this line:
>
> 1 7 540 0 -800 0 0 0.50 0 0.50 0 -0.50 0 0 baihous1.ldr
>
> The whole house is scaled down (to datsville scale
> I guess?). This will cause my 'improved' guessing
> code to handle the whole town0-1.ldr as a single
> part. But it can be corrected by fixing the header
> of town0-1.ldr only instead of all the boxed
> parts.

I didn't take the decision to scale down Anders' house, but as insane as that idea is, I think it's better than not scaling it down.
IRL, his home is smaller than my mother's house, portrayed in btehous1.ldr. But he built it in a totally different scale, so it would look more than twice as big if using the same scale.

As much as I appreciate your fuzzy logic approach before hard-boiled stick-to-the-protocol attitude, I wouldn't recommend guessing that it's a part becuse it contains scaled-down objects.
If you like to build a mixed model with standard LEGO parts and LEGO Modulex together, the Modulex section shall be scaled down by the factor 0.625.
1 7 0 0 0 1 0 0 0 1 0 0 0 1 Lego_model.ldr
1 7 40 -72 100 1 0 0 0 1 0 0 0 1 Modulex_model.ldr
0

Also there can be a comical, commonly used basic cartoonic animation effect in stretching and shrinking objects.
From my scene model "Plasteroid Team, Fri May 13th 2011" - http://news.lugnet.com/cad/dat/models/?n=2292
0 // 3 x "Spiffcraft 1" by Dan Jassim
0 // The first Spiff with the Artificial Gravity Device
0 // not perfectly calibrated...
1 4 -240 -32 -50 1.1 0 0 0 0.6 0 0 0 1.1 spiff01.ldr
1 1 0 -78 -48 0.996 0.087 0 -0.087 0.996 0 0 0 1 spiff01.ldr
1 14 240 -98 -50 0.927 0 -0.375 0 1 0 0.375 0 0.927 spiff01.ldr
0

See result in this pic: http://dl.dropbox.com/u/15954981/minifigs.jpg
(On second thought, I guess they must have by mistake left all three Artificial Gravity Devices in the red Spiff-craft.)

Or: What minifigs do when we look away: http://dl.dropbox.com/u/15954981/playrom3.jpg

(I also used a scaled up B\3005.dat's to build a complete house wall in my heavily simplified versions of the buildings in Datsville Lite. But after purchasing a more powerful PC and finally surrendered to Win7, I trashed the Lite version as it took so much more time to produce than the real version.)

The bottom line is that scaled objects may for various reasons appear not only in parts.

[Tore Eriksson]

Chris Dee Wrote:
-------------------------------------------------------
> I think my personal conceptual defintion of
> "Shortcut" is much more narrow. It is merely a
> representation of something as delivered already
> assembled, by LEGO, rather than something that
> needs to be built be a user.

Oh, that's what I thought was Assembly.

(But OTOH, I have my strong opinion that a Classic window + glass pane is a true "part", while a window frame or a glass pane is either a sub-part or a broken part. )

[Roland Melkert]

Tore Eriksson Wrote:
-------------------------------------------------------
> As much as I appreciate your fuzzy logic approach
> before hard-boiled stick-to-the-protocol attitude,
> I wouldn't recommend guessing that it's a part
> becuse it contains scaled-down objects.

You might be right, this is why we need a new meta, but for now it's a good standard fallback mechanism I think.

I might add options to control the behavior (no header info -> assume model / fuzzy logic / ask) or by giving the user the possibility to define locations from which to assume everything is a model or a manual 'this is a model' override within the editor.

> If you like to build a mixed model with standard
> LEGO parts and LEGO Modulex together, the Modulex
> section shall be scaled down by the factor 0.625.
> 1 7 0 0 0 1 0 0 0 1 0 0 0 1 Lego_model.ldr
> 1 7 40 -72 100 1 0 0 0 1 0 0 0 1
> Modulex_model.ldr
> 0
>

I have no idea what Modulex is, but I setup my editor with 'real world projects' in mind, so current version wont allow manual matrix adjurations and assumes rotation only usage in models. But it won't stop you from editing the files in notepad, it will affect rendering quality though.

> Also there can be a comical, commonly used basic
> cartoonic animation effect in stretching and
> shrinking objects.
> From my scene model "Plasteroid Team, Fri May 13th
> 2011" -
> http://news.lugnet.com/cad/dat/models/?n=2292
> 0 // 3 x "Spiffcraft 1" by Dan Jassim
> 0 // The first Spiff with the Artificial Gravity
> Device
> 0 // not perfectly calibrated...
> 1 4 -240 -32 -50 1.1 0 0 0 0.6 0 0 0 1.1
> spiff01.ldr
> 1 1 0 -78 -48 0.996 0.087 0 -0.087 0.996 0 0
> 0 1 spiff01.ldr
> 1 14 240 -98 -50 0.927 0 -0.375 0 1 0 0.375 0
> 0.927 spiff01.ldr
> 0
>
> See result in this pic:
> http://dl.dropbox.com/u/15954981/minifigs.jpg
> (On second thought, I guess they must have by
> mistake left all three Artificial Gravity Devices
> in the red Spiff-craft.)
>
> Or: What minifigs do when we look away:
> http://dl.dropbox.com/u/15954981/playrom3.jpg
>

Those are fun

> (I also used a scaled up B\3005.dat's to build a
> complete house wall in my heavily simplified
> versions of the buildings in Datsville Lite. But
> after purchasing a more powerful PC and finally
> surrendered to Win7, I trashed the Lite version as
> it took so much more time to produce than the real
> version.)
>
> The bottom line is that scaled objects may for
> various reasons appear not only in parts.

Nothing stop you from doing so (especially with the extra header info), but again it will degrade render quality.

[Tore Eriksson]

Roland Melkert Wrote:
-------------------------------------------------------
> For example loading the town.ldr in my editor eats
> 'only' about 400MB of memory but still renders at
> less then a second.

Once again, that's unbelievably impressive! If I could make your editor work on my PC, I guess I could finally delete L3Lab.

Could you do some benchmarking on your PC? Just to compare our systems and maybe be able to estimate the time your editor would take to render town-ldr on my PC.
Town.ldr renders at 63.2 seconds in LDView, and 4.634 seconds in L3Lab on my PC. (That's the reason why I have been so utterly upset about using LDraw colors that L3Lab can't handle...)
What's the score for your PC on those two programs rendering town.ldr?

Here's the version of L3Lab I use, in case you don't have it: http://dl.dropbox.com/u/15954981/L3Lab.exe

Thanks in advance,
/Tore

[Tore Eriksson]

Roland Melkert Wrote:
-------------------------------------------------------
> Tore Eriksson Wrote:
> --------------------------------------------------
> > If you like to build a mixed model with
> standard
> > LEGO parts and LEGO Modulex together, the
> Modulex
> > section shall be scaled down by the factor
> 0.625.
> > 1 7 0 0 0 1 0 0 0 1 0 0 0 1 Lego_model.ldr
> > 1 7 40 -72 100 1 0 0 0 1 0 0 0 1
> > Modulex_model.ldr
> > 0
> >
>
> I have no idea what Modulex is, but I setup my
> editor with 'real world projects' in mind, so
> current version wont allow manual matrix
> adjurations and assumes rotation only usage in
> models. But it won't stop you from editing the
> files in notepad, it will affect rendering quality
> though.

I see absolutely no reason why notepad should affect rendering quality, except for the kind of embarrassing human mistakes I just made by using copy+paste and forgetting to edit.
> > 1 7 40 -72 100 1 0 0 0 1 0 0 0 1 Modulex_model.ldr
should of course be:
1 7 40 -72 100 0.625 0 0 0 0.625 0 0 0 0.625 Modulex_model.ldr

A few Modulex links:
http://lego.wikia.com/wiki/Modulex
http://www.peeron.com/inv/catpic/save/824127118
http://www.bricklink.com/catalogList.asp?q=modulex
http://homepage.ntlworld.com/chris.w.dee/Modulex/
http://news.lugnet.com/cad/?n=17385


/Tore

[Roland Melkert]

Tore Eriksson Wrote:
-------------------------------------------------------
> Could you do some benchmarking on your PC? Just to
> compare our systems and maybe be able to estimate
> the time your editor would take to render town-ldr
> on my PC.
> Town.ldr renders at 63.2 seconds in LDView, and
> 4.634 seconds in L3Lab on my PC. (That's the
> reason why I have been so utterly upset about
> using LDraw colors that L3Lab can't handle...)
> What's the score for your PC on those two programs
> rendering town.ldr?
>
> Here's the version of L3Lab I use, in case you
> don't have it:
> http://dl.dropbox.com/u/15954981/L3Lab.exe
>
> Thanks in advance,
> /Tore

LDView seems to be slower then my editor while rotating but it does more eye candy (for some reason it doesn't display render speed on my system), but L3Lab reports ~9 seconds for rendering when the whole city is in view (had to figure out how to zoom first), so your system might even be faster. It basically comes down to your VGA card being somewhat recent (I have a Radeon 3870 512MB which is 'old' they say )

Anyhow you are welcome to join the group of testers I send alpha versions to, as asked in this thread:

http://forums.ldraw.org/showthread.php?tid=173,173

I'm hoping to fix a bug/issue this weekend in order to release the 2nd test version, next week / end of the month.

[Tore Eriksson]

Roland Melkert Wrote:
-------------------------------------------------------
> LDView seems to be slower then my editor while
> rotating but it does more eye candy

Eye candy is a good thing - if it is possible to deselect with speed to gain.

> It basically comes down to
> your VGA card being somewhat recent (I have a
> Radeon 3870 512MB which is 'old' they say )

Not really when it comes to L3Lab, and probably not with LDView either(?) I know L3Lab only uses CPU power, and I don't know about LDView, but I think it's the same thing there. No 3D acceleration from the VGA card.

> Anyhow you are welcome to join the group of
> testers I send alpha versions to, as asked in this
> thread:
>
> http://forums.ldraw.org/showthread.php?tid=173,173

Done!

[Roland Melkert]

Tore Eriksson Wrote:
-------------------------------------------------------
> Roland Melkert Wrote:
> --------------------------------------------------
> -----
> > LDView seems to be slower then my editor while
> > rotating but it does more eye candy
>
> Eye candy is a good thing - if it is possible to
> deselect with speed to gain.
>
> > It basically comes down to
> > your VGA card being somewhat recent (I have a
> > Radeon 3870 512MB which is 'old' they say )
>
> Not really when it comes to L3Lab, and probably
> not with LDView either(?) I know L3Lab only uses
> CPU power, and I don't know about LDView, but I
> think it's the same thing there. No 3D
> acceleration from the VGA card.
>

LDView should use VGA acceleration unless it isn't available, if that's the case I'm afraid my editor won't do much better. But the model being that big it could even be 'slow' with acceleration, I kinda assumed the 60+ seconds you mentioned is including loading time though.

> > Anyhow you are welcome to join the group of
> > testers I send alpha versions to, as asked in
> this
> > thread:
> >
> > http://forums.ldraw.org/showthread.php?tid=173,173
>
> Done!

I'll contact you when I have a new test version ready (need to fix a problem concerning relative references first.)

[Travis Cobbs]

Roland Melkert Wrote:
> LDView seems to be slower then my editor while
> rotating but it does more eye candy (for some
> reason it doesn't display render speed on my
> system),

In LDView's "General" preferences tab is a "Show frame rate" check box. If you check that, you should get an FPS display, but only while the model is spinning. Note that for a model as big as DATSVille, drawing edge lines (and particularly conditional edge lines) will slow things down dramatically.

[Roland Melkert]

Travis Cobbs Wrote:
-------------------------------------------------------
> Roland Melkert Wrote:
> > LDView seems to be slower then my editor while
> > rotating but it does more eye candy (for some
> > reason it doesn't display render speed on my
> > system),
>
> In LDView's "General" preferences tab is a "Show
> frame rate" check box. If you check that, you
> should get an FPS display, but only while the
> model is spinning.

that's enabled, but it still only displays the fps occasionally. Sometimes it pops up after you click one of the buttons (e.g lighting on/off), but most of the time it just wont ever display during rotating. I believe I reported that issue to you a couple of years ago, but you couldn't reproduce.


> Note that for a model as big as
> DATSVille, drawing edge lines (and particularly
> conditional edge lines) will slow things down
> dramatically.

I'm using the excellent shader approach Alex Tyler posted last year on lugnet, so it isn't that much of an impact on a decent card. But one thing I added as a result of rendering datsville is 'distance based edges on/off' in order to prevent the whole scene going 'gray' from the faraway edges, I'm also doing some very rudimentary frustum culling.

[Tore Eriksson]

Roland Melkert Wrote:
-------------------------------------------------------
> ... one thing I
> added as a result of rendering datsville is
> 'distance based edges on/off' in order to prevent
> the whole scene going 'gray' from the faraway
> edges,

I have to say, that's a really brilliant idea.

/Tore

[Michael Horvath]

Note that a lot of activity on my part occurred in early 2012 at https://code.google.com/p/datsville/.

[Roland Melkert]

171.000 parts, very cool.

It certainly bring my graphics card to it's knees, it only manages about 1FPS using my LDCad.

Seems much has changed in the last couple of years.

I'm also glad it's not a single mpd anymore.

[Michael Horvath]

Unfortunately I don't have enough RAM to render it in POVray. So I'm kind of stuck until I get a new computer.

[Roland Melkert]

My development system has 16GB, so if you have the povray script online somewhere I'm willing to give it a try using the 64bit povray 3.7 RC (3.6 is way to slow using only a single thread).

[Orion Pobursky]

Note that it actually is impossible to render all of Datville with Pov on 32 bit machines since you run up on the max RAM limit

[Roland Melkert]

Yes, Actually needing more then ~2GB memory in a single program is just about the only good reason to use a 64 bit version of a program (on windows).

[Michael Horvath]

Thanks!

If you could render this file at a large resolution that would be great.

http://www.mediafire.com/?z2w8duk4bxma27o

The actual render doesn't take that long. It's the parsing that seems to go forever.


Mike

[edit]

The script depends on LightsysIV which you can get here:

http://www.ignorancia.org/en/index.php?page=Lightsys

[Michael Horvath]

Wow, that's good to know!

[Roland Melkert]

I had to turn on the swap file it seems to need about 32GB

Parsing takes forever indeed, the result might be a bit disappointing though:



Seems you need to tweak your camera settings (correct for LDraw neg Y axis etc)

Or I'm I doing something wrong?

[Orion Pobursky]

Yeah, I figured that out a while back when I rendered a small animation using only a piece on Datsville. Even then I had to reduce the settings to allow the thing to process.

[Michael Horvath]

Ooops!

Try opening datsville.pov and changing

#include "ldr_wrapper_showcase_1.4.inc"

to

#include "ldr_wrapper_1.4.inc"

[Travis Cobbs]

Most programs also perform 10%-15% faster with a simple 64-bit recompile, at the expense of using somewhat more memory to perform any given task. So big single-program memory space definitely isn't the only reason to have 64-bit versions of programs. I haven't tested POV's performance, but I have tested LDView, and in CPU-bound operations, 64-bit LDView is measurably faster than 32-bit LDView. (In LDView, you generally have to enable conditional lines on a large model in order to see this difference, since most of its geometry drawing isn't limited by the CPU, but by the graphics card.)

[Roland Melkert]

I wrote 'just about'

Taking in account the affords it take to make a 64bit version of existing (older) software it just not always worth the effort to create a 64bit version just for the sake of having it.

[Roland Melkert]

I'll try again later tonight.

I have to dig up an old sata HDD first, because I don't like running a swap file on my SSD.

[Roland Melkert]

Try opening datsville.pov and changing

#include "ldr_wrapper_showcase_1.4.inc"

to

#include "ldr_wrapper_1.4.inc"

Something is wrong, using that it only needs 30 secs to render and doesn't look very HQ at all. Also the camera seems wrong povray complains:

Parse Warning: Camera vectors are not perpendicular. Making look_at the last statement may help.

And it's using ortho mode.

[Michael Horvath]

Yes, it's not supposed to look realistic. I'm using it as a map/guide and therefore quality settings aren't important, but the orthographic perspective is.

[Roland Melkert]

I forgot to incl the rendering.

I did change the ortho camera to perspective, did not change/fix the up/right vectors though.



I have no idea why the two buildings in the back are all black, I've tried playing with the (recursion) settings but it doesn't matter. I'm afraid my povray skills are very rusty

[Travis Cobbs]

It's true that your "just about" makes your statement true when taken literally. However, I would maintain that the implied meaning of what you said varies significantly from the literal truth of what you said. It's similar (although to a far lesser extent) to saying that just about the only thing a car is good for is protecting you from lightning strikes during a thunderstorm. In the case of LDView, the main gain you get when running the 64-bit version under Windows is the increased performance, not the larger memory space.

Very few LDraw models are large enough to cause 32-bit LDView to crash due to using more than 3.5GB of memory. (In Windows, a 32-bit program can be compiled to use roughly 3.5GB of memory. If the program is run on a 64-bit copy of Windows, it automatically gets this extra space above 2GB if the option was enabled during compilation. If the program is running on a 32-bit copy of Windows, this only happens if Windows itself is configured to allow it.)

On a side note (not meant to add further disagreement to your statement, but to hopefully add useful information), I have discovered after porting multiple programs to 64 bits that doing so often uncovers previously unknown intermittent bugs (things like uninitialized variables and off-by-one array overflows). Where the 32 bit version might almost never behave wrong due to a given bug, the 64 bit version might always crash, or very frequently crash. So porting a program to run both in 32 bits and 64 bits can in fact improve the overall quality of the program.

[Michael Horvath]

LOL, I /need/ it to be in ortho though. Also, the buildings are black because they were designed to be black.

Anyway I can see that the pov file is not up-to-date, so here is a new one.

http://www.mediafire.com/?2a99ptd0rdgvc9m



Mike

[Roland Melkert]

You are right, I was just thinking at user level.

While making the 64bit Linux version of LDCad, most problems I had were due to G++ being much more strict then the (old) mingw version, which indeed generated more warnings. Haven't really tested speed differences though, have to build a 64 bit Windows version for that someday (So i can run both in the same environment).

Memory wise 32bit is indeed more then enough for LDraw, datsville for example uses about 500MB in LDCad. But the model as a whole is to slow to edit, but you can double click on any of the submodels and edit those at full speed. And switch back to inspect the overall changes when needed.

[Roland Melkert]

Here you go



Why ortho? it just makes it look weird imho.

[Michael Horvath]

Thanks!!

[Michael Horvath]

Sorry to bother you again, but could you re-render with these settings at 6144x3072 pixels?

http://www.mediafire.com/?wd3myxgvs1ns9au

Thanks!

[Michael Horvath]

I'm heavily considering placing Datsville on the following terrain at the location marked with an X.

http://i421.photobucket.com/albums/pp292...c7791a.png

What do you guys think?

My main concerns are:

1. there really isn't a lot of flat ground to place the city at
2. it's going to be tough to remake and position the SNOT roads every time they're needed without some sort of script to auto-generate them
3. if I decide on this terrain there will no longer be a river and I will have to come up with some other creative use for my bridge model

If you want to try your own hand at generating some terrain, I am using World Machine and the project file can be found in SVN here:

https://code.google.com/p/datsville/sour...%2Fterrain


Mike

[Roland Melkert]

I don't mind these LQ versions render quite quickly.

Here you go