Datsville

[Michael Horvath]

Download it here:

https://sourceforge.net/projects/datsville/

https://github.com/mjhorvath/Datsville

Most changes since the previous release are "under the hood". I made the file headers uniform and filled in missing information. I de-inlined a number of sub-models. I made sure only parts used the DAT extension. There should be no change in outward appearance, so the previous town map still is accurate.

[edit]

Here's the project's Facebook page:

https://www.facebook.com/groups/188200221436/

[Michael Horvath]

Could some with a powerful computer please render this POV-Ray file for me? I keep running out of RAM. The screen resolution should be 1600x1200.


Thanks!!


Mike

[edit]

Never mind. There are issues I need to resolve with the model first that are preventing the render from completing.

[Michael Horvath]

Let's try this again...

If anyone has the time and hardware, could you please render this POV file at 1080p? Thanks!!

https://www.mediafire.com/?lg8xqbgkyekwci9

Also if you have any tips on improving the scene, I would like to hear them!

[Michael Heidemann]

Would be great if you tell us which files we need additionally to download from:
http://www.ignorancia.org/en/index.php?page=Lightsys

In the include file there are three lines with that link as reference.

Code:

#include "CIE.inc"            // http://www.ignorancia.org/en/index.php?page=Lightsys
#include "lightsys.inc"            // http://www.ignorancia.org/en/index.php?page=Lightsys
#include "lightsys_constants.inc"    // http://www.ignorancia.org/en/index.php?page=Lightsys

I can only guess what I should download.
So please add a little bit more description what i need to download. After I got that information I try.

[Roland Melkert]

I had to reinstall it to:

It needs the main download, just drop the contents of the LightsysIV inside the zip alongside to the ldview pov script.

But it also needs lgeo color defs, and I can't find those (we really need to fix the download section).

@Michael Horvath: Maybe you could just zip the whole thing in the future so it works out of the box on a clean povray env.

[Michael Horvath]

Ah, yes. It requires LightSys and LGEO.

LightSys: http://www.ignorancia.org/en/index.php?page=Lightsys

LGEO: http://www.digitalbricks.org/lgeo.html

I also have these lines in POV-Ray's QUICKRES.INI file:

Library_Path="W:\Povray\Include\LightsysIV"
Library_Path="D:\LDraw\lgeo\lg"
Library_Path="D:\LDraw\lgeo\ar"

You'll have to change the lines to however your system is set up as you may not have the same folder structure as I do.

I don't think I did anything special in addition to this.

[Michael Horvath]

The file you need to download is lightsys4c.zip (392Kb) from here:

http://www.ignorancia.org/en/index.php?page=Lightsys

I haven't used any of the "extras" listed below it.

[Roland Melkert]

It still won't render for me, it's missing the *_slope material identifiers.

[Michael Horvath]

It still won't render for me, it's missing the *_slope material identifiers.

I'll look into it. Thanks!

[Michael Horvath]

I think I've fixed the issue. You can download the new archive here:

http://www.mediafire.com/download/e2tlxk...rev369.zip

This time I've included everything needed to render "ldview_povray_datsville_rev369.pov". The only thing you need to do is switch over to the provided INI file in POV-Ray. Do this by selecting "Render" > "Edit Settings/Render" > "Browse", then find "ldview_povray_render_settings.ini" using the "Choose INI File" dialog. Lastly click "Save" or "Render" and return to the main window.

The problem with zipping everything together however is that the LightSys and LGEO libraries do not belong to me, and therefore I am really not supposed to redistribute them. :|

[Roland Melkert]

I'm sorry but it is now missing lg_2752.inc

Although I fixed that by:

Code:

//#include "lg_2752.inc" // Panel  6 x  6 x  9 with Curved Top
#declare lg_2752_clear = lg_6002_clear;

Thinking they are basically the same, but then it ends on, unknown identifier:

LDX_48_slash_1_dash_12ri38_dot_dat_in_part


About the zip, I have added lgeo and lightsys to the quickres ini so you don't need to include them (for me) anymore.

[Michael Horvath]

"Part lg_2752.inc" is supposed to be "lg_2572.inc". Apparently LDView is not parsing "lg_elements.lst" correctly because I already repaired the error in that file.

I don't know about the other part.

[Travis Cobbs]

LDView doesn't parse lg_elements.lst. It has an XML file (LGEO.xml) that I generated from lg_elements.lst. LGEO.xml is in the LDView program directory, since it wasn't really intended to be modified by end users. However, you can easily modify it, and doing so should make the problem go away.

[Michael Horvath]

OK here's an updated POV file. Could you try again Roland? I tried it myself and ran out of RAM before it finished parsing.

http://www.mediafire.com/download/ua45s6...v369_b.zip

[Roland Melkert]

I'm sorry, it still misses: LDX_48_slash_1_dash_12ri38_dot_dat

About your memory if you are running win64 enabling more swap should make it render. It needs about 12GB to parse up to the error)

[Michael Horvath]

Crap.

My virtual memory was set to "System managed" of about 7500MB on one of my drives. I've raised it to 20480MB.

Do I need a paging file on each of my drives or is just one OK?

[edit]

Is there a way to limit a program to using X amount of RAM, and getting the rest from Virtual Memory? I would like a little true RAM left over for other apps.

[Travis Cobbs]

Please try rendering this file here:

datsville.pov.zip

Hopefully it will fix the LDX_48_slash_1_dash_12ri38_dot_dat problem.

[Roland Melkert]

I think it will, it goes on longer on my system, but crashes POV-Ray because it runs out of memory (I have no swap space enabled, I don't want to wear out my ssd).

But the old (l3p) ones which Micheal wanted to render usually ran under 16GB so maybe some fine tuning is in order in the LDView export ?

[Roland Melkert]

No that I know of, Windows decides what goes where, you only can influence the (max) amount.

Maybe it's time form some additional memory, they are practically giving the basic ones away for free these days

But again this is only useful if you are running an 64 bit os.

[Michael Horvath]

I think the difference is that in my earlier isometric images I turned the quality settings way down. I.e. I turned studs off, reflections off, etc. etc. The scene Travis generated has all that stuff turned on, which is what I wanted this time. Unfortunately it seems we are both running out of RAM when trying to render the new scene.

[Roland Melkert]

I'll try again later with swap enabled. But I'll have to clean up an old drive first.

[Michael Horvath]

I've reported some other issues I am having with POV 3.7 here:

http://news.povray.org/povray.bugreports...ay.org%3E/

Not sure if they're bugs or not, but definitely weird behavior.

[Roland Melkert]

Ok the one Travis gave renders, it takes ages but it resulted in this:

huge png (15MB)

Only complaint was about missing (no longer optional) ';' in one of the lgeo include files (71173). But that was easily fixed.

It took over 2 hours to parse but renders in a couple of minutes.

personally I don't really like the perspective distortion, are you still using ortho?

ps: I might take it down in a couple of days given it's size

[Michael Horvath]

No I didn't choose the orthographic (Travis did), but I am still amazed!! Too bad the sunlight is coming from behind the models.

Would you care to do another one? I can fix the camera and lights in that case.

I don't want to push you too hard.


Mike

[edit]

BTW, what are your hardware specs? I would like to get an idea of what kind of hardware I need to buy.

[Michael Horvath]

Anyway, here is the Travis's version except with a non-orthographic view of the town.

Please render it at 1080p.

[Michael Horvath]

I tried it myself again, this time with a 40GB swap file, but it still wasn't enough to finish without a memory allocation error.

[Roland Melkert]

Here's the result of your .pov from above.

Please don't tell me you want the camera to pull back

It's the big res one, 'cause I missed the 1080 part. But you can easily rescale with any decent viewer.

As for my hardware it's an AMD 8 core, with 16GB ram. But I had to also enable a pagefile of 64GB to render this.

It takes 2 hours and 25 minutes to render but that's mainly because of the insane amount of swapping that's going on.

I've seen it peak at 60GB allocated in taskmon, but i was doing other stuff to.

So to be sure you have to buy 128GB of memory, you will also need a more expensive motherboard for that

Or maybe a cheap ssd for exclusive pagefile usage, although it might wear out in a couple of months that way.

[Michael Horvath]

Please don't tell me you want the camera to pull back

Yes! How did you know?

Just kidding.

I will try one more time with 128GB swap memory...

[Travis Cobbs]

For what it's worth, here are the LDX camera settings for the scene so it is zoomed to fit at a similar angle, with a 35 degree vertical FOV, set up for a 16:9 aspect ratio output:

Code:

// Camera settings
#declare LDXCameraLoc = < 13594.801758,-6095.160156,-15380.354492 >;    // Camera Location vector
#declare LDXCameraLookAt = < -342.94620243551617022604,2271.64038036753117921762,-1442.60908459125130320899 >;  // Camera look-at point vector
#declare LDXCameraSky = < -0.27628840440359375696,-0.92050493555211576613,0.276288583217460193 >;       // Camera sky vector (<0,-1,0> will usually work for standard "up")

And here is the camera itself (angle is the only thing that should vary; I think I'll pull that out into a variable also, so the camera section is always the same, and the only thing that varies is the "Camera Settings" section):

Code:

// Camera
#ifndef (LDXSkipCamera)
camera {
        #declare LDXCamAspect = image_width/image_height;
        location LDXCameraLoc
        sky LDXCameraSky
        right LDXCamAspect * < -1,0,0 >
        look_at LDXCameraLookAt
        angle 58.54398
}
#end

[Michael Horvath]

I am using LDXSkipCamera now because I have the *real* camera configured separately in an INC file.

[Travis Cobbs]

FYI, if you're trying to decide on a camera angle from inside LDView for Datsville, it helps to enable "Part bounding boxes only" on the Geometry tab of the preferences.

[Michael Horvath]

OK I tried again with two 40GB swap files. I let it run for 14 hours over night. However it was in the same place as when I left it. No errors or anything. It simply didn't want to keep going. I'm still waiting for POV-Ray to de-allocate the memory.

[Roland Melkert]

Parsing takes very long especially if it continuously swapping memory. When the parsing is done it will decrease memory allocation dramatically over some time, before the actual rendering starts.

I guessing your pc is in that stage if it's not reporting any errors in the log.

sidenote: I wish Pov-ray used some kind of parsing cache so a re-render at a different resolution or camera wouldn't take so long.

[Tore Eriksson]

Great work!

I'd like to make a small comment on the naming of some of my vehicle models, just in case you haven't noticed. Many "gray" vehicles, for example vehicle_058_graybicyclewithrider.ldr , aren't really gray but rather "neutral" color 16. This I have done to be able to reuse the same model in different colors.

Example:

Code:

0
1 22  -40 0 -20  1 0 0  0 1 0  0 0 1  vehicle_058_graybicyclewithrider.ldr
1 27  40 0 20  1 0 0  0 1 0  0 0 1  vehicle_058_graybicyclewithrider.ldr
0

/Tore

[Michael Horvath]

I did notice this for some files. However I don't have a complete list of models that are affected. Do you know which models need to be fixed?

[Tore Eriksson]

Do you know which models need to be fixed?

No, I don't. And actually, I don't believe it causes any visual errors that need to be fixed. It's just the file names and model names that are a bit misleading when read by humans.

[Michael Horvath]

I fixed this in the latest revision #386.

[Michael Horvath]

Ok the one Travis gave renders, it takes ages but it resulted in this:

huge png (15MB)

Only complaint was about missing (no longer optional) ';' in one of the lgeo include files (71173). But that was easily fixed.

It took over 2 hours to parse but renders in a couple of minutes.

personally I don't really like the perspective distortion, are you still using ortho?

ps: I might take it down in a couple of days given it's size

I seem to have misplaced your original PNG version of this image. Could you upload it once again or email it to me?

[Roland Melkert]

I deleted it thinking the other one is better and to save bandwidth.

the link is working again for the time being.

[Michael Horvath]

Ok, I got it now. Thanks!

[Michael Horvath]

I bought RPG Maker VX Ace when it was on sale and progressed a little farther in the game's development. You can see it in action here:

https://www.youtube.com/watch?v=hQphS3Q-HvE

I'm not sure about the story. I guess I need help with that.

[Michael Horvath]

I've attached a test image of the interior of one of Datsville's buildings. There are a couple of problems.

1. The size of an RPG Maker surface tile is 32x32 pixels. This affects character movement as well. You can only move in 32x32 pixel increments. However, the basic unit of measurement (generally) is one stud in the X and Z directions. In the case of this project, 4 studs = 32 pixels. This means that in LDraw you can build structures, walls, windows, doors etc. that don't align neatly with the game's 32 pixel grid, causing your character to get stuck and able to use doors or fit in hallways.

2. Currently, the player character always walks on top of the image. Really, the character should walk "behind" or underneath the south walls of the building, as well as any other object that is further toward the bottom of the screen. RPG Maker is not an isometric game where you can split the scene into smaller parts and assign a z-order to them so that you can pass in front or behind them properly.

I'm not sure what to do here. I'm considering creating alternative interiors with elements that neatly align to the game's 32 pixel grid, and are reworked to avoid z-order issues. Assuming it will even work.

Any ideas on what I should do?

[Brian Sauls]

I gotta' say, it's been awhile.  Seeing how the city has grown since i last visited, however, makes me smile.  Thanks to everybody who contributed their time, especially those who worked at the rendering!

Now that I have a Lego room again, it's time to actually build some of these.  I have a city to populate!

BBQ

[Michael Horvath]

I gotta' say, it's been awhile.  Seeing how the city has grown since i last visited, however, makes me smile.  Thanks to everybody who contributed their time, especially those who worked at the rendering!

Now that I have a Lego room again, it's time to actually build some of these.  I have a city to populate!

BBQ

Thanks so much!

Here's a later test video of the RPG. I have not touched it in several years.



The next thing I plan to do is re-implement the "boxed" (partially) versions of the buildings that Tore Eriksson made earlier and which I (unfortunately) unmade. I don't know when I will get around to doing this. It's a tricky task, and I am frustrated by the tools.

[edit]

I updated the links in the first post of this thread.

[Michael Horvath]

Just released revision 501 yesterday I think.

datsville_modelmap_rev501 by Michael Horvath, on Flickr

I redid parts of the commercial district. I also added corn fields in the bottom right. Lastly, I went about updating the metadata, making sure each model had the correct information and that it was formatted correctly.

Datsville is up to ~220k parts, and is once again too big to properly render in POV-Ray using my newest computer. I have to use LDBoxer first or I run out of memory. 8GB is not enough. POV-Ray took ~30 hours to parse and ~1 hour to render the model.

Here is an animation showing how the town has grown over the years. It does not include revision 501 though.

[Michael Horvath]

Revision 528

1. Reverted some earlier changes WRT procedural terrain created using World Machine.
   
2. Sidewalks are a bit less wide now.
   
3. Lots of reorganization of files and file naming to be more consistent.
   
4. Many other small tweaks and fixes.
   

[Roland Melkert]

Revision 528

1. Reverted some earlier changes WRT procedural terrain created using World Machine.
   
2. Sidewalks are a bit less wide now.
   
3. Lots of reorganization of files and file naming to be more consistent.
   
4. Many other small tweaks and fixes.
   

This project is still my favorite performance/render engine test case

2fps (on my 6 year old radeon)using 575MB of system ram in LDCad 1.6a,
I'm hoping to get it at 3 or 4 fps in 2.0 maybe even 'playable' by using game engine like frustum culling.

[Michael Horvath]

Revision 528

1. Reverted some earlier changes WRT procedural terrain created using World Machine.
   
2. Sidewalks are a bit less wide now.
   
3. Lots of reorganization of files and file naming to be more consistent.
   
4. Many other small tweaks and fixes.
   

This project is still my favorite performance/render engine test case

2fps (on my 6 year old radeon)using 575MB of system ram in LDCad 1.6a,
I'm hoping to get it at 3 or 4 fps in 2.0 maybe even 'playable' by using game engine like frustum culling.

Yeah, it's up to 225k+ parts at the moment. I have to use LDBoxer to simplify the parts before I can render the model in POV-Ray. Otherwise, I run out of memory really fast. (I have 8GB RAM.)

I also noticed that I am getting much better framerate in LeoCAD than in LDView for some reason. LDView has better lighting, could that be the cause?

[Merlijn Wissink]

Revision 528

1. Reverted some earlier changes WRT procedural terrain created using World Machine.
   
2. Sidewalks are a bit less wide now.
   
3. Lots of reorganization of files and file naming to be more consistent.
   
4. Many other small tweaks and fixes.
   

This project is still my favorite performance/render engine test case

2fps (on my 6 year old radeon)using 575MB of system ram in LDCad 1.6a,
I'm hoping to get it at 3 or 4 fps in 2.0 maybe even 'playable' by using game engine like frustum culling.

I've got about 6fps on my GTX1070, but according to Task Manger (ok, probably not the best monitoring tool ) gpu usage is 'only' about 50%. Any reason for that?

[Roland Melkert]

I've got about 6fps on my GTX1070, but according to Task Manger (ok, probably the best monitoring tool ) gpu usage is 'only' about 50%. Any reason for that?

Probably because it's hitting a bottleneck due to the insane amount of triangles.