Procedural Terrain Rendering How-To

Development
386

Hi Akenre, no problem with the sketches or additional information, I’ll gladly provide more insights, please give me a few days (or probably the weekend), I need to make up my mind how to visualize these topics beforehand. Will try to give an update during the next days.

PS: No the techniques do not require any engine code modifications, in fact you can apply those to any of the major engines like Unreal Engine or CryEngine easily. With regards to the physics engine impact I can’t tell how well these techniques apply as I havent worked on that part that much yet. But at least since KSP uses the same technique and their physics work quite well I guess it wont be a problem.

2 Likes
387

Sorry for the late update. Quite a few things prevented me from updating the thread and answering your question @Akenre , my daugther was born three weeks ago (yeah), and some implementation changes took my few remaining time as well. Furthermore I underestimated how difficult it is to visualize some of the techniques. So sorry for the no/late update, I still try to do these.

Even though the current Unity3D implementation worked well, I had some “aha”-moment where I realized I want to change the implementation approach another time a bit. That happended in a 5 minutes brute force implementation where I simply added a velocity property to the classes where I handle star and planet positions, and when I then saw the sun moving with the right velocity to my direction I thought “if its that easy, where not move further away from Unity and decouple as much as possible”.
At least shortterm I’d stick to the techniques mentioned above, at least as long as I work within Unity. But want t o move to a more decoupled approach where the planets, positions, movement etc. are handled independend from Unity (so Unity is only the “representation” or “rendering” part in the end, at least for now). Indeed currently I reimplement a lot of classes you typical know from an engine (e.g. Unity) in double precision currently. But its big fun.

Here is the first visualization of floating origin

floatingorigin.jpg1549×1069 289 KB

Still working on the other parts, to follow…

6 Likes
388

Congratulations!:hatched_chick::birthday:

3 Likes
389

Hello friends!

Wow, some pretty amazing contributions in this thread. I’ve been away from this subject for a while, and it’s so nice to everyone keeping this discussion thread alive.

@JoergZdarsky - Unity 5.6 is out, and check this:

In 5.6 we now support Procedural Instancing, where instance data is supplied via a custom source in the Shader, rather than from Material Property Blocks and Support for DrawMeshInstancedIndirect, where draw arguments are supplied from a ComputeBuffer. This new way of rendering instances via script has almost no CPU overhead, resulting in a massive performance boost, assuming the CPU is the limiting factor for your framerate. – source

That sounds like exactly what we need to solve the batching issue.

And the developments on Asynchronous data transfer (GPU->CPU) in this thread excite me even more! (You’re already aware of this, but I figured everyone else might benefit from the link being re-posted)

With these two pieces of the puzzle solved, it might be time for me to revisit this subject. I’ve recently started writing a number of simple games (one-a-week in scope) to improve my Unity chops, and have come to appreciate more of what Unity can do despite the limitations it places upon one’s code architecture. Your recent move to handle most of the scene graph outside of Unity, and simply use Unity as the rendering “front-end” is a great solution, and likely worth the time cost of setting it up.

Anyhow, that’s all I have to share for now, other than to say hello again to my fellow devs and hope life is treating you all well!

-Navy (although “Civilian” is a more accurate title now…)

2 Likes
390

Hey Guys, im back in Business.

Here a short Video with ChunkedLOD + Tessellation. Seems Direct3D11 is a big advantage.

Try to implement Tessellation, anybody have experience with this?

The nice thing is, it only get tessellated when im near the Ground, that means no Cracks??? :wink:

4 Likes
391

Looks great! You can do tesselation in OpenGL as well, but I don’t know how similar the APIs may be. Have you figured out how to have the tesselator evaluate your terrain function? If you get to that point, please share!

1 Like
392

Welcome back NavyFish,

glad to read you again :slight_smile: ! Hope you do well! Yeah a lot of new exciting things to try, Unity3D evolves a lot currently, procedural instancing and the asynchronous GPUCPU data transfer seem exactly what we’ve been missing.

I’ve underestimated how much (or less) time I’d have left for continuing working on all the procedural terrain stuff, so things progress pretty slow. Still haven’t found the time to continue on documentations and descriptions, but tried to continue on the custom sandbox implementation.

Great fun, and again learning a lot while trying to do reimplement the typical Unity stuff on my own (especially the Matrix4x4d and Quaterniond classes were tough) with the important classes in being in double precision. But there was some progress, I finished the first bunch of classes I wanted and currently setup the underlying engine loop (that updates the timer or queries all objects for physics update). The design is very similar to Unity, e.g. so that you can add components to gameobjects that change their behavior, e.g. a ridigbody for physics.

Currently I did a rough (bugfixing not done) implementation of:
MainLoop (the central engine loop, I don’t know if this is good practice :slight_smile: )
SystemTime and Time (custom Timer for Time.deltaTime, used e.g. for physics within the engine).
SceneManager and Scene (SceneManager manages all scenes and scene the objects within a scene).
Bounds (double precision)
Color
Color32
Component
Mathd (double precision)
Mathf
Matrix2x2d (double precision)
Matrix3x3d (double precision)
Matrix4x4d (double precision)
Meshd (double precision - probably overkill to keep mesh’s vertices in doubles? :slight_smile: )
MeshTopology
Object
Quaterniond (double precision)
Rigidbody (with global gravity and just a few functions only yet, to be extended)
Space
Transform (double precision - most important as it manages position, scale and rotation!!)
Vector2d (double precision)
Vector3d (double precision)
Vector4d (double precision)

Today I did my very first debug test. And the magic happens, with 1000 objects with a ridigbody components applied to each one. Seems to be fast, the FPS loss happens mostly in Unity due to the 1000 UnityEngine.GameObjects instances and their separate Update() Component-Scripts. But as that is only for debugging I dont care. And I think there is room for optimization.
Pretty happy, as the positions, scales and rotations are all handled in double precision, and the first part of the physics (single gravity effect) seems to work.

Nextup is a bugfixing and testing phase of the components, and then extending the Rigidbody class. I am thinking of adding a “LocalGravity” variable (Vector3d) to Ridigbody too, so that all objects are not only affected by a single global gravity value directing into the same direction (like in Unity3D), but to allow that all objects do influence each other with their gravity (maybe a bit parameterised, so that you can control which objects affect others or are influenced by LocalGravity).
Afterwards I might start with an easy collider component.

Sorry for no terrain stuff and just a few grey spheres and cubes :wink:

EDIT:
I’ve added a LocalGravity for each rigidbody (which can be combined with GlobalGravity), which is per object a result of all surrounding objects that have a gravity influence.
The LocalGravity can be configured per ridigbody in terms of

  1. “emitLocalGravity” - Does the ridigbody have a gravity-effect to the other ridigbody’s LocalGravity.
  2. “receiveLocalGravity” - Is the ridigbody’s LocalGravity affected by the surrounding ridigbodies.
    idea is that for very small objects (e.g. a person) I do not have to check if it influences other ridigbodies, however it could still be that it is pulled by larger rigidbodys (a planet).
    The calculation is not parallelized yet, so ~250 physical rigidbodies each one emitting and receiving localgravity is an upper limit in terms of performance right now. Its still performed in the Render thread. When I’ve done Collision detection I am going to have to think about how to structure the engine into several separate threads.

@montify
Wow very nice!!! Very very quick, impressive. Have you extended the implementation for the terrain heights? Would really like to see an update too!

1 Like
393

Wow, dang Joerg!

Sorry for the long overdue delay – I must have missed the email notification and only tonight realized you’d replied.

No questions here (other than how much coffee you drink??), and nothing really to share at the moment, though part of me wants to revive that old economy prototype thread from 2007 and start it afresh on these forums…

-Navy

394

Time to pull this good old thread up :slight_smile:

I havent imagined how short I could be of time since my daughter is there (I still have a good sleep since she has too, so things could have become worse :wink:). But its pure fun. Lately I had some time to prototype an idea I call “Floating Point Projection” to connect my double precision sandbox within Unity to Unity’s float worldspace.
It works suprisingly well. I still have to figure a few things out since I would like to allow flexible use of floating origin together with this technique, as well as local physics around the camera. Its not a big deal to implement this into the new prototype, I only how to decide which way to go since there are basically two options. (one would only need one camera, the other one two (one that uses floating origin, the second one being fixed at (0,0,0) origin). The benefit is that its not necessary to deal with spaces and separate scales anymore like KSP for example does (and which idea I took over previously), with problems when an object should move from one space to another etc.

This prototype starts with a velocity of 299792458.0d m/s and approaches the sun, a quadsphere mesh with sun scale and earth-to-sun distance (1392520000 meters diameter and 149597870700 meters distance). The background stars (6000) are quadsphere meshes of a diameter of 1.22 * Ro (Ro=Sun radius) and placed randomly in a range of +/- 500 lightyears. An additional function keeps them ~1 pixel wide visible before fading away if very distant, otherwise the scene would be almost black. A velocity of nearby 70.000.000 SI (lightspeed) is needed to see the stars moving or to reach a siginficant speed to move to the next object. Space is sooooooooo huge. :slight_smile:
Objects positions and scales are stored in double, the double to float projection to sync with Unity is performed each frame. For optimization it should be changed so that this only done if the viewer moves or if there is zero reference frame velocity. Alternatively the latest projection value could be cached and only synched at a certain threshold. However even doing this each frame with 6000 stars still works fine as the projection calculation is very very quick (as the formula is absolutely simple), what makes it slow is the access of Unity’s transform. That part of optimization is tbd.

2 Likes
395

Nice bump, I did a bit of work on volumetric clouds since my last post, decided to integrate it into the prototype. And I would strongly suggest to you not to make an engine. :confused:

3 Likes
396

I was experimenting with C# and SharpDX and afterwards classical C++ and DX11 to build an own engine up. I have to admit its a hell lot of work (especially if you have to get used to C++ again) and takes time to get your own engine content running in it,time I don’t have (unfortunately) atm. So yes I’ll stick to my sandbox code running in Unity3D for the moment - which is absolutely worth it, as working with custom gameobject and transform classes and code is a lot faster than using Unity’s ones…

Nice, volumetric clouds is a cool feature unfortunately lots of engines and games are lacking. Some closer recordings would be nice! I always kept away from them as my brute-force implementations always became a performance problem quickly :slight_smile:

3 Likes
397

Welcome back, @JoergZdarsky!

I have some process too. I’am propagate slowly in case of my main work.
I’ve made soup from proland. Yeah. Because i can.
BTW, opensource. (Search for SpaceW in github, if interested)
@JoergZdarsky, our methods are very similar. After a ton of experiments, tryouts i’ve made the ideal flow for engine full of space.
Keep it going.
Feel free to ask your questions.

2 Likes
398

How are you guys doing this? This is amazing! I know this might be little out of place but are you guys planning to go open-source when you finish your projects or are you going to sell them?

399

Ghosting in from the abyss. I’m in the midst of a long sojourn away from the world, in the mountains and in the woods. Just myself, alone with nature, God, my faithful truck and the wind at my back. And yet in today’s age we are never but a moment away from each other. Well, unless there’s no cellular service :slight_smile:

Thought you guys would enjoy this…

(Also, this channel is fantastic, highly recommend a sub)

See you when I see you :slight_smile:
Navy

8 Likes
400

That’s really good job you did ! Could you elaborate a bit more on the Floating Point Projection you came with ?

401

Well in the end it is a rather simple approach

What I did for this is to first create some classes that store for each object the position, scale and rotation in double vectors (vector3d). For that I reimplemented, beside many other classes, especially Object, GameObject, Component, Transform and Rigidbody in double precision. It offers (almost) all the same functionality and behavior which the Unity3D classes do, but in double precision for all vectors, and completely in C# (to stay away from C++ wrapping which is a performance killer).

When I know want to manipulate the objects, I completely work with my custom classes, so e.g. gameobject.transform.localPosition += new Vector3d(0,0,10); etc. and all that stuff. So within my classes I can store positions and scales in a double range, without the float restriction.

Then, when I consider all manipulations done (so after changing positions due to regidbody velocities, physics, etc etc etc.) in the very last step I sync my objects with Unity3D representations. So each custom gameobject class has a Unity3D gameobject representative that is synched at the very end at each frame. You could, I think, push that even further and directly render stuff.

In order to push the double positions and scales of my classes into Unity’s float based scene coordinates I use at that last step the two point form of a linear equation. You have to manipulate the scale according to the position changed by that equation.

You could say, what KSP did with separate Layers and fixed 1:xxxx scales I do, depending of the double-based distance, each frame with dynamic scales.

The side benefit is, you only have to access each Unity3D GameObject transform once (or more precisely, once for rotation, once for position, once for scale). Unity’s Transform kills your performance when you work too much with it. So I can freely change my custom physics and everything else on my objects, and only at the very end there is one sync step on Unity’s Transform.

Thats basically it.

402

NavyFish inspired me to give it a try for wormhole / vortex travelling.
After following an article that worked with fixed angle tunnel elements for my very first test I picked up a different idea by NavyFish to work with slice elements where each tunnel segment is defined by n slices, and where each slice defines the current radius of the tunnel, the center position and the direction vector to the next slice center element.
Thanx again @NavyFish!

Works pretty well. I manipulate the direction to the next slice center point by use of simplexnoise, as well as changing the radius (clamped within a certain range).

The outside lookat direction and velocity direction during moving through the stars is synched with the tunnel travelling. I experimented a bit with transparent tunnel walls in order to get a glimpse to where the tunnel is going to (thats why I also arranged the triangle index in a way so that the tunnel wall can be rendered inside/outside/on both sides. But well I have to admit I horribly lack of shader or visual howto knowhow, so it looks a bit dark and not what it could look like. But anyway, coding tunnel systems can be fun.
For quantum travel I plan to use a similar effect but render the tunnel in a straight direction without simplexnoise manipulation for curves.

(unfortunately the scene is a bit dark, due to the compression the quality lacks a lot. I tried to brighten it up with youtube but that made things worse, so I reuploaded it with original brightness. I might record a different brigther scene with a different texture, need to figure out what works OK visually.

4 Likes
403

Wow.

7 Likes
404

Anybody done anything with fake plate tectonics? I’m trying something out, but haven’t finished it yet.

1 Like
405

I tinkered with plate tectonics a long time ago using Voronoi tessellations. Create the tessellation, select sea and land cells, shift the land cells around, and use them to feed the terrain generation. When calculating the height field, look to see how many land cells overlap the position. The more land that overlaps a given position, the higher the terrain.

I stopped prior to calculating the height field.

Beyond that would be feathering out the effects of the cells. That is, if India and Asia overlapped, you’d get the Himalayas, but the transition to the surrounding terrain would be pretty abrupt, producing lots of vertical cliffs.

I did very coarse cells and called each one a plate. It might be interesting to create finer cells and use an algorithm to create plates from multiples.