Quick Progress Reports - Engineering

Battlescape
21

Wednesday, 2 March 2016

Yesterday the plumber came again to repaint my appartment ( yup, you read that right, the plumber is also a painter. Yeah, I was surprised ). Therefore this has been a forced vacation day.

… or almost. I still had a bit of time to investigate some issues, work on the facebook authentication problems, do a couple of bug fixes, and help some people who seemed to have access issues. I can’t say it has been the most productive day though.

Since I was forced to early retirement for a day, I investigated some hardware replacements that I had been delaying for a while. In particular, my main screen monitor is borderline garbage now ( it’s 8 years old ) with washed-out colors. I’m planning on ordering a new QHD ( 2560 x 1440 ) monitor. I won’t go higher in resolution for price reasons and lack of a video card that could handle UHD anyways…

Short-term road-map ( March )

We are planning to release a first patch to the game client today. This will be an optional patch, and will mostly address some of the most urgent / pending issues, like those people being unable to connect through Facebook. There will be a bunch of minor tweaks too.

Looking ahead, this month we are going to focus on stabilizing the game, sorting out access rights / launch problems or other technical issues, adding new placeholders and doing some tweaks.

  • Keith will mostly be busy on finishing the launcher’s automatic patching system.

  • He also has to do some website updates, and resume work on our custom pledge system.

  • We want to get ride of IndieGoGo as soon as possible. The way we’re forced to poll at regular intervals to get the list of backers is extremely annoying and taxing server resources. There are also update delays, which means backers have to wait a bit for their data to be refreshed before they can access the game. Quite annoying for everybody.

  • Our artists are going to work on concept art for, well, the entire game.

  • They’ll also continue working on the existing assets, experimenting with the new artist pipeline ( based on decals ), the cockpit, better structures for the asteroid base, some new lights placements…

  • And, most importantly, producing a new set of placeholder assets ( those will be untextured too ) which we can use to iterate game on fairly quickly.

  • As for me, I’m going to keep working on client issues and gameplay.

  • For example my top priority will be to figure out the weird cockpit flickering bug. I’ve already tried a few hours during the KS campaign, but due to being so busy I couldn’t keep working on it. Now the dev-access release is done, I’ll have enough time to investigate it properly.

  • I’ll also address some of the easiest / most frequently seen remarks or bugs. Drifting failing while flight auto assist is off ? Sounds like it’ll be an easy bug to fix. There’s probably going to be a lot like that.

  • I’m also going to start some of the more heavy work on the client/server. A lot of data is client-sided at the moment, which is no good. I did it that way before the KS because it was faster to implement, but of course this cannot stay like that.

  • I’ll also work on adding the new placeholder content to the game.

  • When I have a bit of free time I’m going to do some experimental improvements on the look of planets. They’re not yet at our target level in terms of visual quality.

  • One thing I will NOT do just yet is to address the elephant in the room: the networking hit combat issues. Those aren’t easy to solve, in fact going forward it’s probably going to be our #1 challenge. We’ll wait until the game has stabilized a bit until we revisit it.

Mid-term road-map ( April - summer )

  • We’ll address the network issues at this stage.

  • We also need to do add better multithreading to the engine, do a bunch of optimizations, implement streaming and more engine work.

  • We’ll start experimenting with Vulkan, although I can’t give a timeline on that just yet.

  • We’ll also start doing experiments for our in-game GUI solution.

  • We also have to update or replace a bunch of old third-party libraries: physics, sound ( OpenAL is really obsolete at this stage, we’re planning on using Wwise ).

  • That’s pretty much when we’ll add new ships, new weapons and iterate on the real gameplay: attacking/defending objectives like factories or NPC ships.

  • We’ll add more planetary environments at this stage too.

Long-term road-map ( Q4 2016 and later )

  • Content in the game will still be pretty thin at this stage, but we’ll work diligently on increasing it. There will still be some placeholder content remaining.

  • Alpha will be released around that time.

  • Balancing the game “seriously” will also start around that time.

  • Small ships gameplay should be faily well established at this point, but capital ships gameplay will require a lot more work.

  • Note that the longer ahead we are looking at, the less certain we are about development priorities. A lot of things can change depending on how things are going. So take these roadmaps with a grain of salt.

40 Likes
22

Saturday, 5 March 2016

Patch 0.1.2.0 is out, woooh hoo. Adresses some issues with Facebook authentication in the launcher, detail textures for the Glimmerfall asteroid ( we’ll soon extend these detail textures to the roids in the rings ), tweaks to the HUD and a bunch of bug fixes.

Our artists have started to work on the new concept art for all the game’s content, which will be followed up by a bunch of placeholders. I’m not sure yet how the ships placeholders will be integrated into the game since we have no GUI to select a startup ship. Maybe a ship will be selected randomly when spawning, or maybe we could add a preference setting in the config file. We’ll figure that out later.

A few users are still having issues to launch the game. It seems related to fonts. One of them has a problem with the launcher’s fonts ( which uses Windows standard fonts, so not sure why it’d be a problem ) and the other has a problem with a game’s custom font ( the function we use to load / install that font is quite simplistic so I’m not sure what could go wrong ). Still investigating.

I’d like to remind everybody that warp mechanics are barren. I’ve only worked on them for a couple hours in the past months, so they’re in need of some serious love. The sudden acceleration / slow down is one thing I want to address soon. Overall I’d like the warp to work like an Alcubierre drive, special effects coming at a later stage, but functionally it shouldn’t affect the ship’s current velocity. The current implementation of the warp simply changes the acceleration by a factor which is depending on gravity ( so the closer you are to a planet, the lower the warp factor ). That gravity dependency is something I’d like to keep, however the fact that the “real” velocity changes during warp is something I’d like to remove. Oh, and for long-term travel, we’ll probably use a different mechanism which provides FTL. We want players to “jump” to the other side of the solar system in like 30s, not half an hour…

Some of you might have noticed “Glimmerfall Ops” in the middle of the rings, far away where the Glimmferall asteroid actually is. It’s not technically a bug, simply a debug test I forgot to remove from the server entities config. My bad. Will remove it next time the server crashes. Which can take a while. As I am speaking, the server’s uptime is close to 5 days. Before that, it did reboot… because Windows forced an update and had to reboot. Prior to that it hadn’t stopped since the launch last sunday. Therefore, it’s official: the server hasn’t crashed once. Of course, we haven’t changed much since the KS campaign, but going forward once we resume severe work on the server, it’s probably going to crash a lot more frequently.

A lot of people have reported bugs or issues though our launcher’s report tool. It seems to work pretty well. On the list of issues, some are false positives, like players who did not immediately understand how things work. That’s fine, keep reporting everything, we can sort out everything on our end :slightly_smiling:

One “bug” that I see people report often is in fact not a bug: the lights flickering on the ship in 3rd-person mode. We have a basic damage system; when a light gets hit by a blaster shot, it gets damaged and flickers randomly forever. But I admit it looks like a bug. However what is a real bug is the cockpit visibility flickering, which I’m going to work on diligently in the coming days ( hoping, not weeks… )… but it looks like a tough nut to crack.

There are a few bugs which I haven’t been able to replicate so if anybody has hints, feel free to make a post:

  • Players showing up as “Unknown” via Google login: each time I tested it used my account’s name ( INovaeFlavien ) so I’m not sure how to replicate it.

  • A few crashes: please send your log file, not your crash file. I think we’re going to get rid of crash files, cause they contain less relevant information than log files, and most players actually send those instead of the log files :frowning:

  • It is useless to send us a crash report without the log file. As a reminder: the log file is located in C:\Users\YourName\Documents\Infinity Battlescape\Logs . A report just saying “I launched the game and it crashed” is unfortunately not very useful to us.

  • NPC ships lights shining through building walls: technically not a bug, we just haven’t enabled shadowing for NPC ship lights. We might do it later. I agree it’s sometimes confusing when you’ve spawned in a hangar and see weird lights shining from the walls :sunny:

32 Likes
23

Friday, 11 March 2016

Unfortunately these last few days I’ve had a bunch of IRL issues related to my Internet connection and digital TV. I won’t go into details, but at this point, I’ve wasted more than a dozen hours with my provider’s tech support, replacing equipment and having appointments with technicians to inspect my installation. The good news is that everything has been upgraded and that my problems are apparently solved ( although the problem kind of solved magically itself in the end so I’m not confident it won’t happen again ). It really sucks and frustrates me that I have to spend so much time on these issues, but oh well… for now everything seems to be working again.

On the code side I’ve started to do some refactoring of the client/server. This is only a first step and there’s still a lot more to do in the future, but it’s still an important first step. I was also investigating the ability for Visual Studio to debug a child process and came up with a cool plug-in which does it. Just install the plug-in and when you call win32 api’s CreateProcess it’ll appear in the debugger, call stack and threads list. Why isn’t this a standard in VS yet ? It seems to work perfectly and I see no immediate reason it shouldn’t be available in standard…

Next step now is to do a bunch of additions to our material editor. We’re missing some nodes which could be handy to our artists. Some of these nodes are already implemented on the engine side and were “forgotten” ( yeah that happens ) when we created the material editor ( like absolute value, 1 - x, swizzle ), and should take only an hour or two to add. Since Jan worked on the detail textures for the Gimmerfall asteroid we also realized some Photoshop-style filters would become handy, like an overlay node. It’s possible to recreate the overlay formula via other nodes, but obviously it requires the artist to assemble dozens of nodes to do so, so these new nodes will save effort and reduce the material node graph complexity by a lot. All these improvements should be done by this week-end.

After that, it’ll be time to start the serious investigations about this cockpit / thrusters flickering bug. I have a few ideas on what the cause could be, so I’ll start with that, and hopefully it won’t take too much time before I narrow it down.

24 Likes
24

Tuesday, 15 March 2016

I’m finally done with the material editor updates for now…

I’ve added a bunch of new nodes that artists might find useful. Here’s the quick list:

  • Abs ( taking the absolute value of a number )
  • 1-complement ( 1 - x, this is simple but saves a node in the graph, so it’s mostly to reduce graph complexity )
  • Swizzle ( shifting the components of a vector, ex.: vector.zxwy )
  • Triplanar mapping

hHAy58S.png1920×1079 264 KB

Also many blend modes from Photoshop / Gimp:

  • Overlay
  • Linear burn
  • Linear dodge
  • Linear light
  • Soft light
  • Screen

Back to working on the game now !

33 Likes
25

Sunday, 20 March 2016

My new screen monitor, which I received 2 weeks ago, seems broken. There’s a vertical line on the right side, which changes from white to purple to green sometimes. Quite distracting. And quite annoying for a brand new screen… now that my TV/Internet trouble are solved, I thought I’d be ok for a while… guess not.

As for development news, solving triplanar mapping has taken more time than anticipated. I thought it’d only take a few hours, but in the end I worked more than 2 days on it. In my last report I said that triplanar mapping was working, but that was only for diffuse textures. When I tried to use the same code for normal maps it resulted in incorrect normals due to the tangent space changing due to the triplanar mapping. In the end, I had to create a new tangent space for each projection axis, and then convert the weighted normal back to the “real” mesh’s tangent space. The resulting code is only 10 shader code lines long, but it took a dozen hours to figure out. I’m happy I’m not getting paid by the line of code…

Now for the good news: I’ve fixed a couple of “major” bugs in the client already, most notably the bug that was #1 on my list: cockpit ( and particle systems ) flickering. It ended up being a problem with the octree updates and some objects not being updated in the right order of dependency. It affected the cockpit, lights and particle systems, but technically all other scene entities too.

While I was working on flickering issues I investigated the reason why damaged lights continued to blink even though the ship is powered down. That was a different bug, where our blinking behavior component wasn’t correctly inheriting the light’s visibility status. In order words, the light was correctly set off, but its attached blinking behavior kept turning it back on or off. This is now fixed.

I’ve also fixed a bunch of other bugs which were less important like the color of enemy ship HUD icon when warping, changed light speed limit to its real more accurate value, crash fix in the launcher server browser when the list is empty, some client/server infrastructure cleaning etc…

Keith is continuing his work on the patching & installation systems and seems to be getting close to a first version. It might still take a while until it goes live though. Accounting & taxes have distracted him quite a bit these last 2 weeks.

On the art side, Jan, Kristian & Dan are working on the new ship concept art and placeholders. As soon as we have a first version integrated into our project I’ll make a new build including the new ships placeholders and bug fixes; hopefully that will be sometimes next week. They’re also making a bunch of asset fixes to the existing art. More on that later.

35 Likes
26

Friday, 1st April 2016

Not much to report lately. The art team has finished the first round of placeholders ( interceptor still missing but should be ready next week ), so I’ve been working on integrating all of these into the game. Which mostly means playing with configuration files, adjusting parameters, testing physics and stuff like that.

The corvette is unfortunately in a bit of a mess, currently. Its size is 115m, which is too big. We’re already talking about halving its size to around 60m, so it’ll need some redesign already. At 115m it becomes very awkyard to control through the mouse, which is going to be a problem for slower ships no matter what. But even at 60m, it can’t be controlled like an interceptor as it is thousands of times more massive. The problem with the current mouse scheme is that you need to keep rolling the mouse in order to turn around. Now it’s fine when the ship turns fast ( in a single hand movement you can turn by 180° ) but when your ship is massive, you need to repeat the roll movement many times. Therefore, I wonder if an alternative control scheme would work better; maybe something similar to Star Citizen or Elite: Dangerous, where the position of your mouse cursor compared to the screen center determines your ship movement. Feedback on this sensitive issue is most welcome. Note that for capital ships I’m thinking of a completely different movement system ( I’ll elaborate on that later when we implement cap ships ), so we’re only talking about 30m-60m sized ships here.

To be clear, I’d like to keep the current scheme for the fast, agile ships such as the interceptor. The question is, should we use a different scheme for slower ships like the bomber & corvette, or would you still rather use mouse-rolling movement for those even though it’d be more difficult ?

19 Likes
27

Monday, 11 April 2016

New patch 0.1.3.0 is out ! Not too happy with Azure US East datacenter going down the exact moment I was trying to update the server. The downtime was pretty long too, a dozen hours or so. So I’m not exactly impressed by their service, especially as we decided to go to Microsoft because we thought they’d be robust and that kind of incident would likely never happen.

Anyways, as some of you might have noticed, I changed some physics parameters for the ships in the prototype, and they now have a proper mass ( in comparison, the bomber in the previous versions always had a mass of 1.0 ton ). I compensated this increased mass by equivalently stronger thrusters, but unfortunately I overlooked other places in the code which didn’t take the mass into consideration ( in particular atmospheric physics ), and as a result the behavior of those ships in atmosphere is now unsatisfying. I will do another pass on this soon.

My plans for this week is to do major improvements on the controls and input mapping. I will add support for joysticks/gamepads through Direct Input and XInput ( I already tested support for the Xbox 360 controller and PS4 controller, but I’m pretty sure it’ll work out of the box for the Xbox one or PS3 controllers too ). The keyboard mapping profiles will get moved out of ClientConfig.xml into their own config files, and I will add better mapping of actions and support for more keys into the input mapper.

Kristian has finished the interceptor placeholder. It’s still very rough/basic modelling, but this week I will add it to the prototype as well as its cockpit. Expect another build soon, maybe next week-end.

That’s all for now ! Oh, and I received my CV1 Oculus Rift last tuesday, so I spent a couple hours playing VR games and testing demos. It works very well on the hardware side, but I’m a bit more reserved on the software side. If you’re interested you can read my feedback in the following thread:

I played an hour or two of Eve: Valkyrie too. I think the game is currently overpriced ( 60$ ) for what it is ( multiplayer team deathmatch in 5-6 maps ). There aren’t too many players, and most games are filled with bots. It’s not exactly impressive technically or in terms of gameplay ( low-res textures, although I do like their special effects / particles ), but boy… once you put the helmet on… dat immersion. But I hope they add more content and more game modes soon.

25 Likes
28

Friday, 15 April 2016

So, unless I’m missing something, XInput does not support rumble on the left/right triggers. Damn. It does support vibrations though, so I guess it’s a step up from DirectInput. But it looks like XInput is getting obsolete and Microsoft is replacing it with their new gaming api in the universal crt. Sigh.

I’ve been working 3 days on the new input mapper. The input mapper is the engine component which maps key bindings to actions. For example, you could map “P” to “power toggle”. The old input mapper had a lot of limitations, so I simply scrapped it, redesigned it and rewrote it.

The new input mapper is much, muuuch more powerful. In fact, in the context of Battlescape some of its functionnality might not end up getting used. For example I added support for multiple users on the same computer. It also now supports additional mouse buttons 4 & 5, secondary ( alternative ) keys, and of course gamepads/joysticks. Everything behaves more coherently than before.

The hardest part was to add support for key combinations. Unlike the old mapper, it is now possible to assign actions not only to a single key, but to a combination of multiple keys. Ex.: “eject” to “leftctrl + E”. The downside is that implementation suddenly became more complex, because you have to detect which events might be triggered/untriggered based on state changes.

Another change was to move the key profiles/bindings out of ClientConfig.xml into their own profile xml files. This was done so that updating the client config would not always require rewriting the changes you might have made to the key profiles.

I’m putting the last touches on the input mapper today ( I’ve added a key detection mode, that I plan to use later on when we have in-game menus and you want to hit a key to remap an action, instead of having to edit the profile xml files with key names manually ). Tomorrow I’ll probably start working on the interceptor placeholder in the prototype. I don’t think I’ll be ready to release a new patch this week-end as the input mapper took more time than expected, but I should be able to do it next week.

28 Likes
29

Tuesday, 19 April 2016

I’m still working on the input mapper & controls. It feels like this thing is never ending.

So where did I go wrong ? Last time I was hoping to “put the final touches” on the input mapper, and that involved some testing with the actual game prototype. Guess what ? Everything was broken. Tssss, I should’ve known better.

So let’s have a quick look at what went wrong:

  • XInput and DirectInput have opposite conventions for the Y axes values. In other words, let’s take an x360 controller. If you press the left stick down, XInput reports -1.0, DirectInput reports +1.0.
  • Some buttons are recognized on DirectInput but not XInput. Example: XBox system button ( yeah, the big “X” in the top-center of the controller! ). If anybody knows how to detect it in XInput, please let me know. I see nothing in the API.
  • The left & right triggers range from 0…1 in XInput, but -1 to +1 in DirectInput. Which means, if the controller is idle, the axes ( other than triggers ) will return 0 but the triggers themselves will return -1.
  • … which wouldn’t be a problem if you had a way to detect if an axis is a trigger, or what its “idle” value is. I spent countless hours googling for a way to identify axes in DirectInput, to no solution. Therefore I cannot know whether, when I receive a value of -1.0, it means “axis in down position” or “trigger in idle position”.
  • … so I had to implement a trick by detecting the “idle” state of the controller. It checks, when the controller is idle, if an axis has a value of 0.0 ( it’ll be an axis like a stick ) or -1.0 ( it’ll be a trigger ).
  • … which doesn’t work, because after you’ve initialized the controller device, you receive 0.0 values for ALL objects until you press the controller at least once. At which point you’ll start to receive -1.0 for triggers in idle state. That seems to be a problem with the way hardware works ( interrupts vs polling ).

So DirectInput seems to have lots of problems, yet not all devices will support XInput, so we have to offer DirectInput as a fallback…

To make things worse, even XInput as an API is messed up. Let’s have a look ( here’s the link: https://msdn.microsoft.com/en-us/library/windows/desktop/hh405051(v=vs.85).aspx )

  • XInput 1.4 is Windows 8+ only, so we can’t use it because Windows 7 is our minimum target.
  • XInput 9.1.0 lacks some functionality ( granted, it’s nothing major, but it can’t do audio/voice or get battery information )
  • XInput 1.3 is an obsolete API ( dates from 2010 ).

Also note that XInput offers vibrations but not triggers force-feedback.

Not enough ? Here’s the best part: Microsoft is dropping support for XInput. They’re now moving on the gaming API in the Universal CRT. So we have a third API ( which we don’t support yet, as it comes with its own issues ) for input.

Sigh.

So for now, we’re using XInput 9.1.0 as the default and DirectInput as a fallback. From my tests, this seems to work pretty well and stable despite a few limitations. Triggers force-feedback isn’t supported. Wheels & HOTAS should theorically work on DirectInput too, however since I don’t have any of them they’re not tested. I guess we’ll figure that out later.

Last but not least, the input mapper key combinations isn’t fully working yet. Some keys are conflicting with each other, and when they’re not conflicting sometimes they’re overlapping. For example, at the moment the “ship yaw” event is assigned to the horizontal mouse axis, while the “cockpit head yaw” event is assigned to horizontal mouse axis + left control. Because both of these events “share” the horizontal mouse axis, when you press left control and move the horizontal mouse axis, both events fire at the same time. In other words, you move your head in the cockpit AND rotate the ship at the same time…

Considering all these problems I have no idea when I’ll be able to start on the interceptor placeholder. Hopefully it’s now only a matter of hours (days?) until the input mapper & controls is solved…

If you had a headache reading/understanding this post, just imagine how my head feels after a week working on these issues :wink:

27 Likes
30

Tuesday, 3 May 2016

Not too happy with the rate of releases of those “quick progress reports”. But for my defense, this last week I’ve been very busy working on patch 0.1.4.0 ( see patch notes here ).

The new input mapper seems to work quite well and a few people have already tried to map their joysticks/devices to the new system with success. Time will tell if non-mouse controllers have sufficient precision for combat, though. I did a quick dogfight with Madfly last sunday ( using my Xbox controller ) and it seemed okay, but I doubt I could hit a 1-pixel ship that is 10 Km away using a game controller. At lower ranges it might be do-able though. Unfortunately I don’t really see any easy fix to that problem, other than nerfing significantly rotational accelerations, which would slow down turning quite a lot. But that’s not really the kind of game I’d like to see, personally.

The new atmospheric flight model is in, too. It is significantly improved compared to the last version, but one thing we lost is the lifting capabilities of our ships. They behave more like heavy bricks, generating a lot of drag in atmospheres now. This is still WIP; I’ll probably reintroduce some lift in a way or another in a later patch so that you can pitch-and-roll more easily than yaw. I’m not sure yet how to increase the lift without resorting to cheating it in, but I’d like to minimize that cheating to the minimum.

An idea that’s been floating in chat and on the forums is to streamline the ship by its shielding. Shields aren’t implemented yet but it’ll definitely come at some point, which will reduce drag artificially quite a bit in a way that doesn’t feel too much like cheating. Max speeds in atmospheres will naturally increase too ( at the moment it’s hard to get past the sound barrier at 340 m/s ), and will provide some cool gameplay consequences to losing your shields in atmospheres ( suddenly your ship will shake a lot, causing drag, making you a sitting duck ). I must say, I’m quite fond of the idea.

Next steps

In the coming patch ( next week hopefully ) I’ll do another round of urgent bug fixes ( strafing at high target speeds with flight assist is currently broken ) and parameters adjustment ( especially on the atmospheric flight model ). I’m also planning to revisit warping mechanics ( long awaited personally ) and space physics. In particular, I want to introduce a limiter to the orientation of the ship which will prevent you from “looking” at a direction that’s too far from your flight velocity vector. This is the first step to combat “jousting”, however this will only work in flight auto assist. In decoupled mode you’ll still be able to point your ship at 180°, so the reason I nerfed decoupling ( by removing orientation stabilization ) in this patch was in anticipation of this change. I expect a lot of feedback from our dev-access backers on this change, as it’s probably going to be the most significant one in terms of physics / fun.

After this next patch we will enter a new development phase where I will concentrate on refactoring and client/server architecture updates ( as well as networking improvements ). There probably won’t be any new major content for a while for a month or two, but we’ll continue with regular parameter adjustment and bug fixes patches.

In July, it will be time to visit team and resource mechanics, as well as more weapon types.

22 Likes
31

Tuesday, 17 May 2016

Last time I mentionned a new patch, but it turns out it might be delayed for a while until we can make an update to the launcher.

Meanwhile I started to work on the new netcode and an update to our physics systems.

Until now, we’ve been working with the Open Dynamics Engine physics lib ( ODE ) 0.13.1. Unfortunately ODE is an old library. Very old. And it shows signs of its age, design and architecture wise. It doesn’t get updated anymore that often, but the worst part is that AFAIK it doesn’t really support multithreading, or any “new” advanced physics features. Pure single-threaded rigid bodies. Last time I looked ( last year IIRC ) its source code was riddled with singletons and global variables, which means I don’t have much hopes that it’ll get a proper architecture fix any time soon in the future. Therefore, we are considering switching to another physics engine… provided that we find one that satisfies our needs.

ODE, for all its flaws, has a double-precision mode which is critical for us to support large worlds ( aka a solar system ) seamlessly.

I investigated other physics engines, and here are my conclusions:

Bullet

So far seems to be the best candidate for a replacement to ODE. It does support double-precision too and seems more optimized. Unfortunately it seems to be in a pretty messy state regarding multi-threading and/or GPU support via OpenCL. Bullet3 is by default single-threaded, which is pretty shocking in 2016. It does have a WIP branch for OpenCL, but unfortunately it seems to lack features, doesn’t seem stable ( when I tested it some rigid bodies seemed quite unstable ) and worst, has an API that is quite different ( and needs much, much more work ) than its CPU counterpart.

PhysX

The other candidate is the industry veteran: NVidia’s PhysX. It seems really good performance wise, however it has a major show stopper: it doesn’t support double precision. Doing some googling, I found out from one developer that due to SSE optimizations it couldn’t support fp64, despite the fact that they have a type that can be changed in the source code. Seems like they’re actively working on “large world support” though, but who knows when that will be ready…

In the coming days, I’m going to be working on updating our physics interface as a generic, neutral API that can support any of these physics engines. It’s already partially the case ( the way the I-Novae engine is designed is through modules and interfaces; therefore we have a physics interface and a module/DLL that implements it in ODE or any other physics lib ) but there are a lot of concepts/objects that are still ODE-specific, and that I need to make more generic. I’ll probably end up implementing a partial Bullet module and a partial PhysX module, just to validate that our new physics interface is compatible with all of those in the future.

I’ve already started to work on these 2 new physics modules ( Bullet & PhysX ) while maintening the old one ( ODE ). I must say, oh boy… where to start… compiling all these solutions is a nightmare. Bullet uses cmake which means it cannot generate a single solution for both win32 and x64, which means integration into our own I-Novae solution ( which does have a win32 and x64 configs ) isn’t straightforward. Worse, I’ve been running into CRT conflicts, as we use Multithreaded DLL ( /MD) while Bullet uses Multithreaded /MT by default. There’s an option to use /MD in cmake, but guess what… it breaks Bullet’s demos/sample browser. A fix is available but requires updating the make files manually :frowning:

PhysX integration so far has been going smoother, although I had to solve CRT conflicts too. I went as far as being able to initialize the physics world. PhysX is multithreaded which has some consequences on the design of the module, which is the main reason I’m interested in updating our physics interface to be future-proof.

24 Likes
32

Wednesday, 1st June 2016

Feel like forever since I wrote the last update :slightly_smiling:

Physics refactoring is almost done. I’ve been able to clean up a lot of design issues with our physics interface and to make everything compatible between ODE / PhysX / Bullet. In terms of implementation, ODE is back to working again, however I haven’t fully tested all the features, so there might be some surprises when I re-test the Battlescape client later on. Bullet only utilizes a bare framework ( initializing / shutting down the lib ) and no functionality. PhysX is in a better position, as not only it is initializing and shutting down the lib, but it also has support for rigibid bodies dynamics ( I’ve been mostly testing with boxes, adding new types of bodies should be straightforward later on ). I’m much more confident in our physics modules now.

The linking CRT conflicts I mentionned in my last update have been solved for Bullet & PhysX. I also tested PhysX with multiple threads, and it seems to work, but for some reason performance seems quite… unstable. In some frames it’s very fast ( 15 ms ) and others relatively slow ( 60 ms ), and that’s with a constant charge of work ( and stable scenario ) so I’m not sure what’s going on. In comparison, ODE is single-threaded and performs at a constant 34-35 ms; so it seems slower in average, but its performance is a lot more stable. I dunno if that will turn out to be a problem later on.

Today I’ve been fixing a stupid problem with my PhysX implementation. After doing some tests in the past days, I realized something was wrong. Some boxes seemed to fall ( and stay ) partially under my test ground plane, as if some contacts weren’t correctly generated. It turned out it had nothing to do with contacts, and everything to do with… tadaaa… matrices. The orientation matrix had to be transposed ( aka reverting the orientation ), and suddenly everything seemed to match perfectly.

I’ve also started to move all our physics to an asynchronous task ( which will use a separated thread ). This has nothing to do with the single or multi-threaded capability of the physics API; I’m talking purely about moving our own physics interface to a separate thread here. So even ODE or Bullet would end up running in a separate thread, and not be locked to the gameplay thread.

This is a major, if not THE major improvement to our physics implementation, so I was a bit nervous when I started to do the changes, but I got something running in a separate thread within an hour ( even though I still have a lot of work remaining to synchronize all the different threads ), so I’m pretty happy with the progress.

Separating the gameplay / rendering / physics into their own thread is critically important for performance of the engine. So far, everything was running in a single thread. In terms of physics, this could easily lead to what is called the “spiral of death” ( see here for more info: http://gafferongames.com/game-physics/fix-your-timestep/ ), a well known problem for physics programmers. It happens when the time to simulate a time step takes longer than the time step itself. Another problem is when you have a very high framerate, but relatively slow physics, which leads to jerkiness in the framerate. Separating the rendering and the physics solves a lot of these issues, although it’s more complex to implement and debug.

32 Likes
33

Saturday, 18 June 2016

Physics refactoring is now completely finished, including a new asynchronous mechanism for ray casting. So far everything seems quite stable, so I hope we won’t discover weird new bugs once we test in multiplayer in the next patch later on.

Speaking of the next patch, I resumed work on it last week. The bomber got some design updates, but most importantely we’ve started to add support for the new space stations ( atm placeholders like everything else ). The main issue with space stations is one of design variety. We didn’t want to model 2 or 3 stations and copy/paste them dozens of times in the solar system ( Star citizen / Elite Dangerous basically did this ). Instead we decided to go for a modular system. Jan designed and modelled approximately 50 blocks ( “station modules” ), which can get assembled together in various ways. We then created 7 different configs ( to start with; the full game will have dozens ), each resulting in pretty different station designs, although all made from the same basic blocks. I must say, I am very happy with the results, even at this stage. I’m surprised by how different the stations look like (in terms of global details & shape), while technically re-using all these blocks minimizes our workflow and allows us to get away with a lot of content, for a minimal budget. I won’t spoil anything yet, but you’ll see them in the next patch.

I’m not sure yet when that next patch will be released. There are a couple of out-standing issues to fix before we can make it. Keith has upgraded the launcher to support our new patching system, but unfortunately it’s not functionnal yet. Even if the launcher worked, I still have to revamp the player spawning system ( we need it to spawn in these new stations ), and I’ve also noticed a couple of… interesting… sound bugs. I also have plans to tweak some ship/atmospheric parameters as well as rework the warp systems, although that could get delayed to the next patch. We’ll see how it goes.

28 Likes
34

Thirsday, 21 July 2016

Revisiting keyboard input

Today I made a new pass on our Input system. HOTAS seemed to miss some inputs ( Z axis ), so I added “sliders” from Direct Input ( which are basically 2 extra axes ) which will hopefully address the problem. As I was doing that and running some tests, I realized some other things were broken/bugged. The most mind-boggling thing was Alt-Gr generating a “left-control” key pressed event. After investigating, it seems like it’s not a bug but ( thanks to Microsoft ) a “feature”. Don’t believe me ?

I found a blog which explains exactly why Windows Input is in such a sorry mess. From API bugs ( yup ) to virtual keys mapped to more than one key ( yup ) to weird behaviors, if you’re a little into programming you’re going to be amazed:

As advised by this article I switched from Windows messaging to the Raw Input API and these problems are now solved. As a bonus, keypad enter is now differenciated from the normal return key, and numlock no longer affects the key code.

Deployment status

Our new deployment platform is now pretty much done. Keith is finishing the last touches ( and fixing the last bugs ). One of the main delays happened 2 weeks ago when he upgraded his dev environment ( Visual Studio ) to Update 3. This broke our code ( internal compiler error, never a good thing ) and forced us to switch from DLLs to static libs everywhere. The good news is that we had planned to do that for a while, so it was the occasion to do it. It solved various dependency issues and has more potential for optimization. On the downside, compiling ( especially in release mode ) takes a lot longer now which is really annoying when you have to iterate a lot on a program.

Another downside is that it introduced various subtle bugs, like for example the order of static variables initialization. It took a couple days to fix all these new bugs.

Patch status

The new patch is pretty much done now and is scheduled for a public release some time by the end of the week end. It has a bunch of new content ( the new space stations, the new cockpit interiors etc… ) and tweaks ( the atmospheric flight model no longer caps speeds to ~300m/s, as I introduced a drag coefficient per ship ). Also some small performance improvements ( physics now run in a separate thread ) and fixed some shadowing artifacts ( light bleeding line between shadow cascades ).

As a side note, one of these performance improvements required shader model 5.0. I don’t think there’s any non-sm-5.0 video card that could run the game at a decent framerate anyway, but in case, if you tried to run the game on a super old card and it no longer works, you’ll know why…

23 Likes
35

Saturday, 20 August 2016

Last week, I came back from a week of vacation in France. It took a bit of time to get back on tracks, and I ordered a new PC config ( cpu: 6700K; gpu: gtx 1070; ram: 32 GB ) that should let me work in good conditions until Battlescape is released. I received it a few days ago and everything seems to run well. I panicked for a few minutes when I saw that one of the drives wasn’t detected by Windows until I realized it hadn’t been formatted yet ( unlike the other drive, which came up as pre-formatted ). It took a few hours to install Windows 10, all software ( 2 hours just for Visual Studio 2015, tss ) and tools. But the worst part was transferring all my data from my old computer to the new one, using external USB drives. Even on USB 3, it took almost an entire day to complete the process. My entire I-Novae Studios folder is over 200 GBs as I’m writing this, and that’s excluding all the old Quest-For-Earth data ( contributions, backups etc… ).

Unfortunately, the new patch with factories and land bases is not gonna get released this week. I already have got factories in game, but I’m missing a fundamental part: mapping factory modular assets to the planetary terrain. At the moment, fixed coordinates are specified, but due to the uneven terrain ( even on areas that look flat overall ) many buildings are floating a bit above the terrain. Mapping buildings to the surface isn’t trivial, simply because the server currently has no notion of heightmaps/displacements, so I cannot query the “altitude” on which a building should be spawned at. Our procedural heights generation is tied to materials which themselves require a renderer… which the server does not have. So we’ll need a bit of brainstorming to address this issue.

We’re also facing multiple cooking issues, from random freezes to crashes. As more assets are added to the project ( the latest one is Jan’s bomber’s cockpit ), we seem to hit undiscovered bugs & issues, which take some time to fix. We’ve also identified quality issues between the editor and the game, even in older assets ( like the fighter’s cockpit ). For some reason, some textures/shaders are correctly displayed in the editor, but once cooked and imported into the game, look of a relatively poor quality. I suspect some settings aren’t properly exported when cooking.

Keith is wrapping up the pledge upgrades and will soon go back to working on the various issues discovered on the launcher. Between missing dependencies, wrong install directories, to having to relog in order to be able to install the game, the launcher still needs some polish. We’ll eventually skin it with a visual style too. I’m personally pretty fond of Blizzard’s launcher which has a pretty slick look & feel.

Finally, Kristian is finalizing the hauler. He created new texture packs and is currently adding decals and lights to the asset. I’m not sure when that ship will be imported in game, as it’s supposed to be a NPC ship hauling resources from factories to space stations. It will need AI but we’re still far from being at this stage, so meanwhile it’s possible we’ll just spawn a couple of haulers in various places statically, just so that people can look at it.

20 Likes