Quick Progress Reports - Engineering

Battlescape
15

Wednesday, 17 Feb 2016

Yay, it is Christmas ! Vulkan is out ! Woo hooo… better late than never :wink:

I haven’t spent too much time on it yet, only an hour or two browsing the API docs, samples, and simply getting a “feel” for what this new API looks like. Here are my initial thoughts ( which may very well be wrong, you know, those are just first impressions ).

  • The concepts look pretty much identical to DX12’s. The total lack of surprise is a surprise in itself :smile:
  • No, seriously. At this point I’m wondering why the trouble of coming up with a different API. They’re so similar. The only difference is that Vulkan is not MS-proprietary, but other than that, if you simply renamed a bunch of functions from DX12 you’d end up with an API that looks pretty close…
  • It feels like DX12 and Vulkan are little brothers to each other… I won’t go as far as to say twins, because there are still some differences, but conceptually… yeah…

Now the positives/negatives that I could spot:

  • :heavy_plus_sign: :heavy_minus_sign: The API seems to have been designed with future expansions in mind. Lots of parameters are currently left unused for future use. That’s both a plus and minus, a plus because it should theorically make future expansions easier; a minus because it makes it look more complex than DX12.
  • :heavy_minus_sign: :heavy_minus_sign: The API is awfully verbose. By that I mean that they chose to pass function call arguments as structs, and sometimes structs-into-structs. This means that every VK call requires a ton of lines, a lot more than DX12. Sometimes it gets to ridiculous levels, I’ve seen an example with a single function call requiring as much as 60 lines of setup, just to pass parameters. Look at that one for instance: 2800 lines for a… cube ? Wtf ?
  • :heavy_plus_sign: At first glance, synchronization through events or fences seem easier. I’m not sure I’ve understood the difference with mutexes just yet.
  • :heavy_plus_sign: Resources creation / allocation also seems easier than in DX12 at first glance.
  • :heavy_plus_sign: Vulkan introduced the concept of a Render Pass, which I think wasn’t present in DX12 unless I missed it ? Not sure how useful it will be, but it might potentially be interesting.
  • :heavy_minus_sign: :heavy_minus_sign: Extensions crap, please, nooooo. Release day and already a dozen extensions. You know that OGL got annoying because of its extension hell, right ? With hundreds of often vendor-specific extensions and multiple ways of doing the same thing. It was a jungle. Do not repeat the same mistake. The fact that we need an extension to create a swap chain ( one of the most basic things in creating a rendering surface ), day #1, already makes me angry. I get it, you did that because it’s OS dependent, but don’t tell me there was no way to make that part of the core, I won’t believe you.

So in the end, are we going to switch to Vulkan or DX12 ? The fact that Vulkan is cross platform is a big advantage. The fact that it will be available on Windows 7 ( whereas DX12 is Win 10 only ) is another big one. So even if the flaws mentionned above end up being very real, I think the good outweights the bad. That doesn’t mean we’re jumping in adding Vulkan support to the engine any time soon though. First of all, we currently lack the resources to do so. This will be a major technical challenge. The entire renderer will have to be remade, which is man-months of work. This might have significant implications on to the rest of the engine too, maybe the scene graph. So the first step is simply to wait for the API to become more mature, to gain more experience with it ( we’ll probably run some experiments with it first ), and once we gain confidence, assuming ressources permit, we’ll switch to Vulkan. That definitely won’t happen until at least next year, if ever.

25 Likes
16

Friday, 19 Feb 2016

A couple days ago, we did a bunch of editor improvements / fixes. Some things were getting really annoying, for instance opening the model viewer was taking minutes, even for a simple object. After investigation we discovered that it was caused by the wireframe material…

  • There isn’t just one, but actually two wireframe materials. They’re not compiled on-demand ( like when you press the wireframe button ), but the first time you open a model…

  • Those wireframe materials were compiled for all shader models ( SM3, 4, 4.1, 5 ) even though only one is in use at a time.

  • Those materials were compiled with max shader optimization level, when the user probably doesn’t care about performance in the first place.

I fixed the 2 last points, which reduced the initialization time from a few minutes down to around 10 seconds. That’s still a lot just to open a cube model, I think we’re definitely going to have to implement a caching or precompilation mechanism for some special materials.

In other news, I’ve started to investigate decal materials. Before I get into details, I must make it clear we’re not talking about decals as in “decal billboards” such as damage effects here. We’re talking about decal geometry, such as what Star Citizen uses to add all their details to their models. There is plenty of information on the technique on the net, so google for it if you’re interested in the topic.

This is a major art pipeline change that we’ve wanted to use for a while now. It does have a lot of benefits compared to the old “uv-map, skin and make unique textures for every single 3D model” pipeline:

  • Higher quality: decal geometry can re-use texture chunks, so all details are much higher quality than what you could do in a unique texture. Imagine a screw in a cockpit for instance. With a unique texture, there’s no way you’re gonna have any decent resolution for the screw details. With decal geometry, you simply add a quad which samples another texture with the details at a much higher resolution.

  • Lower VRAM: this is actually the #1 motivation for using this technique. The prototype currently uses close to 3.5 GB of VRAM, and it’s not like it features dozens of different ships. We really need to decrease our VRAM usage, and decals help a lot.

  • Faster work: a side effect is that you no longer need to model a high-poly model, uv map it and compute the difference to generate a normal map. That saves a lot of modelling time. You still need normal maps for the details though.

It also has some downsides:

  • Higher polycount: decal geometry is geometry, even if simplistic, it definitely increases the total model’s polycount.

  • How to do unique texture details, such as rust, scratches, etc… When those details are isolated, you can do them as decals, it’s easy. But when those details are shape / geometry dependant ( such as rust/weathering near plating edges ) it becomes problematic. We think that’s probably the reason with SC’s ships look all shiny & clean. Our artists are currently investigating what are the limits of this technique in this area.

  • Level-of-detail and Z-fighting. I don’t think Z-fighting will be too much of a problem ( SC seems to have lots of it when far away, but we do Z-buffer tricks due to the nature of our planetary engine, which makes those problems disappear ). LOD however, well, you can’t simplify the decal geometry since it’s already quite simplistic to start with ( if a screw is a quad, you can’t really simplify it… ). The other problem is that if an asset has a lower LOD level, if you use the decals from a higher LOD level, the two don’t match anymore. We planned to do automatic LOD, but I’m starting to think that won’t be an option due to this issue. Artists might have to manually create every single LOD and their matching decals.

On the engine side, I’ve started to implement the missing features to support these geometric decals & materials. Exposing things such as a ZOffset/Zscale to the user ( to avoid z-fighting ), a render order index ( to ensure that the material is rendered after its base material ), writing masks ( some decals might only want to write normals, others only diffuse, others both… ), alpha blending or masking etc…

Alpha blending ended up being a major issue with our deferred renderer, and incompatible with our current GBuffers setup. I’m going to have to change it so that we can do decals blending. I’ll probably work on that today the entire day.

25 Likes
17

Monday, 22 Feb 2016

This has been a pretty hectic week-end.

We have conducted our first deployment test. As you can see from your ability to read this and to still go over our forums and website, it hasn’t been a total disaster, ahah. In fact, you could say it’s been pretty successful. The Kickstarter / IndieGogo databases have been merged to our I-Novae “live” DB, so people with Dev access should now be able to see the Dev-restricted forums. Everybody should have their forum badges ( we haven’t had time to replace the graphics just yet, so it’s using generic icons for now ). And the first version of our web api is up and running, which will be the foundation for running game servers.

Speaking of which, I’ve uploaded a first build of the server to our Azure virtual machine, and it runs but… game clients can’t connect to it. A new issue with server certificates… dang it. I’ll be investigating that in the coming days. But overall, everything is more or less working as expected.

The next step ?

We are tentatively going to make the dev-access game prototype release this next week-end. At this moment, we can’t announce the exact date or time yet, but it’s probably going to be saturday 27 or sunday 28. That’s assuming we can fix all our last issues and that we don’t discover new tricky ones.

To verify if you are elligible for playing, use this page to check for your game access:

https://inovaestudios.com/Battlescape/Access

The procedure is pretty simple, you simply have to associate your Kickstarter/IGG e-mail to your I-Novae account. Either your main account’s e-mail addresses already match, and you’re good; either they don’t match, and you need to add a secondary e-mail address ( and verify it ). Make sure you don’t introduce typos, because those can’t easily be fixed.

If you think you have entered all these information correctly but your access still doesn’t look right, shoot us an e-mail ( or send me a private message on these forums ). Make sure to copy/paste the content of the box from the access page, and include your user name and e-mail address(es). You can also leave a comment in this thread:

23 Likes
18

Tuseday, 23 Feb 2016

We are pretty satisfied with how things have been going, deployment-wise. Which is not to say that there are no issues… but to be honnest we were kindda expecting more issues. However we haven’t announced the access page “officially” yet ( it’s only referenced from this forum ), but at some point in the coming days, once we’ll be more comfortable with our ability to release the prototype this week-end, we’ll send a newsletter and make an official announcement.

Here’s the list of currently known issues:

  • website refreshing issues, mostly under Firefox. When you sign up for a new account or log in, firefox doesn’t handle the redirection correctly, leaving you thinking nothing happens. Just hit F5, or use another browser. We’ll address these issues once we have more time… which means, probably not too soon :frowning:

  • desynchronization issues: the main website and the forums ( Discourse ) use 2 different databases, which we try to keep synchronized. Sometimes it happens that you’re seen as logged in in one but not the other. You might have to re-login regularly on the forums. We’re still not sure how this happens. Clearing your cookies might help. Testing another browser too.

  • changing your main e-mail address on the forums: a lot of people went in the forums to change their e-mail address for their account to match their Kickstarter’s. Unfortunately, due to the 2 databases being distinct, the main account’s still using the old one. Until we fix this, you can simply use the “add e-mail” feature from the Access page to add your new e-mail. Try to avoid changing your e-mail from Discourse; Ideally we’d simply disable the feature, except that changing the Discourse sources is not easy, since it’d get overwritten by any update we do to the forums…

  • developper forum access and badges: when you add a secondary e-mail to your account, the access rights are updated immediately, however the dev access and the badges are only refreshed every 24h to avoid stressing the database. So you won’t immediately get them. Just be patient.

  • secondary e-mails: the add a secondary e-mail functionnality is still experimental, but we’d like to remind you that these e-mails still have to be confirmed before you get granted access. Make sure to check out your spam folder !

Yesterday I was able to connect the client to the game server. These damn certificates are driving me crazy. Anyway, this now works. I was able to test the public server registration and how it appears and gets refreshed in real-time in the launcher’s server browser. I added some better error handling and messages. I tested the client on a brand new machine, and started to run into trouble in terms of dependencies.

The goal is still to provide a .zip that contains the launcher and the game client. However, there are many redistributable packages that need to be installed for the executables to run. Those currently are:

  • OpenAL redist
  • .NET framework 4.6.1
  • VC 2013 x64 redist
  • VC 2015 x64 redist
  • Universal CRT redist

These will be included in the .zip for easier installation, but you’ll still have to install them manually.

Today, a lot more little fixes and a polish pass on the launcher. Time to redesign some screens a little bit !

28 Likes
19

Friday, 26 Feb 2016

I won’t lie: it’s getting tough. Release definitely won’t happen saturday, but there’s still hope for sunday.

On the engineering side we’re almost good to go. On the content side however there’s still a bit of work to do. We need to integrate some of the new assets / placeholders, generate the configuration files for them, finalize the new ship ( placing lights, sockets for thrusters/guns etc… ), tweak the physics parameters, update the planets textures and a ton of other things like that.

Once everything is done, we still need to do a final cooking for all the assets ( to optimize them ), generate the build ( which involves running a bunch of scripts ) and upload it. The entire process, to generate a single build, takes 4-5 hours.

So yeah, I’m quite busy and I don’t think I’ll have much time for a new progress report until the release.

41 Likes
20

Monday, 29 Feb 2016

Warning: long post incoming.

So, here we are. The long anticipated “developer-access” launch has happened !

We were crunching hard all during these past days, some of us not going to sleep before 4 to 7 am. I personally have been crunching for most of February, but even so, this was a “soft crunch”. These past 3 days however… oh boy… last time we worked that hard was the days before the Kickstarter launch. Needless to say, we’re all pretty tired and we’re gonna slow down a bit for the coming days.

So, how did the launch go ? In fact it went pretty smoothly. Which is not to say that it was perfect, it absolutely wasn’t, and many people had issues with their accounts access rights or the installer. But there wasn’t any major issue ( other than some well known bugs ) in the game itself. No major stability issue, no major framerate issues ( a lot of users seemed to be pretty happy with the performance, and was above their expectations. A few even ran it on laptops at decent framerates ).

But overall, we are most than satisfied with the launch, and we think it has gone as well as we could realistically expect considering our low resources and budget.

Personally, on a scale from 1 to 10 ( 1 being a catastrophic failure and 10 the perfect launch ) I’d say it was a solid 7 or a 8.

The game was launched yesterday at around 8 pm GMT.

The server hasn’t crashed once during that time, and we had a peak of 35 players a few hours after the launch. It was really fun to see players getting used to the controls of the game, fly around, explore planets and shoot at each other. Then press ESCAPE by accident. My evil plan has succeeded, it seems… :smiling_imp:

And so much for the “cease fire” gentlemen’s agreement. Within minutes, everybody was firing at each other and chasing others around the asteroids. I even got killed a few times :rage:

While the server itself was stable, it seems like there was a few network hiccups, with sometimes players failing to update for many seconds. After a while it stabilized again. The server isn’t exactly too powerful and it has limited bandwidth, but I don’t think we were close to the limits. We might have to investigate later with a better server and more stable bandwidth.

So far feedback from our testers on the prototype has been overwhelmingly positive. Of course, the game isn’t perfect and features a lot of placeholders and systems that are barely functional ( I’m looking at you, warp systems :wink: ). There seems to be a bug with flight auto-assist disabled that prevents you from side strafing; the annoying cockpit flickering bug once you die is still there; but overall it’s working pretty well for a first release.

I’d like to share two quotes ( I won’t name the authors, they’ll recognize themselves ) that made me burst into laughter when I read them.

“Man, I must’ve been following this off and on for 9 years. It’s kinda surreal to be playing this prototype”

and

“whaouu this new cargo ship is awesome !!”

The first one is self-explicit. The second one made me laugh because the “cargo ship” is an untextured placeholder, and I definitely wouldn’t quality that as “awesome”.

A lot of the new content on the asteroid base didn’t even exist a few days ago. Two or our artists, Jan and Dan, worked on it until the last minute ( adding the hangars where you spawn for example ) to give the asteroid base some sense of scale from these industrial insallations. It’ll definitely look much better once textured and lit.

Kristian has made new texture packs for planets ( I love some of the new Mars-like surfaces ) and the new asteroids. They are more colorful and varied than the ones from the Kickstarter, although I’m sure nobody remembers :smile:

The UI also got a revamp, but it is functionaly identical to the KS one. It’s still a placeholder though. In a few months we’ll do a serious pass on it, both in terms of functionality and visual look.

The bomber ( the new flyable ship ) currently behaves more like a fighter. This is a conscious choice, since I didn’t want player’s first experience to be with a sluggish ship that has no missiles or torpedoes yet. Once we have an actual fighter implemented into the game, there will be massive tweaks to all controls and physical parameters to make the ships feel more like what their roles imply.

Before I close this post I should list 2 VODs of the event from yesterday:

and

I’d also like to mention Caesar’s 4K screenshots which are absolutely stunning:

31 Likes
21

Wednesday, 2 March 2016

Yesterday the plumber came again to repaint my appartment ( yup, you read that right, the plumber is also a painter. Yeah, I was surprised ). Therefore this has been a forced vacation day.

… or almost. I still had a bit of time to investigate some issues, work on the facebook authentication problems, do a couple of bug fixes, and help some people who seemed to have access issues. I can’t say it has been the most productive day though.

Since I was forced to early retirement for a day, I investigated some hardware replacements that I had been delaying for a while. In particular, my main screen monitor is borderline garbage now ( it’s 8 years old ) with washed-out colors. I’m planning on ordering a new QHD ( 2560 x 1440 ) monitor. I won’t go higher in resolution for price reasons and lack of a video card that could handle UHD anyways…

Short-term road-map ( March )

We are planning to release a first patch to the game client today. This will be an optional patch, and will mostly address some of the most urgent / pending issues, like those people being unable to connect through Facebook. There will be a bunch of minor tweaks too.

Looking ahead, this month we are going to focus on stabilizing the game, sorting out access rights / launch problems or other technical issues, adding new placeholders and doing some tweaks.

  • Keith will mostly be busy on finishing the launcher’s automatic patching system.

  • He also has to do some website updates, and resume work on our custom pledge system.

  • We want to get ride of IndieGoGo as soon as possible. The way we’re forced to poll at regular intervals to get the list of backers is extremely annoying and taxing server resources. There are also update delays, which means backers have to wait a bit for their data to be refreshed before they can access the game. Quite annoying for everybody.

  • Our artists are going to work on concept art for, well, the entire game.

  • They’ll also continue working on the existing assets, experimenting with the new artist pipeline ( based on decals ), the cockpit, better structures for the asteroid base, some new lights placements…

  • And, most importantly, producing a new set of placeholder assets ( those will be untextured too ) which we can use to iterate game on fairly quickly.

  • As for me, I’m going to keep working on client issues and gameplay.

  • For example my top priority will be to figure out the weird cockpit flickering bug. I’ve already tried a few hours during the KS campaign, but due to being so busy I couldn’t keep working on it. Now the dev-access release is done, I’ll have enough time to investigate it properly.

  • I’ll also address some of the easiest / most frequently seen remarks or bugs. Drifting failing while flight auto assist is off ? Sounds like it’ll be an easy bug to fix. There’s probably going to be a lot like that.

  • I’m also going to start some of the more heavy work on the client/server. A lot of data is client-sided at the moment, which is no good. I did it that way before the KS because it was faster to implement, but of course this cannot stay like that.

  • I’ll also work on adding the new placeholder content to the game.

  • When I have a bit of free time I’m going to do some experimental improvements on the look of planets. They’re not yet at our target level in terms of visual quality.

  • One thing I will NOT do just yet is to address the elephant in the room: the networking hit combat issues. Those aren’t easy to solve, in fact going forward it’s probably going to be our #1 challenge. We’ll wait until the game has stabilized a bit until we revisit it.

Mid-term road-map ( April - summer )

  • We’ll address the network issues at this stage.

  • We also need to do add better multithreading to the engine, do a bunch of optimizations, implement streaming and more engine work.

  • We’ll start experimenting with Vulkan, although I can’t give a timeline on that just yet.

  • We’ll also start doing experiments for our in-game GUI solution.

  • We also have to update or replace a bunch of old third-party libraries: physics, sound ( OpenAL is really obsolete at this stage, we’re planning on using Wwise ).

  • That’s pretty much when we’ll add new ships, new weapons and iterate on the real gameplay: attacking/defending objectives like factories or NPC ships.

  • We’ll add more planetary environments at this stage too.

Long-term road-map ( Q4 2016 and later )

  • Content in the game will still be pretty thin at this stage, but we’ll work diligently on increasing it. There will still be some placeholder content remaining.

  • Alpha will be released around that time.

  • Balancing the game “seriously” will also start around that time.

  • Small ships gameplay should be faily well established at this point, but capital ships gameplay will require a lot more work.

  • Note that the longer ahead we are looking at, the less certain we are about development priorities. A lot of things can change depending on how things are going. So take these roadmaps with a grain of salt.

40 Likes
22

Saturday, 5 March 2016

Patch 0.1.2.0 is out, woooh hoo. Adresses some issues with Facebook authentication in the launcher, detail textures for the Glimmerfall asteroid ( we’ll soon extend these detail textures to the roids in the rings ), tweaks to the HUD and a bunch of bug fixes.

Our artists have started to work on the new concept art for all the game’s content, which will be followed up by a bunch of placeholders. I’m not sure yet how the ships placeholders will be integrated into the game since we have no GUI to select a startup ship. Maybe a ship will be selected randomly when spawning, or maybe we could add a preference setting in the config file. We’ll figure that out later.

A few users are still having issues to launch the game. It seems related to fonts. One of them has a problem with the launcher’s fonts ( which uses Windows standard fonts, so not sure why it’d be a problem ) and the other has a problem with a game’s custom font ( the function we use to load / install that font is quite simplistic so I’m not sure what could go wrong ). Still investigating.

I’d like to remind everybody that warp mechanics are barren. I’ve only worked on them for a couple hours in the past months, so they’re in need of some serious love. The sudden acceleration / slow down is one thing I want to address soon. Overall I’d like the warp to work like an Alcubierre drive, special effects coming at a later stage, but functionally it shouldn’t affect the ship’s current velocity. The current implementation of the warp simply changes the acceleration by a factor which is depending on gravity ( so the closer you are to a planet, the lower the warp factor ). That gravity dependency is something I’d like to keep, however the fact that the “real” velocity changes during warp is something I’d like to remove. Oh, and for long-term travel, we’ll probably use a different mechanism which provides FTL. We want players to “jump” to the other side of the solar system in like 30s, not half an hour…

Some of you might have noticed “Glimmerfall Ops” in the middle of the rings, far away where the Glimmferall asteroid actually is. It’s not technically a bug, simply a debug test I forgot to remove from the server entities config. My bad. Will remove it next time the server crashes. Which can take a while. As I am speaking, the server’s uptime is close to 5 days. Before that, it did reboot… because Windows forced an update and had to reboot. Prior to that it hadn’t stopped since the launch last sunday. Therefore, it’s official: the server hasn’t crashed once. Of course, we haven’t changed much since the KS campaign, but going forward once we resume severe work on the server, it’s probably going to crash a lot more frequently.

A lot of people have reported bugs or issues though our launcher’s report tool. It seems to work pretty well. On the list of issues, some are false positives, like players who did not immediately understand how things work. That’s fine, keep reporting everything, we can sort out everything on our end :slightly_smiling:

One “bug” that I see people report often is in fact not a bug: the lights flickering on the ship in 3rd-person mode. We have a basic damage system; when a light gets hit by a blaster shot, it gets damaged and flickers randomly forever. But I admit it looks like a bug. However what is a real bug is the cockpit visibility flickering, which I’m going to work on diligently in the coming days ( hoping, not weeks… )… but it looks like a tough nut to crack.

There are a few bugs which I haven’t been able to replicate so if anybody has hints, feel free to make a post:

  • Players showing up as “Unknown” via Google login: each time I tested it used my account’s name ( INovaeFlavien ) so I’m not sure how to replicate it.

  • A few crashes: please send your log file, not your crash file. I think we’re going to get rid of crash files, cause they contain less relevant information than log files, and most players actually send those instead of the log files :frowning:

  • It is useless to send us a crash report without the log file. As a reminder: the log file is located in C:\Users\YourName\Documents\Infinity Battlescape\Logs . A report just saying “I launched the game and it crashed” is unfortunately not very useful to us.

  • NPC ships lights shining through building walls: technically not a bug, we just haven’t enabled shadowing for NPC ship lights. We might do it later. I agree it’s sometimes confusing when you’ve spawned in a hangar and see weird lights shining from the walls :sunny:

32 Likes
23

Friday, 11 March 2016

Unfortunately these last few days I’ve had a bunch of IRL issues related to my Internet connection and digital TV. I won’t go into details, but at this point, I’ve wasted more than a dozen hours with my provider’s tech support, replacing equipment and having appointments with technicians to inspect my installation. The good news is that everything has been upgraded and that my problems are apparently solved ( although the problem kind of solved magically itself in the end so I’m not confident it won’t happen again ). It really sucks and frustrates me that I have to spend so much time on these issues, but oh well… for now everything seems to be working again.

On the code side I’ve started to do some refactoring of the client/server. This is only a first step and there’s still a lot more to do in the future, but it’s still an important first step. I was also investigating the ability for Visual Studio to debug a child process and came up with a cool plug-in which does it. Just install the plug-in and when you call win32 api’s CreateProcess it’ll appear in the debugger, call stack and threads list. Why isn’t this a standard in VS yet ? It seems to work perfectly and I see no immediate reason it shouldn’t be available in standard…

Next step now is to do a bunch of additions to our material editor. We’re missing some nodes which could be handy to our artists. Some of these nodes are already implemented on the engine side and were “forgotten” ( yeah that happens ) when we created the material editor ( like absolute value, 1 - x, swizzle ), and should take only an hour or two to add. Since Jan worked on the detail textures for the Gimmerfall asteroid we also realized some Photoshop-style filters would become handy, like an overlay node. It’s possible to recreate the overlay formula via other nodes, but obviously it requires the artist to assemble dozens of nodes to do so, so these new nodes will save effort and reduce the material node graph complexity by a lot. All these improvements should be done by this week-end.

After that, it’ll be time to start the serious investigations about this cockpit / thrusters flickering bug. I have a few ideas on what the cause could be, so I’ll start with that, and hopefully it won’t take too much time before I narrow it down.

24 Likes
24

Tuesday, 15 March 2016

I’m finally done with the material editor updates for now…

I’ve added a bunch of new nodes that artists might find useful. Here’s the quick list:

  • Abs ( taking the absolute value of a number )
  • 1-complement ( 1 - x, this is simple but saves a node in the graph, so it’s mostly to reduce graph complexity )
  • Swizzle ( shifting the components of a vector, ex.: vector.zxwy )
  • Triplanar mapping

hHAy58S.png1920×1079 264 KB

Also many blend modes from Photoshop / Gimp:

  • Overlay
  • Linear burn
  • Linear dodge
  • Linear light
  • Soft light
  • Screen

Back to working on the game now !

33 Likes
25

Sunday, 20 March 2016

My new screen monitor, which I received 2 weeks ago, seems broken. There’s a vertical line on the right side, which changes from white to purple to green sometimes. Quite distracting. And quite annoying for a brand new screen… now that my TV/Internet trouble are solved, I thought I’d be ok for a while… guess not.

As for development news, solving triplanar mapping has taken more time than anticipated. I thought it’d only take a few hours, but in the end I worked more than 2 days on it. In my last report I said that triplanar mapping was working, but that was only for diffuse textures. When I tried to use the same code for normal maps it resulted in incorrect normals due to the tangent space changing due to the triplanar mapping. In the end, I had to create a new tangent space for each projection axis, and then convert the weighted normal back to the “real” mesh’s tangent space. The resulting code is only 10 shader code lines long, but it took a dozen hours to figure out. I’m happy I’m not getting paid by the line of code…

Now for the good news: I’ve fixed a couple of “major” bugs in the client already, most notably the bug that was #1 on my list: cockpit ( and particle systems ) flickering. It ended up being a problem with the octree updates and some objects not being updated in the right order of dependency. It affected the cockpit, lights and particle systems, but technically all other scene entities too.

While I was working on flickering issues I investigated the reason why damaged lights continued to blink even though the ship is powered down. That was a different bug, where our blinking behavior component wasn’t correctly inheriting the light’s visibility status. In order words, the light was correctly set off, but its attached blinking behavior kept turning it back on or off. This is now fixed.

I’ve also fixed a bunch of other bugs which were less important like the color of enemy ship HUD icon when warping, changed light speed limit to its real more accurate value, crash fix in the launcher server browser when the list is empty, some client/server infrastructure cleaning etc…

Keith is continuing his work on the patching & installation systems and seems to be getting close to a first version. It might still take a while until it goes live though. Accounting & taxes have distracted him quite a bit these last 2 weeks.

On the art side, Jan, Kristian & Dan are working on the new ship concept art and placeholders. As soon as we have a first version integrated into our project I’ll make a new build including the new ships placeholders and bug fixes; hopefully that will be sometimes next week. They’re also making a bunch of asset fixes to the existing art. More on that later.

35 Likes
26

Friday, 1st April 2016

Not much to report lately. The art team has finished the first round of placeholders ( interceptor still missing but should be ready next week ), so I’ve been working on integrating all of these into the game. Which mostly means playing with configuration files, adjusting parameters, testing physics and stuff like that.

The corvette is unfortunately in a bit of a mess, currently. Its size is 115m, which is too big. We’re already talking about halving its size to around 60m, so it’ll need some redesign already. At 115m it becomes very awkyard to control through the mouse, which is going to be a problem for slower ships no matter what. But even at 60m, it can’t be controlled like an interceptor as it is thousands of times more massive. The problem with the current mouse scheme is that you need to keep rolling the mouse in order to turn around. Now it’s fine when the ship turns fast ( in a single hand movement you can turn by 180° ) but when your ship is massive, you need to repeat the roll movement many times. Therefore, I wonder if an alternative control scheme would work better; maybe something similar to Star Citizen or Elite: Dangerous, where the position of your mouse cursor compared to the screen center determines your ship movement. Feedback on this sensitive issue is most welcome. Note that for capital ships I’m thinking of a completely different movement system ( I’ll elaborate on that later when we implement cap ships ), so we’re only talking about 30m-60m sized ships here.

To be clear, I’d like to keep the current scheme for the fast, agile ships such as the interceptor. The question is, should we use a different scheme for slower ships like the bomber & corvette, or would you still rather use mouse-rolling movement for those even though it’d be more difficult ?

19 Likes
27

Monday, 11 April 2016

New patch 0.1.3.0 is out ! Not too happy with Azure US East datacenter going down the exact moment I was trying to update the server. The downtime was pretty long too, a dozen hours or so. So I’m not exactly impressed by their service, especially as we decided to go to Microsoft because we thought they’d be robust and that kind of incident would likely never happen.

Anyways, as some of you might have noticed, I changed some physics parameters for the ships in the prototype, and they now have a proper mass ( in comparison, the bomber in the previous versions always had a mass of 1.0 ton ). I compensated this increased mass by equivalently stronger thrusters, but unfortunately I overlooked other places in the code which didn’t take the mass into consideration ( in particular atmospheric physics ), and as a result the behavior of those ships in atmosphere is now unsatisfying. I will do another pass on this soon.

My plans for this week is to do major improvements on the controls and input mapping. I will add support for joysticks/gamepads through Direct Input and XInput ( I already tested support for the Xbox 360 controller and PS4 controller, but I’m pretty sure it’ll work out of the box for the Xbox one or PS3 controllers too ). The keyboard mapping profiles will get moved out of ClientConfig.xml into their own config files, and I will add better mapping of actions and support for more keys into the input mapper.

Kristian has finished the interceptor placeholder. It’s still very rough/basic modelling, but this week I will add it to the prototype as well as its cockpit. Expect another build soon, maybe next week-end.

That’s all for now ! Oh, and I received my CV1 Oculus Rift last tuesday, so I spent a couple hours playing VR games and testing demos. It works very well on the hardware side, but I’m a bit more reserved on the software side. If you’re interested you can read my feedback in the following thread:

I played an hour or two of Eve: Valkyrie too. I think the game is currently overpriced ( 60$ ) for what it is ( multiplayer team deathmatch in 5-6 maps ). There aren’t too many players, and most games are filled with bots. It’s not exactly impressive technically or in terms of gameplay ( low-res textures, although I do like their special effects / particles ), but boy… once you put the helmet on… dat immersion. But I hope they add more content and more game modes soon.

25 Likes
28

Friday, 15 April 2016

So, unless I’m missing something, XInput does not support rumble on the left/right triggers. Damn. It does support vibrations though, so I guess it’s a step up from DirectInput. But it looks like XInput is getting obsolete and Microsoft is replacing it with their new gaming api in the universal crt. Sigh.

I’ve been working 3 days on the new input mapper. The input mapper is the engine component which maps key bindings to actions. For example, you could map “P” to “power toggle”. The old input mapper had a lot of limitations, so I simply scrapped it, redesigned it and rewrote it.

The new input mapper is much, muuuch more powerful. In fact, in the context of Battlescape some of its functionnality might not end up getting used. For example I added support for multiple users on the same computer. It also now supports additional mouse buttons 4 & 5, secondary ( alternative ) keys, and of course gamepads/joysticks. Everything behaves more coherently than before.

The hardest part was to add support for key combinations. Unlike the old mapper, it is now possible to assign actions not only to a single key, but to a combination of multiple keys. Ex.: “eject” to “leftctrl + E”. The downside is that implementation suddenly became more complex, because you have to detect which events might be triggered/untriggered based on state changes.

Another change was to move the key profiles/bindings out of ClientConfig.xml into their own profile xml files. This was done so that updating the client config would not always require rewriting the changes you might have made to the key profiles.

I’m putting the last touches on the input mapper today ( I’ve added a key detection mode, that I plan to use later on when we have in-game menus and you want to hit a key to remap an action, instead of having to edit the profile xml files with key names manually ). Tomorrow I’ll probably start working on the interceptor placeholder in the prototype. I don’t think I’ll be ready to release a new patch this week-end as the input mapper took more time than expected, but I should be able to do it next week.

28 Likes
29

Tuesday, 19 April 2016

I’m still working on the input mapper & controls. It feels like this thing is never ending.

So where did I go wrong ? Last time I was hoping to “put the final touches” on the input mapper, and that involved some testing with the actual game prototype. Guess what ? Everything was broken. Tssss, I should’ve known better.

So let’s have a quick look at what went wrong:

  • XInput and DirectInput have opposite conventions for the Y axes values. In other words, let’s take an x360 controller. If you press the left stick down, XInput reports -1.0, DirectInput reports +1.0.
  • Some buttons are recognized on DirectInput but not XInput. Example: XBox system button ( yeah, the big “X” in the top-center of the controller! ). If anybody knows how to detect it in XInput, please let me know. I see nothing in the API.
  • The left & right triggers range from 0…1 in XInput, but -1 to +1 in DirectInput. Which means, if the controller is idle, the axes ( other than triggers ) will return 0 but the triggers themselves will return -1.
  • … which wouldn’t be a problem if you had a way to detect if an axis is a trigger, or what its “idle” value is. I spent countless hours googling for a way to identify axes in DirectInput, to no solution. Therefore I cannot know whether, when I receive a value of -1.0, it means “axis in down position” or “trigger in idle position”.
  • … so I had to implement a trick by detecting the “idle” state of the controller. It checks, when the controller is idle, if an axis has a value of 0.0 ( it’ll be an axis like a stick ) or -1.0 ( it’ll be a trigger ).
  • … which doesn’t work, because after you’ve initialized the controller device, you receive 0.0 values for ALL objects until you press the controller at least once. At which point you’ll start to receive -1.0 for triggers in idle state. That seems to be a problem with the way hardware works ( interrupts vs polling ).

So DirectInput seems to have lots of problems, yet not all devices will support XInput, so we have to offer DirectInput as a fallback…

To make things worse, even XInput as an API is messed up. Let’s have a look ( here’s the link: https://msdn.microsoft.com/en-us/library/windows/desktop/hh405051(v=vs.85).aspx )

  • XInput 1.4 is Windows 8+ only, so we can’t use it because Windows 7 is our minimum target.
  • XInput 9.1.0 lacks some functionality ( granted, it’s nothing major, but it can’t do audio/voice or get battery information )
  • XInput 1.3 is an obsolete API ( dates from 2010 ).

Also note that XInput offers vibrations but not triggers force-feedback.

Not enough ? Here’s the best part: Microsoft is dropping support for XInput. They’re now moving on the gaming API in the Universal CRT. So we have a third API ( which we don’t support yet, as it comes with its own issues ) for input.

Sigh.

So for now, we’re using XInput 9.1.0 as the default and DirectInput as a fallback. From my tests, this seems to work pretty well and stable despite a few limitations. Triggers force-feedback isn’t supported. Wheels & HOTAS should theorically work on DirectInput too, however since I don’t have any of them they’re not tested. I guess we’ll figure that out later.

Last but not least, the input mapper key combinations isn’t fully working yet. Some keys are conflicting with each other, and when they’re not conflicting sometimes they’re overlapping. For example, at the moment the “ship yaw” event is assigned to the horizontal mouse axis, while the “cockpit head yaw” event is assigned to horizontal mouse axis + left control. Because both of these events “share” the horizontal mouse axis, when you press left control and move the horizontal mouse axis, both events fire at the same time. In other words, you move your head in the cockpit AND rotate the ship at the same time…

Considering all these problems I have no idea when I’ll be able to start on the interceptor placeholder. Hopefully it’s now only a matter of hours (days?) until the input mapper & controls is solved…

If you had a headache reading/understanding this post, just imagine how my head feels after a week working on these issues :wink:

27 Likes
30

Tuesday, 3 May 2016

Not too happy with the rate of releases of those “quick progress reports”. But for my defense, this last week I’ve been very busy working on patch 0.1.4.0 ( see patch notes here ).

The new input mapper seems to work quite well and a few people have already tried to map their joysticks/devices to the new system with success. Time will tell if non-mouse controllers have sufficient precision for combat, though. I did a quick dogfight with Madfly last sunday ( using my Xbox controller ) and it seemed okay, but I doubt I could hit a 1-pixel ship that is 10 Km away using a game controller. At lower ranges it might be do-able though. Unfortunately I don’t really see any easy fix to that problem, other than nerfing significantly rotational accelerations, which would slow down turning quite a lot. But that’s not really the kind of game I’d like to see, personally.

The new atmospheric flight model is in, too. It is significantly improved compared to the last version, but one thing we lost is the lifting capabilities of our ships. They behave more like heavy bricks, generating a lot of drag in atmospheres now. This is still WIP; I’ll probably reintroduce some lift in a way or another in a later patch so that you can pitch-and-roll more easily than yaw. I’m not sure yet how to increase the lift without resorting to cheating it in, but I’d like to minimize that cheating to the minimum.

An idea that’s been floating in chat and on the forums is to streamline the ship by its shielding. Shields aren’t implemented yet but it’ll definitely come at some point, which will reduce drag artificially quite a bit in a way that doesn’t feel too much like cheating. Max speeds in atmospheres will naturally increase too ( at the moment it’s hard to get past the sound barrier at 340 m/s ), and will provide some cool gameplay consequences to losing your shields in atmospheres ( suddenly your ship will shake a lot, causing drag, making you a sitting duck ). I must say, I’m quite fond of the idea.

Next steps

In the coming patch ( next week hopefully ) I’ll do another round of urgent bug fixes ( strafing at high target speeds with flight assist is currently broken ) and parameters adjustment ( especially on the atmospheric flight model ). I’m also planning to revisit warping mechanics ( long awaited personally ) and space physics. In particular, I want to introduce a limiter to the orientation of the ship which will prevent you from “looking” at a direction that’s too far from your flight velocity vector. This is the first step to combat “jousting”, however this will only work in flight auto assist. In decoupled mode you’ll still be able to point your ship at 180°, so the reason I nerfed decoupling ( by removing orientation stabilization ) in this patch was in anticipation of this change. I expect a lot of feedback from our dev-access backers on this change, as it’s probably going to be the most significant one in terms of physics / fun.

After this next patch we will enter a new development phase where I will concentrate on refactoring and client/server architecture updates ( as well as networking improvements ). There probably won’t be any new major content for a while for a month or two, but we’ll continue with regular parameter adjustment and bug fixes patches.

In July, it will be time to visit team and resource mechanics, as well as more weapon types.

22 Likes
31

Tuesday, 17 May 2016

Last time I mentionned a new patch, but it turns out it might be delayed for a while until we can make an update to the launcher.

Meanwhile I started to work on the new netcode and an update to our physics systems.

Until now, we’ve been working with the Open Dynamics Engine physics lib ( ODE ) 0.13.1. Unfortunately ODE is an old library. Very old. And it shows signs of its age, design and architecture wise. It doesn’t get updated anymore that often, but the worst part is that AFAIK it doesn’t really support multithreading, or any “new” advanced physics features. Pure single-threaded rigid bodies. Last time I looked ( last year IIRC ) its source code was riddled with singletons and global variables, which means I don’t have much hopes that it’ll get a proper architecture fix any time soon in the future. Therefore, we are considering switching to another physics engine… provided that we find one that satisfies our needs.

ODE, for all its flaws, has a double-precision mode which is critical for us to support large worlds ( aka a solar system ) seamlessly.

I investigated other physics engines, and here are my conclusions:

Bullet

So far seems to be the best candidate for a replacement to ODE. It does support double-precision too and seems more optimized. Unfortunately it seems to be in a pretty messy state regarding multi-threading and/or GPU support via OpenCL. Bullet3 is by default single-threaded, which is pretty shocking in 2016. It does have a WIP branch for OpenCL, but unfortunately it seems to lack features, doesn’t seem stable ( when I tested it some rigid bodies seemed quite unstable ) and worst, has an API that is quite different ( and needs much, much more work ) than its CPU counterpart.

PhysX

The other candidate is the industry veteran: NVidia’s PhysX. It seems really good performance wise, however it has a major show stopper: it doesn’t support double precision. Doing some googling, I found out from one developer that due to SSE optimizations it couldn’t support fp64, despite the fact that they have a type that can be changed in the source code. Seems like they’re actively working on “large world support” though, but who knows when that will be ready…

In the coming days, I’m going to be working on updating our physics interface as a generic, neutral API that can support any of these physics engines. It’s already partially the case ( the way the I-Novae engine is designed is through modules and interfaces; therefore we have a physics interface and a module/DLL that implements it in ODE or any other physics lib ) but there are a lot of concepts/objects that are still ODE-specific, and that I need to make more generic. I’ll probably end up implementing a partial Bullet module and a partial PhysX module, just to validate that our new physics interface is compatible with all of those in the future.

I’ve already started to work on these 2 new physics modules ( Bullet & PhysX ) while maintening the old one ( ODE ). I must say, oh boy… where to start… compiling all these solutions is a nightmare. Bullet uses cmake which means it cannot generate a single solution for both win32 and x64, which means integration into our own I-Novae solution ( which does have a win32 and x64 configs ) isn’t straightforward. Worse, I’ve been running into CRT conflicts, as we use Multithreaded DLL ( /MD) while Bullet uses Multithreaded /MT by default. There’s an option to use /MD in cmake, but guess what… it breaks Bullet’s demos/sample browser. A fix is available but requires updating the make files manually :frowning:

PhysX integration so far has been going smoother, although I had to solve CRT conflicts too. I went as far as being able to initialize the physics world. PhysX is multithreaded which has some consequences on the design of the module, which is the main reason I’m interested in updating our physics interface to be future-proof.

24 Likes
32

Wednesday, 1st June 2016

Feel like forever since I wrote the last update :slightly_smiling:

Physics refactoring is almost done. I’ve been able to clean up a lot of design issues with our physics interface and to make everything compatible between ODE / PhysX / Bullet. In terms of implementation, ODE is back to working again, however I haven’t fully tested all the features, so there might be some surprises when I re-test the Battlescape client later on. Bullet only utilizes a bare framework ( initializing / shutting down the lib ) and no functionality. PhysX is in a better position, as not only it is initializing and shutting down the lib, but it also has support for rigibid bodies dynamics ( I’ve been mostly testing with boxes, adding new types of bodies should be straightforward later on ). I’m much more confident in our physics modules now.

The linking CRT conflicts I mentionned in my last update have been solved for Bullet & PhysX. I also tested PhysX with multiple threads, and it seems to work, but for some reason performance seems quite… unstable. In some frames it’s very fast ( 15 ms ) and others relatively slow ( 60 ms ), and that’s with a constant charge of work ( and stable scenario ) so I’m not sure what’s going on. In comparison, ODE is single-threaded and performs at a constant 34-35 ms; so it seems slower in average, but its performance is a lot more stable. I dunno if that will turn out to be a problem later on.

Today I’ve been fixing a stupid problem with my PhysX implementation. After doing some tests in the past days, I realized something was wrong. Some boxes seemed to fall ( and stay ) partially under my test ground plane, as if some contacts weren’t correctly generated. It turned out it had nothing to do with contacts, and everything to do with… tadaaa… matrices. The orientation matrix had to be transposed ( aka reverting the orientation ), and suddenly everything seemed to match perfectly.

I’ve also started to move all our physics to an asynchronous task ( which will use a separated thread ). This has nothing to do with the single or multi-threaded capability of the physics API; I’m talking purely about moving our own physics interface to a separate thread here. So even ODE or Bullet would end up running in a separate thread, and not be locked to the gameplay thread.

This is a major, if not THE major improvement to our physics implementation, so I was a bit nervous when I started to do the changes, but I got something running in a separate thread within an hour ( even though I still have a lot of work remaining to synchronize all the different threads ), so I’m pretty happy with the progress.

Separating the gameplay / rendering / physics into their own thread is critically important for performance of the engine. So far, everything was running in a single thread. In terms of physics, this could easily lead to what is called the “spiral of death” ( see here for more info: http://gafferongames.com/game-physics/fix-your-timestep/ ), a well known problem for physics programmers. It happens when the time to simulate a time step takes longer than the time step itself. Another problem is when you have a very high framerate, but relatively slow physics, which leads to jerkiness in the framerate. Separating the rendering and the physics solves a lot of these issues, although it’s more complex to implement and debug.

32 Likes
33

Saturday, 18 June 2016

Physics refactoring is now completely finished, including a new asynchronous mechanism for ray casting. So far everything seems quite stable, so I hope we won’t discover weird new bugs once we test in multiplayer in the next patch later on.

Speaking of the next patch, I resumed work on it last week. The bomber got some design updates, but most importantely we’ve started to add support for the new space stations ( atm placeholders like everything else ). The main issue with space stations is one of design variety. We didn’t want to model 2 or 3 stations and copy/paste them dozens of times in the solar system ( Star citizen / Elite Dangerous basically did this ). Instead we decided to go for a modular system. Jan designed and modelled approximately 50 blocks ( “station modules” ), which can get assembled together in various ways. We then created 7 different configs ( to start with; the full game will have dozens ), each resulting in pretty different station designs, although all made from the same basic blocks. I must say, I am very happy with the results, even at this stage. I’m surprised by how different the stations look like (in terms of global details & shape), while technically re-using all these blocks minimizes our workflow and allows us to get away with a lot of content, for a minimal budget. I won’t spoil anything yet, but you’ll see them in the next patch.

I’m not sure yet when that next patch will be released. There are a couple of out-standing issues to fix before we can make it. Keith has upgraded the launcher to support our new patching system, but unfortunately it’s not functionnal yet. Even if the launcher worked, I still have to revamp the player spawning system ( we need it to spawn in these new stations ), and I’ve also noticed a couple of… interesting… sound bugs. I also have plans to tweak some ship/atmospheric parameters as well as rework the warp systems, although that could get delayed to the next patch. We’ll see how it goes.

28 Likes
34

Thirsday, 21 July 2016

Revisiting keyboard input

Today I made a new pass on our Input system. HOTAS seemed to miss some inputs ( Z axis ), so I added “sliders” from Direct Input ( which are basically 2 extra axes ) which will hopefully address the problem. As I was doing that and running some tests, I realized some other things were broken/bugged. The most mind-boggling thing was Alt-Gr generating a “left-control” key pressed event. After investigating, it seems like it’s not a bug but ( thanks to Microsoft ) a “feature”. Don’t believe me ?

I found a blog which explains exactly why Windows Input is in such a sorry mess. From API bugs ( yup ) to virtual keys mapped to more than one key ( yup ) to weird behaviors, if you’re a little into programming you’re going to be amazed:

As advised by this article I switched from Windows messaging to the Raw Input API and these problems are now solved. As a bonus, keypad enter is now differenciated from the normal return key, and numlock no longer affects the key code.

Deployment status

Our new deployment platform is now pretty much done. Keith is finishing the last touches ( and fixing the last bugs ). One of the main delays happened 2 weeks ago when he upgraded his dev environment ( Visual Studio ) to Update 3. This broke our code ( internal compiler error, never a good thing ) and forced us to switch from DLLs to static libs everywhere. The good news is that we had planned to do that for a while, so it was the occasion to do it. It solved various dependency issues and has more potential for optimization. On the downside, compiling ( especially in release mode ) takes a lot longer now which is really annoying when you have to iterate a lot on a program.

Another downside is that it introduced various subtle bugs, like for example the order of static variables initialization. It took a couple days to fix all these new bugs.

Patch status

The new patch is pretty much done now and is scheduled for a public release some time by the end of the week end. It has a bunch of new content ( the new space stations, the new cockpit interiors etc… ) and tweaks ( the atmospheric flight model no longer caps speeds to ~300m/s, as I introduced a drag coefficient per ship ). Also some small performance improvements ( physics now run in a separate thread ) and fixed some shadowing artifacts ( light bleeding line between shadow cascades ).

As a side note, one of these performance improvements required shader model 5.0. I don’t think there’s any non-sm-5.0 video card that could run the game at a decent framerate anyway, but in case, if you tried to run the game on a super old card and it no longer works, you’ll know why…

23 Likes