Weekly Update #80

Best things usually take time.
Great job Inovae team.
You are getting there guys. This looks really promising.
Good thing i got that “newish” GPU.

4 Likes

I love the chaos!! It isn’t full of ships with “swimblatters” It isn’t like ships at sea!! Every ship has it’s own orientation!!

(bad spelling chose the words I picked) :frowning:

4 Likes

Great news! Can’t wait.

3 Likes

Uuuuu seems alpha could be in october for real!
I’m not proggramer but as i heard networking is biggest obstacle in MP games, i’m glad that one went well.

2 Likes

Looking forward to it, but i do expect that number to get lower with projectiles, missiles and all the other game mechanics adding stress.

2 Likes

Waited for too long :grinning:
Great job you guys!

@inovaeflavien, forget about server-side collision detection, it can be added if needed at some later stage, just make it client-side with some P2P sanity checks.

Also I think dev-backers would love to stress test your networking code even without planets.

4 Likes

[quote=“cybercritic, post:27, topic:5914”]
forget about server-side collision detection, it can be added if needed at some later stage, just make it client-side with some P2P sanity checks.[/quote]

Unfortunately that isn’t easily possible. Server is authoritative. Client predicts physics ( and collisions ) but if it gets it wrong, it gets error-corrected. I can do client-side collisions and disable error correction, but then the simulation diverges and you get various networking problems. In other words, making an hybrid system work is as much of a problem :frowning:

2 Likes

Take the data from the client that has the collision and make it authoritative on the server for everyone else. And if you are paranoid about cheating, check on the server or other clients if the data it sends has a too large margin of error.

Well to do that I’d need to have the client send its player ship’s position / orientation etc… to the server and “replace” the server’s local version for each input. Could be a temporary thing… but to be honnest I’m not too hot on doing this kind of tricks. There’s a reason we’ve spent so much time on a proper and safe network architecture.

4 Likes

How much memory and CPU will terrain generation for 400 players take on the server, it’s an R&D issue and might take you a lot longer than a known implementation.

And you would only have to send the client’s collision data, then you can do everything on the server as you are already doing.

I’m glad to hear this. When server-side collision detection was mentioned in update 68 it sounded like it would be purely server-side…

[quote=“INovaeKeith, post:1, topic:5548”]
we had to move all collision detection to the server during a multiplayer game.
[/quote](Emphasis mine)

As @Lomsor pointed out in that thread, server-side collision detection and error correction could be done asynchronously and it could even be horizontally scaled so physics checking needn’t be a limiting factor on player numbers. - Are you pursuing this, @INovaeFlavien, or will it be synchronous?

Horizontal scaling wasn’t discussed in the other thread so I’ll post my thoughts here…
Making server-side collision detection (and other physics checks) asynchronous is good but it obviously can’t be allowed to lag too far behind real-time or the simulation diverges from what it should be, as Flavien said above.
To mitigate this I figure the work could be divided in some way between horizontally scaled workers.

Regarding the technique for dividing the work: Obviously objects can only collide with things that are in proximity to them (whether that be another ship/station or a procedural surface) and therefore the process that checks for collisions between objects A and B doesn’t need to have any knowledge of objects C and D that are far away from A and B.
Additionally, if there were objects in proximity to a procedural surface it would make sense for all collision detection for those objects to be done by one worker where the procedural surface could be kept loaded, rather than the same area of procedural surface being re-generated on multiple workers.

[details=Glossary/Explanation for less technical readers]

  • Syncronous
    If server-side collision detection is done synchronously, that essentially means when a position update is received from a client, the collision detection is done before sending the resulting (corrected if a collision should have occurred) client position to the other clients (and to the original client). Let’s call the time between receiving and sending, the processing time.
    If you consider that detecting collisions (and other physics events) consumes server processor cycles and memory, it follows that if there are lots of physics objects that need to be simulated, that’s going to take more server resource and once there is not enough resource for all this to happen simultaneously, the processing time is going to be longer as the process needs to wait for server resource.

  • Asynchronous
    When a position update is received from a client, the position can be sent to all the clients immediately. Meanwhile, a separate process can do the collision detection simulations and if a collision should have happened, a correction can be sent to all the clients. This means the updates can be sent ASAP without waiting for the simulation to be run.

  • Horizontal Scaling
    In computing, its often more effective to increase your total computing power by increasing the number of computers (horizontal scaling) rather than increasing the power of the individual computers (vertical scaling).
    Additionally, horizontal scaling can usually be more responsive - ie increasing the number of computers when there is high load, and reducing the number when the load is light.

If your workload is asynchronous that makes it a much better candidate for horizontal scaling than if it were synchronous. [/details]

2 Likes

Great update. Nice to hear such enthusiasm from the dev team. :grinning:


As to server side collision detection. I voiced my opinion back on update 68.

The advantage server side collision detection has over sanity checks is that it blocks players from cheating … when they are all by themselves.
Another advantage is that they could calculate visibility on the server and thus lower network traffic.

If I-Novae thinks that’s a big problem … or that implemeting procgen and hitboxes of all physical objects on the server is the best solution. Their choice.

All the client has to transmit for a sanity check of collision detection is that a player near their vicinity has collided with a surface (or structure … or tree …) at Tick Whatever and their new position whatever and vector whatever. Including when they themselves collide.
Depending on the distance to the player the collision will happen at different times/positions due to how the procgen works.

The server then has enough information to determine if the information from the player that has collided is correct. If it diverges too much the player is kicked from the server.


There’s a battle near the surface. People bumb into the surface, trees, the nearby base and eachother every few minutes.
As the server does not know about any of this the client and server physics simulation desynch at that point. The client tells the server though that this was due to a collision and gives the server its new position and vector.
The server trusts the player and applies this information and sends it to the other players.

Other players near the collided player also registered the collision and send it to the server. They disregard all ticks of the server telling the other players that the collided player actually phased trough the planet. A fem miliseconds later they receive a tick from the server that says “yes I heard he collided and adjusted my simulation” from the server. Then the other players client proceeds to trust the server again.

The server later checks if the collision that happened was plausible by comparing the data of the collided player with the data of other players that “seen” the collsion. If they don’t divert too much the player has not cheated. If they diverted and the player boosted his speed, position or did or did not actually collide with anything the player is kicked.


It’s a tradeoff. Believe the clients when it comes to collisions and you don’t need to do all the stuff necesary for it on the server. Hitboxes and the big one, procedural generation.

It diverges from the “all authorative” phylosophy. But only if a collision happens. Like probably not even 0.1% of total playtime.
To hinder people from using this to cheat. Like “Whups I just collided with nothing and was able to break hard this way” or “whups I just phased trough the floor” you use sanity checks.

Clients have to transmit some additional data when collisions happen near them. The client/server relationship also changes a little. Clients may need to trust their data first when it comes to position after collision of other players in order to mitigate “phasing trough and then setting back” as there’s is quite some delay until the server receives the corrected collision data from the player that collided, applies it to its simulation and then sends it to the players.

So both not just trivial to implement.

4 Likes

Is that true? The clients could be the primary source for collision detection based on the information that the server sends. In the normal case, the clients are making a good-faith attempt at collision detection based on what the player sees. At the same time, the server spends whatever resources are available to do its own collision detection, at whatever fidelity level it can manage. So if the server can perform collision detection on 1% of potential collisions, then that’s what it’s verifying. Statistically, it should be able to identify an unreliable client over time.

Combine that with @hrobertson’s comments about horizontal scaling and you find that applying more resources improves the server’s ability to spot unreliable clients. That might be something that you do only when cheaters seem to be present. Flip a switch (either manually or automatically), bring additional resources to bear and run heavy checks for an hour or so, ban cheaters, turn off additional resources.

All the while, the server supports 400 clients.

A variation of that is to perform more complete checks on specific, suspect, clients.

The server spot checks would be stuff like creating the terrain at a place where many ships are operating. Use that patch until it’s no longer useful to keep it around (perhaps a few seconds). Look for another opportunity for good use of terrain patch generation. Just do as much of that sort of thing as is practical, given the available resources. As resources are added, do more.

The more you can trust your clients, the lower-end the server box can be.

What I like about this is the fact that the base scheme assumes reliable clients, and the system scales to address unreliable ones - as opposed to designing the entire system with the assumption that all clients are unreliable.

That’s an interesting thought; multiple clients voting on what they see. It would tend to increase the reliability of client information about collisions. Though I say that without taking the time to work through the problem of which clients have which information at which times. I wonder if some kind of statistical fuzziness would find its way into the algorithms, making ships and weapons larger or smaller because of network latency across multiple disparate clients.

2 Likes

We’re already doing that. It’s a fundamental part of the new network architecture. If the server disagrees with the client ( aka. the client is kilometers away from its server’s position ), the server won’t send snapshots about other actors out of that range. So you run a risk not seeing the same simulation on the client and server.

Another reason to run the physics sim on the server side is for AI ships. And of course checking projectile hits. It’s one thing to keep collisions with surfaces on client side… but trusting the client “most of the time” for who hits who is far more problematic.

6 Likes

I think this is a huge point and the primary unknown that could prolong the networking implementation from weeks to months as you guys have yourself admitted, while ending up requiring high end servers. I think a few of us here are referring to precisely this being done client-side as an unmentioned assumption, I certainly was referring to planet side collisions being the primary thing that should be client-side, or at least assisted by the client. Ship-ship collision detection server-side should only cost a fraction of resources and implementation time as surface collisions.

2 Likes

i do agree :slight_smile:

2 Likes

Noticed in other games actual players take less resources than any kind of AI, with players it should be 800 unit battles…just thinking out loud.

1 Like

I would have thought player driven units would require more resources. Cool, maybe higher than 400 is possible.:+1::grinning:

1 Like

Easy there Tiger… let’s not get carried away :smile:

8 Likes