Networking an MMO

Battlescape
1

Everyone wants those HUGE single shard MMOs, but does the technology exist to make them viable.

Imho, the one of the biggest bottlenecks is client bandwidth. The server can’t consistently update the position and status of everyone, when the player has a 10Mbit DSL line (or 50Mbit for that matter). The volume of data is just too much for the players home internet connection. The only company that I know of that, MAY have a chance of pulling it off is Novaquark, with “Dual Universe”.

Dual Universe has alluded to how they’re going to attempt this feat. Their server architecture will scale based upon “client density”, AND they will scale their positioning ticks, based upon how close players are to each other.

For those that don’t know, positioning data is sent multiple times a second. The client takes those updates, and attempts to predict the position of the other players in order to account for the time between those updates. If packets are lost, the client continues to predict the most likely path of the other players. Eventually… If enough packets are lost, there is a discrepancy in the prediction and the actual location of the other players. When new packets are received, the good information is corrected by the client, and this is where you get “Rubber Banding”. This is when you see players bounce around the screen.

Dual Universe will conserve bandwidth by updating positioning of people that are close together, and people that are farther away, will have less frequent updates. Honestly, this is a novel concept.

A player doesn’t need information of ALL of the other players in an MMO (or all players a single server is responsible for). Just the information of the players they can interact with. Here’s an example (keep in mind the scale may be off) Say all ships within 10 kilometers should be given priority. 10 to 25 kilometers, fairly frequent updates. 25 to 100 kilometers, once a second updates, and everything after 100km is dropped, because they’re out of sensor range.

Lets look at those players at 25 to 100 kilometers. The client still updates their position, but there will definitely be rubber banding. It won’t be noticeable due to the focal length of the player.

The focal length helps with rubber banding. Here’s what I mean by that. Consider your favorite FPS game. Shooting at a rubber banding player who’s in the same room as you is different than shooting at a rubber banding player that’s 500 yards away. If they jump 10 feet away in a room, they most likely jump out of your sight picture. However, bouncing around 10 feet at 500 yards is hardly noticeable to a sniper. You may need to completely turn around to hit them in the room, but at 500 yards, it’s a minor aiming adjustment.

(Most likely, everything greater than 25km will be a radar blip anyway)

Dual Universe may have stumbled onto something here…

I would also add, there’s no need for a player to see more than 100 of the closest players. Even in a huge space battle with thousands of players packed into a small area. The player just doesn’t need to see EVERYTHING. Just those they’re capable of interacting with.

Of course, with player movement the people that are “in range to receive updates” will change, but the above scenario WOULD allow everyone to interact on a single shard, and your squad/group would be able to meet up anywhere in the galaxy, if they got separated.

1 Like
2

There used to be a game called MAG on PS3. It was an online-only first-person-shooter with up to 256 players on a single map. Yes, most of the time you wouldn’t see all 256 players at the same time, but it was possible to (There used to be a video on youtube where a mag clan had some fun playing zerg rush. Where everyone of the team would rush through the same corridor … Fun times.) So yeah, the technology definitely exists…

3

Doesn’t really sound like something new here. There are a lot of games using such techniques to minimize network load … even the techniques you listed aren’t unheard of and are used in quite a few online games.

Still worthwhile looking at it, as it is, I believe, one of the biggest issues Infinity Battlescape has to solve.

Also Related:

3 Likes
4

Reminds me of a High School corridor between classes. But when was the last time you interacted with someone at the other end of the hallway, while 254 people were standing between you? And what about that 257th player? 256 isn’t “MASSIVELY” in a MMO.

Whether the limit is 100 or 256 or 2000. That’s a balance issue. At a certain point, providing the data to the player just ends up using resources the player will never make use of.

1 Like
5

Is the current practical bottleneck really bandwidth? Most of the limits that games like EVE seem to be hitting are raw per- server performance, with the issue being a lack of capability to load balance networked clients accross multiple servers. Presumably once the host has to do that, things start to scale exponentially with each server cross communicating all positions and relevant data like healthbars, actions.

Even eve doesnt even send healthbars out unless you specifically target, often taking upwards of 10-20 seconds depending on your ship, to actually lock someone. It also doesnt even show players offgrid, and yet it still hits server limits before bandwidth limits.

6

No it’s not. In fact we’re already doing it. Even ICP in 2007 was doing something similar.

It’s pretty easy to implement although as you mentionned it leads to rubberbanding ( which shouldn’t be visible if the ships are far enough ).

More interesting is data compression and sending incremental updates.

6 Likes
7

Thank for the info Flavian.

Since Im not a game programmer, Im not aware of all the tricks you guys use. It would be nice if there were a site that listed “best practices” in MMO programming. :slight_smile:

BTW, I dont know if you took my post as offensive. Being that I brought up Dual Universe. It certainly wasnt intended that way.

4 Likes
8

Our esteemed leader is a tough cookie, I’m sure he’s fine. :wink:

9

So creating the list…

  1. Dynamic scaling of servers
  2. Dynamic scaling of update tics to the client
  3. Data Compression
  4. Dynamic updates

What are some other techniques used to network a large group of players?

10

Nope I wasn’t offended or anything :slight_smile:

Here’s a good reference about best networking practises with tons of resources:

http://gafferongames.com/game-physics/networked-physics/

The part about snapshots compression is pretty much your answer about handling a large group of players:

http://gafferongames.com/networked-physics/snapshot-compression/

5 Likes
11

Yeah was going to say using distance based LOD and networking code doesn’t seem that novel just more like common sense. I always thought the barrier was client CPU/GPU processing. If each ship is somewhere between 20-100k polys, displaying 1000 nearby ships would probably hammer the average pc into oblivion. I think games like EVE can only do it because it is point and click with no real physics or detail.

I remember an old livestream from one of the SC devs playing around in Cry Engine he was spawning Bengal carriers into the editor. I think they were around ~2.5mil polys each at the time. After about the 8th one, Cry Engine crashed with an out of video memory error.

1 Like
12

Whoa!

Gafferongames is great! More stuff like this please :smiley:

Booyaah,

Yeah, you would THINK CIGs graphics artists would have some sort of clue when it comes to the number of polys in an asset.

2 Likes
13

The devs from MAG stated that their tech easily supported 1024 players and beyond, they just chose to stick with 256 players for that game (which wasn’t that bad of an idea. Near the end of that games lifespan it was hard to find enough players even for the 128 player matches). They easily could have gone higher, but never got the chance to do so, due to the closure of the dev…

14

You’re missing my point.

The backend server solution may have the horsepower, but is it necessary?

Let’s pack 1024 people into your local high school hallway. Exactly how many of them will YOU be able to interact with?

Now place those 1024 people in your basic online “city square / merchant area / arena”

Does having ALL 1024 people in an area ADD or SUBTRACT from the gaming experience? How many players do you have to wade through to get to that vendor? (I wonder if an MMO can induce claustrophobia)

Imho, a single shard universe runs the risk of having too many players in a certain area. Why not scale for player experience, and use the extra resources for other aspects of the game

15

INS Kickstarter promise was…

Engage 100’s of players in epic space battles involving everything from small, nimble interceptors up to immense, powerful capital ships.

So at the minimum that needs to be possible, will we be able to have even one server filled with those numbers is a different story. Have a look here, if you have access.

1 Like
16

Well, there’s your reason for having 1000 players in a market: there are likely three or four thieves in there, plus a possible assassin. Which ones are the ones to watch out for? Can I spot them at a distance? Is spotting them amidst the crowd a skill unto itself? Would the thieves and assassins be particularly interesting if there wasn’t a press of humanity in the marketplace? You’d see the 20 players in the market, check them all out, establish that none of them was a thief or assassin and move on. The 1000 players will overwhelm your senses, allowing thieves and assassins to do their thing.

How about the case where I need to work with a few players who are far from me in a crowded area? A scout commander wants to track his scouts, but they’re scattered all over the fleet engagement and the scout commander can only see the closest 100 players.

Ultimately, it depends on the gameplay, and we’re not used to seeing games with 1000 players in the same area. We don’t know how gameplay would unfold. Can you be motivated by the speech of William Wallace before battle if you only hear the bits when he’s standing close by? What about seeing the size of your own army vs the size of the enemy army? “I’m sorry, I can’t even see the enemy army because I’m surrounded by 500 buddies. Well, 463 now, because we never saw the enemy archers who just fired blind based on the report of an enemy scout.”

2 Likes
17

That depends on what you consider an interaction, among other things. Having 1024 people in an area can both, add or subtract to a gaming experience, depending on the game mechanics, personal preferences etc.

I can only agree wholeheartedly with what JB wrote (which, for the record is a rare thing to happen).

18

If you really, really, really want something huge, where you have multiple 100+ space battles in the same star system but at different areas away from each other, then each “instance bubble” is its own separate process from everything else.

Basically you have one process for the whole system, telling you where people are far away, battles that are happening far away, and so on, and this is what the clients are connected to.
Then, when a battle gets large to the point that it’d slow down the rest of the server, you break off and spawn a new process that just runs that battle around an area, dealing with all the clients and things going on in that area. That receives information from the main process that people are connected to, then sends back the tick data to output to everyone.

This is ultimately less efficient, as in more costly, because you are managing a socket between the two processes, which has extra overhead, but it’s much more scalable.

But Flavien knows all these things. It’s a matter of priority, scope, funding, etc.
It’s possible to have hundreds and hundreds of ships battles, that are parts of a war of thousands of ships in a star system spread around a bit. But it’s more work. If this game won’t have the player base for that, there isn’t any point.

19

Just spotted this in the SC reddit. In terms of networking SC, CIG are sponsoring an industry veteran named Glen Fielder who I recognised from that Gaffer on Games site that Flavien linked to.

https://github.com/networkprotocol/libyojimbo/blob/master/README.md

:nerd:

1 Like
20

This video is a very good explanation:

So what you have is a compromise between delay between clients and having a smooth gameplay experience. I urge everyone to watch that video even if you never played Overwatch since it does point at a lot of common techniques used in games to give you the illusion of a smooth gameplay that is fundamentally limited by the laws of physics.

So in massive games what you need to have is a complex combination of several techniques (the balance of which is something the developers have to nail) and, perhaps, try to have a dynamically-scaling per-client interpolation (objects far away update less often, objects close to you update more frequently).

3 Likes