Saturday, November 17, 2012

How MAMEHub works part 2: How connecting works

Part 2: Initialization and Connecting

Part 2 will cover starting MAMEHub: what happens right when a new player joins or a server is created.  If you haven't yet, read part 1, which discusses what is synced.

Starting a client/server

When someone starts MAMEHub in either server or client mode, here's what happens:

  1. All of the items to be synced (see part 1) are marked as MemoryBlock objects
  2. The MAME initialization code runs.  This sets up MAME, initializes the chips in the machine, loads any initial ROM/RAM, and creates/loads the NVRAM.
  3. At this point the server creates a copy of all items to be synced, storing their initial values.  These are called Initial Blocks and are useful in saving bandwidth when doing the initial sync.
  4. MAMEHub starts a UDP server socket and listens for incoming connections.
  5. Every 30 seconds (if the game supports save states), MAMEHub makes a new copy of all the items to be synced.  These are called the Sync Blocks.  If the game does not support save states, the Initial Blocks and the Sync Blocks are the same.


First off, MAMEHub is a peer-to-peer system.  It's also a fully-connected system, meaning that all peers communicate to each other directly.  There is a "server" which is the first player to join the game, and this is for resolving conflicts among the clients, but from a network standpoint the server is just another player.  There are many reasons for this:

  • If there was a centralized server (also called a client-server model), data would have to go from player A, to the server, to player B.  This (almost always) takes longer than sending data from player A to player B directly.
  • The server would need enough bandwidth to receive everyone's inputs and broadcast them to everyone else.  Most household internet services do not have this upload capacity.
You might ask "But wait, doesn't counter strike and (insert favorite net game here) use a client-server model???"  Yes they do, and there are several reasons why they do:

  • If everyone is connecting to a server, only the server needs to have a port exposed to the outside world.  With a fully-connected system, everyone needs a port exposed.  Getting every player to have an open port is difficult.  Most people running game servers are tech savvy enough to open a port, and most players are not.
  • Because the game designers are making the game, they know what things in the game change (like the player's position) and what things do not change (the artwork).  This means the game state (the things that can change in a game) are really tiny.
A Client-Server architecture (Most games)

A fully-connected architecture (MAMEHub)

When the game state is tiny, a client-server architecture is nice because the server can just blast the clients with an entire copy of the game state every second, and this quickly resolve desyncs (desyncs will be explained later).  In the case of a game on MAMEHub, there is no good way to know what chunks of data are going to change and what aren't, so this benefit cannot be exploited.  The game state in something like counter-strike might be 4 KB, but in MAMEHub it could be 128 MB or even larger.

With that out of the way, here's what happens when a new client joins a server (i.e. right after the client has finished the earlier 5 steps):
  1. CLIENT: The client tries to connect to the server's IP address.  If this fails, an error code is returned and the client exits.
  2. SERVER:  The server sends the client's IP address to any other clients that are already connected.
  3. OTHER CLIENTS: The other clients try to connect to the new client (called the candidate).  The client then reports back to the server whether it was able to connect to the candidate or not.
  4. SERVER: The server gathers reports from the other clients.  If any of the clients can't connect to the candidate, the candidate is rejected and the client fails and exits.
  5. SERVER: If the client is accepted, a player is assigned to the client.  If the client is rejected, the server tells the other clients to disconnect from the candidate client and not to wait for inputs from them.
Alright, now that all the hand shaking is done and we are sure this client will be in the game, the server has to get the client up to speed.
  1. SERVER: The server calculates the XOR between the Initial Blocks and the most recent Sync Blocks.  The way XOR works, if the data is the same, the XOR will be zero.  It will only be non-zero where the data has changed.  Because most of the data does not change, there will be a lot of zeroes.
  2. SERVER: The server then LZMA-compresses the result and sends it to the client.  Because there are so many zeroes, even 128MB of data will typically compress down to less than 1 MB.  Of course, the longer the game has been running, the more data has changed and the larger this will be.
  3. CLIENT: The client uses the XOR data and the client's Initial Blocks to recreate the state of the game at the time of the last sync.
  4. CLIENT: The client then has to "catch up" from the time of the last sync to the present.  All of the other clients and the server wait for the client to catch up, and then everyone starts broadcasting inputs (broadcasting inputs will be covered later)
In the next post I'll discuss how inputs are broadcast and received.


Jorge Rosa said...

Beautifully explained, DG! :)
And by the way, I think that you have already enought quality articles and software that deserves to have a mention about MAMEHub, in the Wiki, not? ;)

Alessandro Cristallo said...

Very interesting, thanks to share all of this.
And...another blog thread for the future of this want to start it ?
ciao !