HaxArena Architecture

Date: January 9, 2016 Posted by: Chris St. John In: Uncategorized

We’ve had a number of questions about how HaxArena is being made, so it’s high time I wrote about it.

HaxArena’s front-end is being built in HTML5 and Javascript, using AngularJS as an MVVM framework. The actual game board is being built in Phaser, because of its simplicity and tight integration with physics engines.

The back-end is being built in node.js, a server-side Javascript engine. Node.js works well because, for starters, we will be able to run the exact same physics simulation code on both client- and server-side. It also has tremendous support in the developer community, who routinely release and maintain free-for-use components, such as passport, mongoose, box2d and, most importantly, Socket.IO.

Contrast to HaxBall

HaxBall is built in Flash/Actionscript, and adopts a P2P architecture. In a P2P architecture, every client is running its own simulation of the game, and regularly pushes out updates to ALL peers in the room. This is a good idea in theory: all updates go directly where they’re needed instead of a central server, thereby reducing lag. But in practice, think about 12 simulations all trying to simultaneously keep themselves synchronized, with one simulation (the host) acting as “the boss”. Everybody has been in a game where a single player connects with a laggy connection and ruins the game for everyone. I don’t know the exact specifics of how it works, but now you’re beginning to understand why this happens.

HaxArena uses a central authoritative server. All clients connect to the server and regularly send their input commands there. The server runs the physics simulation and regularly pushes out updates to all connected clients. This means that if a player with a laggy connection enters the room, the game will feel sluggish to him only.

Another difference between the games is that Flash’s P2P technology uses UDP packets sent directly between peers, and this can spell disaster depending on your network topology. In some cases, it’s not even possible to connect because of a firewall, so you need to mess with your router settings at home to open ports, or beg your work or school’s network administrator to reconfigure the network (good luck with that!).

Socket.IO uses WebSockets to establish a connection to a single server only. WebSockets work by upgrading a normal HTTP request to a full-duplex TCP channel; once established, the client and server can communicate normally. TCP is used as a transport layer, so there is the possibility of some congestion due to dropped packets. However, because port 80 is normally used to establish the connection, router settings should never be a problem.

TL;DR: If you can visit websites, you can play HaxArena.

Mitigating lag issues

Let’s face it: the internet is a wonderful combination of switches that means sub-second communication between my machine and yours, but lag is still a reality of our world and we have to do what we can to reduce its effect. In the examples that follow, assume that the server pushes out updates at 10fps (i.e. every 100ms), and assume a network latency of 200ms (this is extreme).

Experienced programmers: this section is for you. Bring on the comments, please!

Naive approach

In the most naive approach, clients send inputs to the server, the server does all the calculations, and the server pushes updates to clients (locations, etc.) This leads to a very sluggish experience, since the effect of moving is not felt for 400ms. Furthermore, even after a player stops holding down the move button, the player will keep moving for 400ms, a “skating” effect. No good.

Client-side simulation

The nice thing about using Javascript on both client and server is that we can run the exact same simulation here as there. So as we hold the move button, we can start moving immediately in the client. Nice! However, server updates are still reaching our client every 100ms, and on the server, we haven’t started moving yet. So we get reset to our initial position once or twice, a “false start” effect. Furthermore, we will still “skate” after we stop holding the move button.

Client-side simulation, buffered inputs (theoretical!)

In this approach, our client sends inputs to the server, as before, but they are numbered sequentially. The client maintains a short buffer of input packages that have been sent. Server updates come equipped with the last of our input sequence numbers that have been observed and processed.

The client first resets the positions and velocities of all objects as specified by the update. It also locally replays all the frames that the server has not yet processed, using the buffer of inputs collected earlier. The inputs used by other players are assumed not to change for this purpose, an example of “dead reckoning”. In this way, the client is predicting the future: what the player sees is actually an approximation of the server’s states a couple of frames from now.

The nice thing about this approach is that the player’s experience should be smooth: he moves immediately when he holds down the key, and when server updates come in, the player does not get teleported back to where he started! Furthermore, when he stops, he will not “skate” forward.

The downside of this approach is that the client’s prediction of the future may not be accurate. Some other players may have taken other actions in the intervening frames that have a conflicting effect on the simulation on the server, and when this conflict then reaches the client, it will replay player inputs on a state that does not accurately reflect what the player saw at the time. This might be … bad.

Client-side interpolation

One of the approaches used to reduce the appearance of lag was including client-side interpolation. In this approach, when an update comes in from the server, the client does not simply teleport the object there. Instead, it schedules a slow slide from its current position to its reported position to reduce the jarring effect of the teleport.

In my experience, this trick does not help. The interpolation gives the impression of velocity that is not actually there, and the resulting effect is not natural especially for the ball: it appears to change direction according to no known physical source. I would prefer for the ball to lag and teleport but stay true to known physical forces, at least this way I can still follow the action. This is simply my opinion.

Current state

The current state of the demo is an approximation of the client-side simulation approach. I say approximation because currently a close-but-not-perfect copy of code is running against Phaser’s box2d plugin on both server and client. My current effort is focused on cleaning this up so that the simulation code run on server and client are exactly identical, and that they run against the box2d engine directly instead of phaser’s plugin. Once this is in place I can begin messing around with the approaches described above.

Summary

I hope this sheds some light on the technical direction of HaxArena. I welcome all comments and feedback, especially from seasoned multiplayer game developers!

7 thoughts on “HaxArena Architecture

  • asd says:

    i agree with the most, except with the stamina bar, i dont like it (if im a good player i want to play all the time, not to be obligated to be changed)

  • sok says:

    “HaxBall is built in Flash/Actionscript, and adopts a P2P architecture. In a P2P architecture, every client is running its own simulation of the game, and regularly pushes out updates to ALL peers in the room. This is a good idea in theory:”

    That’s true but – still – in Haxball one of the players is a game host and the simulation happens on his machine. Other players sends input but the logic happens on on the host.

    “HaxArena’s front-end is being built in HTML5 and Javascript, using AngularJS as an MVVM framework. The actual game board is being built in Phaser, because of its simplicity and tight integration with physics engines.”

    +1 for this. I hope you will not burn out with HTML5 like HaxBall owner did xD ( https://www.reddit.com/r/haxball/comments/3didae/what_if/ct5f9m2 )

    • Aaron Smith says:

      I think that you will find us to be much more motivated than the HaxBall owner 🙂 Not to mention we have the benefit of starting from scratch.

    • drfeli says:

      In Haxball the logic happens on every client. All the host does is receive the input from the clients and send it out to the peers connected to it. Each individual client is responsible for calculating ball position etc.

  • maod (oscar) says:

    ” Does the floating point implementation used by Flash make this possible?”

    Edit: I guess it must, if replays only have inputs and they can be played by anyone.

  • maod (oscar) says:

    I’m by no means a seasoned multiplayer game developer (all I’ve done of the sort was to hack in the Doom Networking module to another DOS game), but I do like to read about the theory and techniques of network models in multiplayer games. So I have a few more questions!

    “TCP”

    I know TagPro uses Websockets, so I guess this will work (at low ping and no packet loss, which is probably going to be the case for most people), but is it the best choice? Glenn Fiedler’s demonstrations show that the physics simulation chokes if there’s the slightest bit of packet loss. Would not WebRTC, which has unreliable datachannels, be a more robust choice? (I know I asked about this in the last blog post.) Does WebRTC also have connectivity issues, as you’ve described for UDP?

    “Client-side simulation”

    “The nice thing about using Javascript on both client and server is that we can run the exact same simulation here as there.”

    What about floating-point consistency? You’ll probably have to send state corrections in addition to movement commands, I imagine, because I doubt that you’d be able to keep synchronized simulations running on different architectures and different javascript engines. I’ve talked to the developer of a web-based RTS that, too, uses Websockets, Node.js, and only sends player input in order to keep the simulation synchronized. He said that he had to exclude players who use Internet Explorer because of desynchronization issues that occur, although both Firefox and Chrome will work (if there are no trigonometric functions).

    Do you happen to know if Haxball operates by only sending player inputs? It appears that it only sends the game state when you join the room, and it’s possible to get desynchronized, so it looks like there is nothing but player input being sent to keep clients in sync? Does the floating point implementation used by Flash make this possible?

    “Client-side simulation, buffered inputs (theoretical!)”

    So, as I understand, if the client has ping p and the server is processing physics tic t, then, at this moment, the client has received input for tic (t – p/2) and is generating input for tic (t + p/2). It is the client’s responsibility to send this input early enough so that it arrives at the server before tic (t + p/2) is processed!

    In order to run the client-side prediction, the client is going to use his own input for tics (t – p/2) through (t + p/2), but, for all remote players, he replicates the input he’s received for tic (t – p/2) for each tic of the predictive step? This would be the dead reckoning you’ve described, which should result in the remote players jumping around like they do in Haxball?

    So then, for the client-side prediction, does this mean that, for each input from the server that you receive, you will be running the physics simulation for (p / dt) predictive steps, where dt is the duration of one physics step? That is to say, if you have r “authoritative” physics updates per second, and you need to run several predictive steps for each authoritative update you receive, you will be running (r * (p / dt)) physics steps per second? If r = 60 and p/dt = 5, this is 300 steps per second?! Will that work?

    • I confess that these are uncharted waters for me, so it should be fun figuring this stuff out. 🙂
      I was thinking of running the server- and client-side simulations at 60Hz, and send correcting “full” states out from the server at a lower rate, say 10Hz. So if the simulations fall slightly out of sync due to architectural differences, it’s not the end of the world. And the client only needs to re-execute a large number of physics steps when those updates come in.

      “Will that work?”

      We’ll find out the hard way. 🙂

      Will that be satisfactory to pro players?

      We’ll find that out too over time.

Leave a Reply

Your email address will not be published. Required fields are marked *

ERROR: si-captcha.php plugin says captcha_library not found.