Bowling Bash: [Literature] NetGames 2002

Summary of selected papers about MMOG architecture from the NetGames 2002 conference.

Aarhus et al., Generalized Two-Tier Relevance Filtering of Computer Game Update Events.

Two-tiered means network communication is limited to a dedicated concentrator layer
TCP
Consistency within the server layer requires to pass the world state between servers.
Clients connect to concentrators based on network topology, not their position in the game world.
The concentrators are constructed to be application independent
Dead-reckoning schemes
Clients connect to a concentrator, not to the server logic. Hence, client connection is never lost if a game server crashes.

distributed content-based publish-subscribe system
subscription language expressive enough to allow game-specific subscriptions (eg x in [10,20] && y <= [40,50], or team == "MyTeam"). In the subscription {x in [10,20] && y >0}, a hub for x is more efficient at relaying an event than a hub for y.
routing mechanism based on a circle of modular software hubs, each hub storing subscriptions.
Evaluation: network topology simulator in which nodes are randomly assigned as hubs [unrealistic]
Metrics:
- Number of publications routed by a node. All nodes end up having the same routing load (ie scalability achieved).
- Number of subscriptions stored by a node. Virtual world is a square, hence the central zone receives more players and subscriptions than peripheral zones.
- Delay for a publication to reach the interested subscribers. Increases linearly with the number of nodes [not scalable!]

Trailing State Synchronization (TSS)
Optimistic algorithm: execute commands as they are received and rollback when late messages are received.
TSS runs a delayed "trailing" copy of the live game state. Trailing copy is able to re-order messages and execute them in chronological order.
It's cheaper to execute a command multiple times than to make snapshots of the game state
Preserving random events was hard

Traffic modeling from logs of a 36 hour LAN. Matches of 8 to 30 players.
Server sends 16kbps to each client
Each client sends 1 kbps to the server
Both client and server receive 20-25 packets/sec.
Packet size varies a lot.
Probability density function of client-server and server-client packet size and latency modeled by Extreme Value distribution: F(x) = exp(-exp(-(x-a)/b)). (long-tail behavior)

World cut in rectangle or hexagon tiles.
Players subscribe to current and tiles adjacent to closest current tile corner.
Each tile contains an environment and an interaction channel.
Environment channel for static data. Static objects generate bandwidth only when someone interacts with them, not by themselves. A static object does not interact with other static objects. Consumes low bandwidth. Uses TCP.
Interaction channel for active objects. Active objects interact with static and active objects.
Tiles are managed by one or more servers (n to n relationship). Server only provides constant data such as terrain and objects within the map. Server only cares about the environment channel. Collision detection on clients. Scales with the number of tiles, not the number of clients.
Authoritative objects are on the machine that instantiates them. Other clients have duplicates/"proxies". Collision detection and other object-object interactions are calculated on clients. Affected object instances publish their updates on the interaction channel of the object's current tile.

Player-to-player interactions require low latency (= high urgency).
In case of congestion, less urgent events may be dropped. Results in a game of lower quality, but still running.
Rare player-to-environment actions require high reliability
Game designers define when they build the game which actions are urgent and which are relevant.
Clients connect to a nearby proxy. Proxies are interconnected.
Participants belong to target groups. Each target group can be contacted through a channel.
Overlay network of proxies distribute events to target group members.

Analyzing network traffic to know how FPS servers work.
Game servers register to server directories. Client gets server list from directory using UDP. Same structure as Napster.
Method: 3 Half-Life US server directories were queried four times a day for a month.
Problem of directories: single points of failure, stale information (70% of servers not active anymore), redundant info (90% of servers register to all directories), and no congestion control

Move intelligence and server functionality to the border of the network
Some server functionality can be delegated to the proxies.
Proxies located close to the players (need ISP support). Therefore, they could execute client code as well (anti-cheat).
Proxies can reduce the server load in doing packet processing and filtering.
Overlay network enables traffic rerouting around congested areas, network fault detection and node-failure recovery.