JON ALLIGOOD'S WEBSITE

---

ABOUT CONTACT

One of the selling points of Unreal Engine is the built in capacity for networked multiplayer games. When a developer using Unreal Engine clicks "Replicates" on an actor, how do we end up with a packet containing the actor's data? We're going to get to the bottom of this question!

Note, this is not an overview of networking in Unreal Engine or how to make a multiplayer game using Unreal Engine. For that information, I would start here. I will also note for the sake of brevity a lot of the code presented has been trimmed (because while validation and feature flags are necessary, we're here to learn). And one final note, this covers the generic replication system. Replication Graph and Iris Replication System are not the defaults and it's good to know what came before to understand the context of the new systems.

To start, I am going to give a high level description of the key players. If you are familiar with Unreal Engine, you know these two:

• World: The world everything takes place in
• Level: A world has a collection of levels.
• Actor: An object that exists in the level. A level has an ActorList which contains actors.

But since we are digging into the networking side, we need to go over a few more concepts:

• NetDriver: Manages groups of connections. A multiplayer game will have at least 1 NetDriver. A server will have a NetDriver with a list of NetConnections, each representing a player in the game. A client will have a NetDriver with one NetConnection representing the server. You can think of this as a connection manager that receives packets on the network and passes them to the appropriate NetConnection.
• NetConnection: Manages sets of Channels. Extending the description above, a NetConnection receives a packet from the NetDriver. It then decomposes it into "Bunches" that have a Channel ID. The NetConnection then sends the Bunch to the appropriate Channel.
• Channel: Responsible for sending and receiving the data. The data arrives from the NetConnection and is read. For instance, an RPC to EquipWeapon arrives, the Channel uses the RPC ID to call the EquipWeapon function on the Actor. Fun fact, each actor has a unique channel.
• PackageMap: Tracks Net IDs and maps them to objects and names. Subclasses like UPackageMapClient are on the client on the connection and used to also track acknowledgements.
• Packet: The actual data sent between clients and servers via NetConnections. They contain meta data such as headers and acknowledgements as well as Bunches.
• Bunch: Data sent between Channels. The NetConnection breaks a received Packet down into bunches which are then passed along to their specific Channels.

Technically NetDrivers exist on the WorldContext but this post is long enough as it is

Before we get into how an actor's data is replicated into a packet, we first have to cover how Unreal knows an actor should be replicated. Actors have a boolean bReplicates that is used to determine if the Actor is in the replication pool or not. (The "Replicates" checkbox in the details panel) bReplicates is controlled by AActor::SetReplicates.

If a game is in progress and AActor::SetReplicates is called, the Actor tells the World to AddNetworkActor:

        if (bActorInitialized)
        {
          if (bReplicates)
          {
            if (UWorld* MyWorld = GetWorld())
            {
              if (bNewlyReplicates)
              {
                MyWorld->AddNetworkActor(this);
              }
            }
          }
        }
      

Source/Runtime/Engine/Private/Actor.cpp

Another way an actor is tracked is when the World is instantiated, it calls InitializeActorsForPlay which calls SortActorList for each level. Then SortActorList calls AddNetworkActor:

        for (AActor* Actor : Actors)
        {
          if (IsValid(Actor) && Actor != WorldSettings)
          {
            if (IsNetActor(Actor))
            {
              NewNetActors.Add(Actor);
              if (OwningWorld != nullptr)
              {
                OwningWorld->AddNetworkActor(Actor);
              }
            }
            else
            {
              NewActors.Add(Actor);
            }
          }
        }
      

/Source/Runtime/Engine/Private/Level.cpp

UWorld::AddNetworkActor loops through NetDrivers, and calls NetDriver::AddNetworkActor. NetDriver::AddNetworkActor adds the actor to the NetworkObjectList (a container for managing replicated actors for a NetDriver).

Now we know which actors to replicate. So when does replication happen?

Unsurprisingly, it starts in the tick. Let's consult our handy-dandy flow chart for replicating a new actor!

Okay, that's a lot. Let's break it down.

At the start, it's straight forward. As Unreal Engine ticks, it tunnels down calling tick until we reach the NetDriver's tick:

1. We start in GameEngine::Tick
2. Which calls World::Tick
3. Which calls NetDrivers::Tick
4. NetDriver::Tick calls NetDriver::ServerReplicateActor

Okay, now we are in the domain of replication! Let's walk from replication to serialization.

1. NetDriver::ServerReplicateActor loops through NetConnections
2. For each NetConnection, get a prioritized lists of Actors (for instance an Actor far away might not be relevant for replication)
3. Calls NetDriver::ServerReplicateActors_ProcessPrioritizedActors on the prioritized list
4. NetDriver::ServerReplicateActors_ProcessPrioritizedActors loops through each actor in the list
5. Performs validation (e.g. did the connection close, is this actor being destroyed)
6. Get the Channel for the Actor (each Actor has a unique Actor Channel)
7. Calls Channel->ReplicateActor()
8. A FOutBunch is created (the Bunch data will be written to)
9. FReplicationFlags is created (to track replication flags such as if the actor is new aka send all replication properties)
10. If this is a new actor, UPackageMapClient::SerializeNewActor to the Bunch. If it is an existing actor, only serialize properties that changed with FObjectReplicator::ReplicateProperties. We're going to assume a new actor (because it's easier, existing actors use delta compression to reduce the amount of data sent)

So now we have an actor, we have a Bunch, and we want to write the actor's data to the Bunch!

1. UPackageMapClient::SerializeNewActor calls UPackageMapClient::SerializeObject to write the reference to the object to the Bunch (so Net ID can be associated with an object's name and path)
2. Then it serializes the Location, Rotation, and Scale for the Actor.*
3. Since this is a new actor, only the Location, Rotation, and Scale are serialized for spawning purposes. (The rest of the replicated properties arrive in the next Actor update)
4. Now we have a Bunch with all the serialized data for the Actor!

* Serialization actually supports quantization (also called Fixed Point Compression). The idea is you only support a specific decimal place of precision when writing data (in this case, 1 decimal place). Then if we have the float 10.1, it can be multiplied by 10 and becomes 101, which allows you to bitpack the integer in your Bunch. Then when you read the value you simply divide it by 10. Now a float which previously took 32 bits can now be represented with an integer allowing you to use up to 24 bits while supporting floats in the range of +/- 1,677,721.6 (2^24/10). 12 bit savings add up. For further reading check out FVector_NetQuantize10 in NetSerialization.h.

I kind of yada yada yada'd the serialization side. But the short answer is:

FOutBunch inherits from FBitWriter which inherits from FArchive. (I am skipping over a few parents in the hierarchy)

FArchive defines operator<<() overloads for common types. These functions call a Serialize function which is left to child classes to override:

        virtual void Serialize(void* V, int64 Length) { }
      

(This provides users a common interface so they can use MyArchive << Data whether reading or writing. Boost::serialization takes a similar approach by overriding operator&())

FBitWriter overrides this function:

        void FBitWriter::Serialize( void* Src, int64 LengthBytes )
        {
            // ... trimmed for brevity
          appBitsCpy(Buffer.GetData(), Num, (uint8*)Src, 0, LengthBits);
        }
      

Which copies the the value into the buffer (in a bitpacked fashion).

Back in `UActorChannel::ReplicateActor` we described how a Bunch was written to based on Actor properties. Now we need to get that Bunch into a packet.

1. If any data is written to the Bunch (since an Actor might not have changed since the last update, it might not need to send any data), it calls UChannel::SendBunch which validates if the Bunch needs to be broken up into smaller Bunches and then adds them to the OutgoingBunches list.
2. Then UChannel::SendBunch proceeds to send all bunches at once via UChannel::SendRawBunch, iterating through the list of OutgoingBunches.
3. This calls UNetConnection::SendRawBunch which writes the header (e.g. whether it's reliable, whether it's a partial bunch).
4. Then the Bunch is written to the SendBuffer via WriteBitsToSendBufferInternal
5. Then if the SendBuffer is full or UNetConnection::Tick is called, UNetConnection::FlushNet is called, and any pending data is sent in a packet via LowLevelSend;
6. UIPConnection::LowLevelSend is where we finally send the data in a packet via a UDP Socket:

        bWasSendSuccessful = CurSocket->SendTo(Packet.GetData(), Packet.Num(), Result.BytesSent, *RemoteAddr);
      

We reached it! We have a packet with our actor's data in it! Thanks for joining me on this journey. Maybe later we can dig into Replication Graph and Iris but for now I need to get back to my current project.