Replication De-Obfuscation - Revision history

Plasma at 12:55, 19 May 2024

2024-05-19T12:55:55Z

← Older revision		Revision as of 12:55, 19 May 2024
Line 1:		Line 1:
	Written by Mongo: mongo@planetunreal.com		Written by Mongo: mongo@planetunreal.com


	= Replication De-Obfuscation Introduction =		= Replication De-Obfuscation Introduction =

Plasma at 12:55, 19 May 2024

2024-05-19T12:55:41Z

← Older revision		Revision as of 12:55, 19 May 2024
Line 1:		Line 1:
	== Replication De-Obfuscation Introduction ==		Written by Mongo: mongo@planetunreal.com

			= Replication De-Obfuscation Introduction =

	Ask any halfway decent UnrealScript programmer what the most confusing aspect of Unreal coding, and he'll be sure to tell you replication. (or perhaps PlayerMovement or pawn/bot AI :) I've been there myself, and I still have trouble with it sometimes, but I think I know enough to write some documentation. Lots of trial and error helped me get far, along with various insights from people that had the source code to the engine. I've tried to piece it all together to create this document. I have however, recently gotten a lot more information about exactly how replication works, which has helped me immensely in understanding replication. Since I can't post that information directly because of NDA, I'll try my best to explain it to you through this networking documentation. Hopefully I'll be able to expand upon what was described in Epic's documents at: http://unreal.epicgames.com/Network.htm and http://unreal.epicgames.com/UTMods.html. Please read it first, as it will help a lot with background and context. In writing this document, I'm going to assume you've programmed in UnrealScript before, you have at the very least considered the problems of replication, you are reasonably intelligent, you know how to read, and you think my jokes are funny.		Ask any halfway decent UnrealScript programmer what the most confusing aspect of Unreal coding, and he'll be sure to tell you replication. (or perhaps PlayerMovement or pawn/bot AI :) I've been there myself, and I still have trouble with it sometimes, but I think I know enough to write some documentation. Lots of trial and error helped me get far, along with various insights from people that had the source code to the engine. I've tried to piece it all together to create this document. I have however, recently gotten a lot more information about exactly how replication works, which has helped me immensely in understanding replication. Since I can't post that information directly because of NDA, I'll try my best to explain it to you through this networking documentation. Hopefully I'll be able to expand upon what was described in Epic's documents at: http://unreal.epicgames.com/Network.htm and http://unreal.epicgames.com/UTMods.html. Please read it first, as it will help a lot with background and context. In writing this document, I'm going to assume you've programmed in UnrealScript before, you have at the very least considered the problems of replication, you are reasonably intelligent, you know how to read, and you think my jokes are funny.

Plasma at 12:49, 19 May 2024

2024-05-19T12:49:20Z

Show changes

Plasma at 12:38, 19 May 2024

2024-05-19T12:38:37Z

Show changes

Plasma: /* Role and RemoteRole Summary */

2024-05-19T12:32:37Z

Role and RemoteRole Summary

← Older revision		Revision as of 12:32, 19 May 2024
Line 349:		Line 349:
	Hopefully this intricate look into some of the more complex and confusing variables has helped you to grasp the concepts of replication some more, and realize many of the differences between the various roles. However, with the above knowledge in mind, I'd like to do a little re-cap, emphasizing the use of dumb, simulated, and autonomous proxies, and help give you a clearer idea of when each should be used.		Hopefully this intricate look into some of the more complex and confusing variables has helped you to grasp the concepts of replication some more, and realize many of the differences between the various roles. However, with the above knowledge in mind, I'd like to do a little re-cap, emphasizing the use of dumb, simulated, and autonomous proxies, and help give you a clearer idea of when each should be used.

	ROLE_Authority - Again, this is the Role for all actors on the server, and is the RemoteRole for all actors on the client. It basically means that I'm the authority on this actor. This is the only one variable that Role is set to, server-side. The remaining ones are all used for the RemoteRole when on the server.
			<code>ROLE_Authority</code>

			<pre>
			Again, this is the Role for all actors on the server, and is the RemoteRole for all actors on the client. It basically means that I'm the authority on this actor. This is the only one variable that Role is set to, server-side. The remaining ones are all used for the RemoteRole when on the server.
			</pre>

Plasma at 12:31, 19 May 2024

2024-05-19T12:31:24Z

← Older revision		Revision as of 12:31, 19 May 2024
Line 346:		Line 346:

	= Role and RemoteRole Summary =		= Role and RemoteRole Summary =

			Hopefully this intricate look into some of the more complex and confusing variables has helped you to grasp the concepts of replication some more, and realize many of the differences between the various roles. However, with the above knowledge in mind, I'd like to do a little re-cap, emphasizing the use of dumb, simulated, and autonomous proxies, and help give you a clearer idea of when each should be used.

			ROLE_Authority - Again, this is the Role for all actors on the server, and is the RemoteRole for all actors on the client. It basically means that I'm the authority on this actor. This is the only one variable that Role is set to, server-side. The remaining ones are all used for the RemoteRole when on the server.


			<code>ROLE_None</code>

			<pre>
			This basically says: Never make this actor relevant. This prevents replication conditions from even being run, and so does not factor into any of the replication statements.
			</pre>


			<code>ROLE_DumbProxy</code>

			<pre>
			This role is used for simple actors that require no real client-side prediction. They don't need any Physics prediction, but they only need to be relevant to the client to get their variables replicated. This is used for all Inventory actors when they reside on the ground, which prevents any higher-bandwidth SimulatedProxy stuff, but yet allows you to see all the stuff you need for the inventory actor. If you do try to move this actor, the client will get periodic updates, usually around every half-second or so on a regular connection. This creates quite a jumpy effect if you are using DumbProxy with a particular Physics type, since it really isn't intended for that. Rather, the Location updates are useful for when moving the actor across the level with a flag respawn, etc. The same is done with rotation updates, with periodic changes coming in to the client every so often. This role also causes animation updates to be sent from the server to the client, unless the client specifically tells it not to. In summary, an actor with this role is force-fed data from the server. It's intended for actor's that don't move (except in jumps like transportation, respawn, etc), yet are still relevant.

			Other quirks: There's a few quirks that can have a significant effect on your netplay development. DumbProxy actors do not have a variety of events run clientside. Simulated Tick(), Timer(), and state code, along with the physics calculations, are not performed on the client of a DumbProxy actor. They are supposed to be a dumb proxy actor that gets data sent over the net, and so these events are not called. One exception to the above rule is that physics are performed clientside for DumbProxy actors if the actor has a falling physics, which allows tossed weapons to fall correctly in netplay. In an early attempt at the weapon in the tutorial at the bottom of this document, I attempted to create a client-side physics that would combine with the location and rotation updates that were inherently sent via the DumbProxy role, but the client would not run my Tick()-based physics calculations, and so that idea had to be thrown out the window. :)
			</pre>


			<code>ROLE_AutonomousProxy</code>

			<pre>
			This is used for things that the client directly controls. In most cases, this will only be used by the playerpawn. This RemoteRole value should only be set for the one particular player, and it should be a SimulatedProxy for every other client in the game. The player for which it is AutonomousProxy is the player possessing that particular PlayerPawn. When it is set to AutonomousProxy, it appears to be a SimulatedProxy for everyone BUT the owner of the proxy or the instigator. This ensures that your PlayerPawn looks like a SimulatedProxy to everyone but yourself. Note that this technique is also used with the guided redeemer, to ensure that it works the same way. Having the server tell you where your guided redeemer is flying would not be appropriate. Rather, you want to tell the server where your guided redeemer is. While AutonomousProxy doesn't make this just 'work' for you, it does differentiate you from the other people viewing your redeemer, which is what is needed to have it act differently for you, compared to everyone else.
			</pre>


			<code>ROLE_SimulatedProxy</code>

			<pre>
			There are two different things that can happen with this particular type. It depends on whether the actor with this role is a pawn or not a pawn. I'll discuss the latter first.
			</pre>


			<code>Non-Pawn ROLE_SimulatedProxy</code>

			<pre>
			This one is basically the opposite of DumbProxy. It expects full client-side prediction of the actor, given initial values. It's like extrapolating out a curve given some data, where the current Physics is the curve fit, and you have initial data available to extrapolate from. ROLE_SimulatedProxy gives you an initial Location, Rotation, and Physics. It then keeps you updated with the Velocities, should they change. These variables should enable the client to perform full client-side prediction on this object, assuming it continues along the physics prediction. In the case of PHYS_Falling and PHYS_Projectile, this is the case. There can be no aberration form these behaviors, (unless you play around with the Physics variables bBounce and such, which are very useful in prediction.) Perhaps I'll explain Physics in more detail in another tutorial. Perhaps. ;) Finally, for this role, you also get animation updates, assuming the client is not the owner and it's not set for client-side animation. In summary, this Role is for actors that smoothly predict their data on the client.
			</pre>


			<code>Pawn ROLE_SimulatedProxy</code>

			<pre>
			When ROLE_SimulatedProxy is combined with Pawns, a different sort of simulated proxy is created, that act differently in netplay. Pawns are one of the most complex actors in terms of replication, because of their need for clientside prediction, yet inherent unpredictability in their movement. There is no way to duplicate this behavior except by subclassing pawn, so don't expect to achieve it in your projectile subclass. A SimulatedPawn gets the best of dumbproxy and simulatedproxy. It gets velocity updates for the clientside prediction, while at the same time getting location updates from the server to 'readjust' the player's locations. You might then wonder why the pawn does not jump when a new location update comes in. Again, there is native code that causes the actor to smoothly head towards the location given by the server if the location position is too far off from the player's real position. This again ensures seamless movement of player's while keeping them accurate to the server's version. All non-players will get by with velocity-only client-side prediction. No physics will be taken into account for these pawns. This is because some non-players can be PHYS_spider, or a variety of other physics that don't work with the following always-walking/falling prediction. Only players (bots and playerpawns) get the special client-side prediction to have falling/walking physics. This physics is an assumed physics, which is accomplished by applying the zone's gravity if the player is not on the ground. This logic will be bypassed if the pawn's bCanFly variable is set, which frees it of the clientside physics, but must be accounted for in your code, since there is no physics replication in pawns. This logic is also bypassed if the pawn is in a water zone, which also differs from the default walking/falling physics. In water zones, actors are predicted using velocity alone, which is sufficient with the low amount of gravity that exists in those zones. The actor's get their locations 'adjusted' anyway, so it doesn't make any real difference in the long run. Finally, pawns get rotation updates. Since rotation (which is directly based off the ViewRotation, or the direction the player is looking) can change quite radically, no prediction is done with it. Instead, the player's rotation is adjusted as new player rotation information comes in from the client. Animation is handled the same as any other non-pawn simulated proxy.
			</pre>

			= Function Replication, and Simulation =

Plasma at 12:27, 19 May 2024

2024-05-19T12:27:39Z

Show changes

Plasma at 12:21, 19 May 2024

2024-05-19T12:21:13Z

@@ Line 57: / Line 57: @@
 If its Role is ROLE_None, then does not pass the first stage.
 If it is bAlwaysRelevant, and time since the last relevancy is less than 1 / NetUpdateFrequency (described farther below) seconds, then it does not pass the first stage. This basically means an actor will only update it's variables to the client NetUpdateFrequency times per second.(NetUpdateFrequency values: Actor: 100, ZoneInfo: 4, GameReplicationInfo: 4, PlayerReplicationInfo: 2, or Inventory: 8).
 If it is bNetOptional, and it has a non-zero LifeSpan (meaning it does not exist forever), and it is not in the first 150 ms of its lifetime, then it passes the first stage. Any attempt at replicating a bNetOptional actor after the first 150 ms, (because it was pushed off the replication list due to bandwidth contraints for that client, etc,) will fail.
 If the time since the last relevancy is greater than 1 / ~NetUpdateFrequency seconds, it passes the first stage. (This is a more finely tuned optimization of NetUpdateFrequency, tuned to the type of actor. It will only perform the calculations needed if it has passed through the weeding techniques above.)
 It does not pass the first stage.
 </pre>
@@ Line 213: / Line 216: @@
 = Writing Replication Statements =

Plasma at 12:18, 19 May 2024

2024-05-19T12:18:38Z

@@ Line 89: / Line 89: @@
 <pre>
-Bots:            3.0
+Bots:             3.0
-PlayerPawns:     3.0
+PlayerPawns:      3.0
-GuidedRedeemer:  3.0
+GuidedRedeemer:   3.0
-Movers:          2.7
+Movers:           2.7
-Projectiles:     2.5
+Projectiles:      2.5
-Carcasses:       2.5
+Carcasses:        2.5
-Other Pawns:     2.0 (Spectators, Titans, Bunny Rabbits, etc)
+Other Pawns:      2.0 (Spectators, Titans, Bunny Rabbits, etc)
-Effects:         2.0 (sparks, blood, smoke)
+Effects:          2.0 (sparks, blood, smoke)
-Simpler effects: 1.4 (shell casings)
+Simpler effects:  1.4 (shell casings)
-Inventory:       1.4 or 2.5
+Inventory:        1.4 or 2.5
-Everything else: 1.0
+Everything else:  1.0
 </pre>
@@ Line 162: / Line 162: @@
 When writing replication conditionals, there are a few things you should be aware of. During replication, a few helpful variables are set for your use in evaluating whether it should be replicated. These variables are defined in actor, and are:

Plasma at 12:14, 19 May 2024

2024-05-19T12:14:19Z

← Older revision		Revision as of 12:14, 19 May 2024
Line 51:		Line 51:

	<pre>		<pre>
	If it is bNetTemporary, then it does not pass the first stage.		If it is bNetTemporary, then it does not pass the first stage.
	If it is not bStatic and not bNoDelete, then it passes the first stage.		If it is not bStatic and not bNoDelete, then it passes the first stage.
	If it is a LevelInfo, it passes the first stage.		If it is a LevelInfo, it passes the first stage.
	If it is a temporary network actor, (as defined by bNetTemporary), and it's been sent to this client already, then it does not pass the first stage. These types of actors will be decribed in more detail later.		If it is a temporary network actor, (as defined by bNetTemporary), and it's been sent to this client already, then it does not pass the first stage. These types of actors will be decribed in more detail later.
	If its Role is ROLE_None, then does not pass the first stage.		If its Role is ROLE_None, then does not pass the first stage.
	If it is bAlwaysRelevant, and time since the last relevancy is less than 1 / NetUpdateFrequency (described farther below) seconds, then it does not pass the first stage. This basically means an actor will only update it's variables to the client NetUpdateFrequency times per second.(NetUpdateFrequency values: Actor: 100, ZoneInfo: 4, GameReplicationInfo: 4, PlayerReplicationInfo: 2, or Inventory: 8).		If it is bAlwaysRelevant, and time since the last relevancy is less than 1 / NetUpdateFrequency (described farther below) seconds, then it does not pass the first stage. This basically means an actor will only update it's variables to the client NetUpdateFrequency times per second.(NetUpdateFrequency values: Actor: 100, ZoneInfo: 4, GameReplicationInfo: 4, PlayerReplicationInfo: 2, or Inventory: 8).
	If it is bNetOptional, and it has a non-zero LifeSpan (meaning it does not exist forever), and it is not in the first 150 ms of its lifetime, then it passes the first stage. Any attempt at replicating a bNetOptional actor after the first 150 ms, (because it was pushed off the replication list due to bandwidth contraints for that client, etc,) will fail.		If it is bNetOptional, and it has a non-zero LifeSpan (meaning it does not exist forever), and it is not in the first 150 ms of its lifetime, then it passes the first stage. Any attempt at replicating a bNetOptional actor after the first 150 ms, (because it was pushed off the replication list due to bandwidth contraints for that client, etc,) will fail.
	If the time since the last relevancy is greater than 1 / ~NetUpdateFrequency seconds, it passes the first stage. (This is a more finely tuned optimization of NetUpdateFrequency, tuned to the type of actor. It will only perform the calculations needed if it has passed through the weeding techniques above.)		If the time since the last relevancy is greater than 1 / ~NetUpdateFrequency seconds, it passes the first stage. (This is a more finely tuned optimization of NetUpdateFrequency, tuned to the type of actor. It will only perform the calculations needed if it has passed through the weeding techniques above.)
	It does not pass the first stage.		It does not pass the first stage.
	</pre>		</pre>

Line 65:		Line 65:

	<pre>		<pre>
	If it is bAlwaysRelevant, then it is relevant.		If it is bAlwaysRelevant, then it is relevant.
	If it is owned by the client or owned by the actor we are viewing through (as returned by PlayerCalcView), then it is relevant.		If it is owned by the client or owned by the actor we are viewing through (as returned by PlayerCalcView), then it is relevant.
	If it is the client or the actor we are viewing through (as returned by PlayerCalcView), then it is relevant.		If it is the client or the actor we are viewing through (as returned by PlayerCalcView), then it is relevant.
	If its Instigator is the client, then it is relevant.		If its Instigator is the client, then it is relevant.
	If it has an ambient sound and is within 'audible distance', then it is relevant.		If it has an ambient sound and is within 'audible distance', then it is relevant.
	If it is a weapon that is owned by a visible Pawn owner, then it is relevant.		If it is a weapon that is owned by a visible Pawn owner, then it is relevant.
	If it is bHidden or and not bBlockPlayers and the AmbientSound is none, then the actor is not relevant.		If it is bHidden or and not bBlockPlayers and the AmbientSound is none, then the actor is not relevant.
	If it is not relevant (per the above tests), but if any of the above conditions made it relevant within the last RelevantTimeout seconds, (by default, five seconds, but is defined in UnrealTournament.ini:[IpDrv.TcpNetDriver],) then it is relevant.		If it is not relevant (per the above tests), but if any of the above conditions made it relevant within the last RelevantTimeout seconds, (by default, five seconds, but is defined in UnrealTournament.ini:[IpDrv.TcpNetDriver],) then it is relevant.
	It is not relevant.		It is not relevant.
	</pre>		</pre>

Line 113:		Line 113:


	ROLE_None		<code>ROLE_None</code>

	<pre>		<pre>
Line 120:		Line 120:


	ROLE_DumbProxy		<code>ROLE_DumbProxy</code>

	<pre>		<pre>
Line 127:		Line 127:


	ROLE_SimulatedProxy		<code>ROLE_SimulatedProxy</code>

	<pre>		<pre>
Line 134:		Line 134:


	ROLE_AutonomousProxy		<code>ROLE_AutonomousProxy</code>

	<pre>		<pre>
Line 141:		Line 141:


	ROLE_Authority		<code>ROLE_Authority</code>

	<pre>		<pre>

	This role is different from the others in that it is always the Role on the server. On the server, all Roles will be ROLE_Authority, and the RemoteRole will be one of the above. On the client, the reverse will be true. This role doesn't have any real significance, beyond it's use in replication statements, described below.		This role is different from the others in that it is always the Role on the server. On the server, all Roles will be ROLE_Authority, and the RemoteRole will be one of the above. On the client, the reverse will be true. This role doesn't have any real significance, beyond it's use in replication statements, described below.
			</pre>

	= General Replication =		= General Replication =

			For each actor that is deemed to be relevant to a particular client, the server needs to determine what subset of its variables will be sent to the client. The most obvious rule is to only send variables if they change. Unfortunately, that rule alone is not enough to fit data over a modem. Instead, the server has to further whittle down its list of variables, and it does this through replication statements. These replication statements are executed for each client, for each relevant actor to that client. When a subset of the variables of a subset of the actors are sent over the network, only then can it be fit on a regular modem. But how can a client operate without all of the information in the game? Once you stop and think about it, it's not quite that unthinkable. Do all of a Bot's AI variables need to be replicated to the client? Only the result of the AI needs to be sent, which basically consists of the same thing for any other Pawn in the game. Also, depending upon the RemoteRole, different things are replicated. Some cause the Location to be updated periodically as a poor man's simulation of its motion, while another role causes the physics to be replicated, and the location not to be, creating true client-side simulation of the actor's motion.

			= No Recursive Relevancy =

			Please note one important point that can easily be overlooked. Relevancy is not recursive with replicated references to variables. Let's say that Actor A contains a reference to Actor B in Property C, and Actor A replicates Property C (currently set to Actor B). Let's also say that Actor A is relevant, and Actor B is not. (Don't worry, there's only three letters in this example.) In this example, Actor A is relevant, and so it's Property C variable will be replicated to the client. Now what does the client know? The client knows about Actor B through Actor A's reference, but Actor B is not relevant. All of Actor B's variables are not checked for replication. The client only knows about Actor B, which really is of no use whatsoever, since the variables are not replicated. I initially thought that replicated variables to actors caused those actors to be relevant and parsed themselves for replicated variables, but that view is false. An actor can only be relevant according to the rules stated above, which make no provision for a reliable reference to them from a relevant actor.

			= Replication Checks =

			When you think about the number of clients in a server, and the number of relevant actors for each client, and the number of replication checks on variables for each actor, there's a large number of such checks. Special care should be taken when implementing replication statements. As each conditional is evaluated, you should be considerate of the server's limited processing ability. Don't call functions from within replication statements, as function calls are much too slow for replication statements. Along the same lines, you should try to keep your checks as simple as possible, only evaluating a condition if its necessary. Finally, note that Epic had a problem with replication checks taking too long for some important actors like Actor or Inventory, both of which replicate many variables. To alleviate the situation on servers, Epic moved the checks to native code. That's why you will see the 'nativereplication' on a few of the Engine classes. Any variables being evaluated in those classes will be covered by the nativereplication code, making any UnrealScript replication statements useless in that class, (except as a form of documentation for UnrealScript programmers.) Any variables that are not defined in a nativereplication class, (defined in a non-nativereplication subclass or superclass,) will be replicated via the unrealscript definitions. Despite the fact that the nativereplication checks are in C++, the UnrealScript checks that remain in those classes are still accurate, and a good guide to go by in determining problems with replication, and ways around it.

			= Special Replication Variables =

			When writing replication conditionals, there are a few things you should be aware of. During replication, a few helpful variables are set for your use in evaluating whether it should be replicated. These variables are defined in actor, and are: