DOTS NetCode: ghosts & prediction · Unity Workflow Library

Scaling replication lives in the netcode scale page (split at v306): relevancy as an anti-cheat boundary, importance scaling, static optimization, snapshot budget and throttles, and the bounded queues that stop a build/spawn spike stealing snapshot budget from combat. This file keeps ghost/prediction mechanics and their failure modes.

Server/client worlds, ghost replication as the event bus, RPCs, commands and prediction. The recurring failure: it works on the listen host and is broken for every real client.

Never mutate prefabs/assets while Play mode runs. NetCode bakes prefabId + serializers at Play start; mid-Play edits diverge live entities from disk. Exit Play before any prefab/asset write.
Ghost baking strips MBs that lack bakers. SystemAPI.ManagedAPI.UnityEngineComponent<T> silently returns zero on a ghost entity — drive visuals through an aspect/companion system, not a queried MonoBehaviour.
Client input drives presentation INSTANTLY via local prediction — never gate input feedback on a server round-trip. When a client action changes how something looks/feels (draw weapon, toggle a stance, equip, fire FX), apply the predicted result CLIENT-SIDE the same frame and reconcile from the authoritative ghost when it lands (drop the prediction on confirm, revert on timeout). The server stays authoritative and the wire path is unchanged — the client just doesn't WAIT for it. Round-trip-gated input (set flag → server applies → ghost echoes back → only THEN the visual reacts) feels laggy over any real RTT/Relay; it's a design smell from day one. Bake snappiness in from the start — it's far cheaper than retrofitting "feel" later. Predict ONLY presentation/view state (a client-local flag, a UI cache), NEVER write ECS/ghost state on the client. Same shape as NetCode predicted movement, applied to any input-driven feedback.
In NetCode, place net systems by NetCode's GROUPS, never by a raw attribute against a system in another group or world. Input → GhostInputSystemGroup; predicted simulation → PredictedSimulationSystemGroup (which re-runs MANY times per frame on rollback — keep one-shot / structural / RPC work OUT of it); ghost spawn + RPC handling relative to GhostSimulationSystemGroup. A server system that [UpdateAfter]s a client-only system is a pure no-op (separate world). The within-group sibling trap (the ecs core page) still applies inside each NetCode group.
UPDATEBEFORE-PREDICTED-GROUP-IS-IGNORED (0.3.83): [UpdateBefore(typeof(PredictedSimulationSystemGroup))] on a plain SimulationSystemGroup system does nothing. NetCode declares the predicted group OrderFirst, and the sorter drops a non-OrderFirst sibling's ordering against it with only a console warning ("Ignoring invalid [UpdateBefore] ... because OrderFirst/OrderLast has higher precedence"), so the system runs AFTER prediction while its attribute says before. To really run first, add OrderFirst = true to its UpdateInGroup; if a one-tick delay is harmless, delete the attribute and write the delay into the system's ORDER note. Read the console on the first Play after adding any ordering attribute; compiling proves nothing here.
The ghost snapshot IS the event bus — the moment ANY client-side consumer reads a piece of server state, that component's ghost attributes are the event contract. Every replication hole has the same shape: server mutates state, a client reader (HUD, minimap, presentation) reads the same TYPE, but nothing was published — the client silently sees defaults/emptiness and the feature "works" on the listen host while broken for every real client. Three sub-traps: (1) component/ buffer PRESENCE never replicates — a server-side runtime AddComponent/AddBuffer is invisible on client ghost instances, so HasComponent<T> gates read false forever (put the component on the ghost prefab BEFORE ConvertToGhostPrefab, zeroed, and make the server WRITE unconditionally — a !HasComponent guard around the stamping silently skips it once the prefab carries the type); (2) an unattributed field in a [GhostComponent] stays at the client's baked default with no error; (3) a replicated IBufferElementData must [GhostField] EVERY field — so a buffer with a per-frame-ticking timer can't ghost as-is; split a thin ghosted mirror (id + count) from the server-only row, or replicate a ticking SecondsRemaining float the server drains (fine at UI cadence). Prefer deriving client state from already-ghosted ids + local catalog over adding fields (hero-ness from template id, not a HeroTag presence check). Audit recipe: list every type client-side code reads (grep GetComponentData<T> / HasComponent<T> / GetBuffer<T> across UI/ + Presentation/) and diff against the [GhostComponent] set — every server-written hit outside it is a live bug. (a mobile strategy game: one sweep found five — OwnedUpgrade research state, UnitAttack console stats, HeroTag presence, hero summon cooldowns, shop stock — all "worked" in single-process testing.)
Unghosting a field defaults the client, it does not keep the last server value (HARD). Removing [GhostField] leaves the baked default. A quaternion default is (0,0,0,0), so any IsValid that requires unit length stays false and spawn/join can sit until timeout while presentation already reports ready. Before unghosting: grep every client read of that field and of any IsValid/Has* that includes it; Play the join path in the same slice (developer-play-pending is not a skip). Record which fields moved so the slice can be reverted without dumping the branch. Adding a [GhostComponent] still requires the type on the prefab before ConvertToGhostPrefab.
A baked subscene singleton + a runtime-published copy of the same singleton = a duplicate that streams in LATE and floods every GetSingleton with exceptions. When a system force-publishes world data at match start (destroying prior copies) and a subscene ALSO bakes that data, the baked entity arrives after the publish — the destroy pass can't have seen it, two "singletons" coexist, and HasSingleton/GetSingleton THROW (they don't return false on 2+). Worse: the stale bake can hide the bug for weeks — a full reimport (namespace rename, Entities upgrade — anything that changes stable type hashes) suddenly activates the baked copy. Own the invariant with a dedup system at InitializationSystemGroup OrderLast (after scene streaming): tag the runtime-published entity, keep it, destroy late baked copies. (a mobile strategy game: baked ArenaLayout + published MapWalkability killed the whole sim the day a product rename forced a rebake.)
Clients send intent, never durable state. Movement/input → ICommandData (or equivalent command stream); one-shots / control plane → RPCs; high-frequency replicated sim state → ghosts. Do not spam reliable RPCs for per-tick gameplay state — that is what ghost snapshots + change masks are for.
DefaultGameObjectInjectionWorld is NOT reliable under NetCode. ClientServerBootstrap creates ServerWorld before ClientWorld; listen-host "default" is usually the server. Any MB/presentation code that needs ECS must resolve by role (ClientWorld / ServerWorld helpers), never assume default. Creating an RPC in the wrong world is the #1 cause of nameless WarnAboutStaleRpcSystem spam.
RPC TargetConnection semantics flip by world. Client + Entity.Null = send to server. Server + Entity.Null = broadcast to all connections. Always create the RPC entity in the correct world and set the target deliberately.
World.All is NoAllocReadOnlyCollection<World> — LINQ over it throws. First / Where / FirstOrDefault die with "To avoid boxing, do not cast NoAllocReadOnlyCollection to IEnumerable", and only at runtime, so a probe or exec snippet looks fine until the moment it matters. Use a plain foreach. The idiomatic one-liner World.All.FirstOrDefault(w => (w.Flags & GameServer) != 0) is wrong twice — see the next bullet.
WorldFlags.GameServer / GameClient are compound flags. Never (flags & GameClient) != 0 — every Live world false-positives. Require full-mask containment: (flags & WorldFlags.GameClient) == WorldFlags.GameClient (same for server). The shared bit is Live (bit 0), not Simulation/Game: GameServer = 1<<8 | Live, GameClient = 1<<9 | Live, GameThinClient = 1<<10 | Live (com.unity.entities/Unity.Entities/World.cs). Measured: the != 0 form returned true for server, client AND thin-client worlds on both queries — wrong in 4 of 6 pairs — so a First(server) + First(client) pair resolves to the SAME world and every reading after it is attributed to the wrong side while looking perfectly plausible.
Owned player/avatar ghost prefabs need GhostAuthoringComponent.HasOwner = true. Without it, NetCode does not serialize GhostOwner as a replicated field even if the server adds it at runtime — clients never get GhostOwnerIsLocal, so "local player" filters and camera-follow systems see nobody while the ghost itself still replicates fine.
IEnableableComponent enable bits do NOT replicate unless [GhostEnabledBit] is on the component. Server SetComponentEnabled alone leaves clients at the baked default; queries gated on the enable bit return empty on the client with no error. Add the attribute and verify with a client-world probe.
GhostAuthoringInspectionComponent is a variant / PrefabType / SendType picker — not a quantization editor. Quantization + composite live on the variant struct's [GhostField(...)] attributes in code. Prefer small Quantization values so Huffman-delta buckets stay tight; rarely-changing fields ride the change mask for free.
Default transform variant is a project decision — register before NetCode's default. A DefaultVariantSystemBase override for LocalTransform (e.g. PositionRotation without Scale) needs [CreateBefore(typeof(TransformDefaultVariantSystem))] or NetCode logs an override conflict and keeps its default.
GHOST-VARIANT-BINDS-AT-WORLD-CREATION — there is no runtime "stop replicating this component" switch. DefaultVariantSystemBase.RegisterDefaultVariants runs in OnCreate, inside a registration phase that throws once closed (ThrowIfNotInRegistrationPhase), so a variant choice cannot read a singleton or any live config — no singleton exists yet. A per-component bandwidth kill switch must therefore be a PROCESS-level flag set before the first netcode world is built, and it must hold the SAME value on the server and every client: the ghost component collection is hashed at connect, so a one-sided DontSerializeVariant refuses the join looking like a transport fault (same failure mode as NETCODE-RPC-HASH-RESTART-BOTH). That makes it deployment-wide, never per-server or per-connection, and it constrains thin-client soak harnesses too. GhostRelevancy is per-ghost and MinSendImportance is global; neither silences one component. The live-looking alternative — stop WRITING the fields so delta compression drops them to the change mask — costs change-mask bits rather than zero and silently alters simulation, so it is not a bandwidth switch.
After NetCode tears down/creates Game worlds, re-load SubScenes into the new world. Placeholder / local-world destruction drops auto-loaded SubScenes and their baked singletons; call SceneSystem.LoadSceneAsync (or project equivalent) into Server/Client worlds after connect bootstrap.
Dedicated server should not pay client render cost. Strip/disable render components (MaterialMeshInfo, render bounds, etc.) on server ghost prefabs or via a first-tick strip system so server Instantiate clones stay renderless.
Ghost importance / distance scaling needs the full singleton + per-connection setup. GhostImportance + GhostDistanceData on ONE singleton entity; every connection needs GhostConnectionPosition before GhostSendSystem (seed a default if unknown). Without MaxSend* caps + relevancy, distance importance alone often shows no gain.
Seed NetCode metrics singletons if you read them. NetworkMetrics / SnapshotMetrics / GhostMetricsMonitor stay empty until something ensures the monitor/singleton exists — HUDs and log systems must seed, not assume NetCode auto-fills.
Editor "Server Tick Batching" warnings are usually main-thread stalls (editor/MCP/compile), not sim overload. WarnAboutBatchedTicksSystem is editor-only; judge real server cost from a dedicated UNITY_SERVER build + headless load. MPPM multi-editor instances measure editor overhead, not shard capacity.
Live NetCode diagnostics go through MCP netcode/* + dots/* probes first (see unity-mcp.md) — connection state, ghost list, snapshot acks, prediction/NetworkTime, entity dumps — before guessing from scene hierarchy or asking the user to click the Multiplayer PlayMode window.
PRE-BATTLE-WINDOW-PRESENTED-GATE - server-timed pre-battle windows (opening-bonus pick, battle countdown) burn behind the client loading screen; gate them on a client "presented" RPC and accept late picks through the NEXT phase. The mechanism: NetCode's server fixed-step accumulator catches up on REAL elapsed time (UnityEngine.Time.time loses hitch seconds to maximumDeltaTime, the tick accumulator does not), so scene-load stalls make the server burn the whole authored window in batched ticks before the player sees a single frame of it. The human's pick RPC then lands after the timeout auto-pick - and if the handler only runs during the pick phase, the RPC is silently never consumed while the client-side confirmation toast still names the clicked pick. Reads as "the bonus does nothing" with a fully correct apply chain. Diagnose by comparing World.Time.ElapsedTime vs UnityEngine.Time.time vs realtime - server elapsed exceeding Time.time is the catch-up tell (and MCP probes stall the main thread, burning the window faster while you watch). Fix shape, three parts: (1) client sends a one-shot presented RPC when the loading outro actually finishes (app-flow InGame), and the server flow system HOLDS every pre-battle phase timer until it arrives (sim-seconds failsafe so headless never hangs); (2) the pick handler accepts picks through the following countdown phase so a late click REPLACES the timeout auto-pick via the normal switch path (live effects reversed/applied), locking only at Playing; (3) the pick UI stays visible and interactive through that same phase. (a mobile strategy game: "Fortified Hold gives no building HP" - authored data, bake, and +300 apply were all correct; the pick never landed.)
GHOST-DRIVEN-SCENE-VISUAL-RECONCILER - a scene-object visual keyed to ghosted sim state (which model a site shows, which banner a flag flies) is owned by a client/local system that RECONCILES it every tick against the ghost field - never by a one-shot server-side push onto scene objects. The one-shot push is triple-broken under NetCode: (1) it runs in the server world, so a joined client never executes it and shows the authored/stale look forever; (2) it races async scene/map applies - whichever writer lands last wins, and the authored re-stamp usually lands last; (3) any later rebuild of the scene objects (map package re-apply, layout refresh) silently reverts the pushed state with no error. The reconciler shape fixes all three at once AND makes runtime state changes (a bonus re-deal, a capture skin swap) free: compare the scene object's current binding to the ghosted id, restamp only on mismatch, cache the scene-object list and refresh it when entries die (Unity fake-null), with a ~1s rescan cooldown so an empty scene never becomes a per-frame FindObjectsByType. Sibling of "the ghost snapshot IS the event bus" (the id must be a GhostField first) and of the no-silent-legacy- fallback rule (the authored stamp is exactly such a fallback). (a mobile strategy game: bonus building models were pushed once by the server spawn system - clients and every second map apply showed the authored layout while the sim ids were shuffled, read as "gold mine bonus not linked to the gold mine model".)
SEAT-BIND-IDENTITY-NOT-ARRIVAL - server seat/slot assignment binds by PLAYER IDENTITY against the lobby roster, never by connection-arrival order; and "works on the first run, wrong on every later run" can be WARM TIMING, not a stale cache. The trap has two layers. (1) First-come-first-served binding (each new connection takes the first free seat) looks correct in every cold test: first-run guests connect slowly (UGS init, cold scene loads, shader compiles) so arrival order happens to match lobby order. On the second lobby of the same app run everything is warm, relay joins race, and seats shift - every per-seat surface stamped in lobby order (names, races, colors) now points at the wrong player. It reports as "something is cached and not reset", sending the debugger on a stale-static hunt when every static is actually clean. When a session-order bug only appears on the Nth run, check TIMING-dependent assignment before hunting caches. (2) Fix shape: the host stamps per-seat player ids in the SAME lobby-order pass that stamps names/races; each client sends a one-shot claim RPC with its id after connecting; the server binds the claimed lobby seat, holding unclaimed connections for a short grace window before first-free fallback (old client / dropped RPC), and binding first-free immediately when nothing is stamped (local play). Bonus: identity claims give mid-match rejoin the SAME seat back for free. Companion defect from the same hunt: per-seat COLOR palettes duplicated across systems drift (proxy tints had seats 0<->2 swapped vs HP bars/minimap - one player read "green" on units and "blue" on the HUD); one canonical seat palette class, every consumer references it. (a mobile strategy game: "third player in lobby is green for me and second is 2, only on the second lobby".)

LATIOS-BOOTSTRAP-STATIC-NEW — route world creation through `LatiosClientServerBootstrap`

Latios's NetCode world-creation overrides are declared static new, not virtual/override — hidden, not polymorphic. A call to plain ClientServerBootstrap.Create{Client,Server}World therefore silently builds a Unity.Entities.World: no InstallKinemation, no skin-matrix binding, every skinned mesh at bind pose at the origin — while the BAKE side stays correct, so prefab inspection looks perfect. Route every call site through Latios.Compatibility.UnityNetCode.LatiosClientServerBootstrap, and verify in Play that World.GetType.FullName == "Latios.LatiosWorld".

Commands and prediction (a survival game, four separate live movement bugs)

Never put per-frame DELTAS or one-shot presses raw in a per-tick ICommandData. AddCommandData keeps ONE command per InputTargetTick (last-write-wins), so above the sim tick rate the intermediate frames' summed look-delta and any edge-triggered button on a coalesced frame vanish from the authoritative stream while local presentation already consumed them — a permanent, invisible desync (it made "walk forward" push the player backwards, with no error anywhere). Wire ABSOLUTE values (idempotent under overwrite/reuse/drop), or accumulate deltas and latch edges until that tick's command is emitted.
Edge-gate one-shot command actions against the PREVIOUS tick's command. GetDataAtTick legitimately reuses one command for several ticks (a client below tick rate), so a held/latched button bit re-fires the action every tick it is seen — classic pogo-jump / double-fire. Compute the edge from the same command buffer on both sides: it needs no extra replicated state and stays rollback-deterministic.
LATE-COMMAND-RESTAMP-REPLAYS-A-TICK-KEYED-PRESS (0.3.82): never dedupe a press by the command's tick. The server's CommandReceiveSystem restamps every received command older than the current server tick ONTO that tick, so a late or redundantly resent command carrying a press arrives again under a newer tick and a "newer than the last input tick" guard fires it twice. The same restamp can land the press past the tick the client predicted it on, so the prediction misses it and sits wrong until its divergence window. Measured on a toggle (equip/holster) at a 30 fps client and 60 Hz sim: 2 of 8 presses toggled twice or were missed; with the fix 8 of 8 applied once and the client agreed within 2 frames. Fix: the command carries a running press COUNT (an absolute value); the simulation takes forward deltas only (a wrapped-byte delta above 127 is an older reordered command, ignored), the first command after spawn sets the baseline, and a press seen while the action is blocked is consumed, not queued. A toggle applies on an odd delta.
In any rate-managed system group, integrate SystemAPI.Time.DeltaTime, never 1/SimulationTickRate. The prediction loop runs a FRACTIONAL partial tick at the end of fast render frames, and the server sim group BATCHES ticks under load (MaxSimulationStepBatchSize: one update advances N ticks with DeltaTime = N × fixedDt). A hardcoded dt over-integrates the client's partial tick (overshoot + rollback snap = per-frame jitter) and silently discards N−1 ticks of server displacement, which reads as input lag that only appears when the machine is loaded. Client AND server.
Rubber-banding on LOCALHOST is never latency — do not smooth it. Reconciliation runs even in host mode, so every localhost snap is the predicted and authoritative sims producing DIFFERENT output for the same tick. Find the input or the maths the two sides disagree on; smoothing/interpolation hides the only signal you have. Related: prediction replay must feed each replayed tick its HISTORICAL input, never a live "Current" presentation value such as camera yaw.
BOOT-LOCAL-IS-NOT-THE-MATCH. A NetCode CreateLocalWorld boot dummy is not an offline match. WorldSystemFilterFlags.Default as the inject mask matches Client and Server systems, so InjectUserSystems / InstallKinemation on that world Burst-JITs the whole game on the main menu (Editor 20+ GB, player ~1 GB AOT). InjectUnitySystems is not a shell either: namespace contains "Unity" keeps Entities Graphics, Physics, and NetCode. System [WorldSystemFilter(Client|Local)] does not keep them out. Do not GetAll(Default) for Local/Editor and then filter — inject an explicit Entities+Transforms core list. Latios AfterLiveBakingSuperSystem parents LiveConversionEditorSystemGroup; install it only on WorldFlags.Editor, not the Play dummy. Kinemation only on Client; sim only on Server+Client created at match start. Solo = local host, never the boot dummy. Burst JIT is sticky in-process — restart the Editor to reclaim; deleting BurstCache forces a recompile. A clean dummy still peaks tens of GB: Editor LLVM lives in Unity.exe and boots at Play domain reload. Do not set BurstCompiler.Options.EnableBurstCompilation = false to save menu RAM — Disable talks to LLVM, can spike 15–20 GB, persist EditorPrefs, and crash Play reload. (a mobile strategy game: Local+Editor bootstraps installed Kinemation + all Default gameplay.)
SPAWN-GATES-ON-GROUND-SUPPORT — release a spawn only after a positive ground-support probe under its column, and refuse to persist a position below the floor or mid-fall. A body released before its tile collider streams in falls THROUGH the world and no later query saves it: once below a one-sided mesh every downward probe misses forever (observed 1,200 m under a loaded tile). The shutdown flush then saves the plummet — it corrupted the developer's save twice. (a survival game)
NETCODE-SERVER-REVALIDATES — the server repeats every client gate BEFORE any mutation. Range, ownership, finiteness (math.clamp does not fix NaN — check finite first), token cost / budget / time validity, target eligibility, current state and rate — validate, THEN move or spend, never the reverse; proximity checks fail CLOSED when a transform is missing; same-tick duplicate requests bypass delayed structural changes, so use immediate guards or processed sets; and the client stays STRICTER than the server (a 9 m client reach inside a 12 m server range) so the preview never shows a spot the server will reject. Acceptance is not success: an "accepted" diagnostic proves nothing changed. (a survival game; Mirror twin: ACTOR-FROM-CONNECTION.)
CODE-GHOST-PREFABS-BOTH-WORLDS — a code-created ghost prefab is built IDENTICALLY (components, UUID, ghost mode, variant) in server, client AND thin-client worlds, and movement commands live on the owned ghost in both. A thin-client GhostCollectionPrefab that maps the player ghost to a missing prefab makes soaks pass against nothing. Add server NetworkStreamInGame BEFORE waiting on GhostCollection — it needs an in-game connection, so waiting first deadlocks the spawn handshake. (a survival game)
BACKEND-SWAP-CARRIES-VALIDATION — a replacement authority system re-attaches every validation the old one carried, in the same slice. The first KCC cut "stood down" the legacy motor and with it the owner-vs-connection check, finiteness, rate budget and envelope clamps, silently. And an anti-cheat displacement envelope measures the MOTOR's displacement, never the final transform: a server-authored rescue teleport measured post-repair read as thousands of m/s, tripped the clamp, and cancelled itself every tick. (a survival game)
NETCODE-RPC-HASH-RESTART-BOTH — adding or changing ANY IRpcCommand (or ghost) changes the NetCode collection hash; restart Play on BOTH endpoints, every MPPM virtual player included, or the join fails with a hash mismatch that looks like a transport fault. Ship a feature's RPC vocabulary in one file, complete, so the hash changes once. (a project docs/pitfalls/building.md:6, storage-haul.md:53; Mirror twin: MSG-SHAPE-IS-THE-WIRE.)
LATE-JOIN-SNAPSHOT-STREAMS-PER-PIECE — a FixedList in an RPC serialises its full CAPACITY, not its length, and a thousand-piece snapshot in one tick overflows the reliable send queue (hard error, connection drops). Stream per piece, N per tick (64), cursor on the connection entity, terminator RPC, and make the client apply IDEMPOTENT by id (create-if-absent / update-if-known / remove-if-known) because live broadcasts are deliberately sent mid-snapshot. (a project docs/pitfalls/building.md:8-12.)
OWNERSHIP-KEYED-ON-DURABLE-ID — persisted ownership and relationships key on a durable account hash, never NetworkId / GhostOwner / ghost id. Those are session-scoped: demolish rights and is-mine UI break on relog. (a project docs/pitfalls/building.md:24, storage-haul.md:21; seat twin: SEAT-BIND-IDENTITY-NOT-ARRIVAL.)
NETCODE-ATTACKER-IS-THE-MULTIPLIER — budget the EFFECTS of a stream (accepted flips), never the messages. A per-tick input flag flip that triggers an uncapped synchronous flood-fill lets one connection collapse the shard tick; a per-RPC Debug.Log, an O(N) scan, or a reliable broadcast to all in-game connections makes one sender a 500x amplifier; pre-approval RPCs may READ at strict budgets and never allocate a record, write disk or enqueue DB work; and a server→client RPC type sent UPWARD has no consumer, so its receive-request entities leak forever — drain unhandled S→C types. (a project docs/pitfalls/input-hardening.md:8-80; the per-gate list is NETCODE-SERVER-REVALIDATES above.)

Host migration (NetCode 6.5) — a mobile strategy game

MIGRATION-CARRIES-GHOSTED-STRUCTS-ONLY — a component rides host migration only if it has a generated ghost serializer (≥1 [GhostField], or [GhostEnabledBit]), and then the WHOLE struct is memcpy'd, non-ghosted members included. So server-only clocks and latches live as plain members of an already-ghosted struct (a match singleton's NeutralCycleRemaining, a seat's OwnerPlayerId) and cross a host swap for free, while a server-only component with no [GhostField] reverts to its prefab default on the new host. Every such default must be re-derivable from replicated state by a fixup pass (hero flag from level, lane follower from lane+faction, Dead from Health), and raw Entity fields in migrated data are pointers into the dead host's process — null them before anything dereferences. Store durations that count down, never ElapsedTime deadlines: the migrated clock survives only when HostMigrationData.Set runs before the new world's first update, and no ticks run during the outage. (NetCode 6.5.0 HostMigrationTests.cs: "There must be at least one ghost field or this will not be tracked".)
SERVER-ONLY-STATE-IS-A-PREFAB-BIT-NOT-AN-ADD — put server-only per-instance components on the ghost prefab with [GhostComponent(PrefabType = GhostPrefabType.Server)] and express "optional" behaviour as an IEnableableComponent bit, so spawn is Set/SetEnabled only. PrefabType on the attribute strips the component from client prefabs even when the struct has no ghost fields (the package's own example strips RenderMesh), so client archetypes stay exactly as lean as runtime AddComponent left them, per-spawn structural changes disappear from the hottest path, host migration re-instantiates a complete archetype, and a consumer that SetComponents a missing component throws instead of silently taking a fallback branch. Queries filter disabled enableables for free; only ComponentLookup.HasComponent sites need && IsComponentEnabled. (a mobile strategy game: 25 per-spawn adds, three structure-kind tags and one HasComponent fallback that had already hidden a melee regression.)
HOST-MIGRATION-PROMOTE-ORDER — the promoted peer keeps its client world; the server world is created, stamped, given HostMigrationData.Set, and Listened BEFORE its first update; the client then disconnects, ticks once, ResetDriverStores, and Connects. Set needs a HostMigrationConfig singleton — the package's ServerHostMigrationSystem.OnCreate already makes one with StoreOwnGhosts=false, so patch the existing entity, never assume absence — and Set itself creates EnableHostMigration and HostMigrationInProgress. Gate your own fixup on a game-owned tag, not on HostMigrationInProgress: the package destroys it on TIMEOUT as well as success, so a failed migration would fall through into a fresh match bootstrap under players who think they are continuing. ResetDriverStore throws while any NetworkStreamConnection entity exists (it is a cleanup component: request the disconnect, wait a frame). Keeping the client world keeps ConnectionUniqueId, so the server re-grants the old NetworkId and migrated GhostOwner values need no fixup. (Package docs host-migration-systems.md recipe + HostMigrationSystem.cs :485/:498.)
RECONNECT-GRACE-NEEDS-A-REJOIN-PATH — a seat reconnect grace is dead code until the client has a way back: detect "own link down while the lobby still shows the host" separately from host loss, rejoin the SAME relay code, and let the identity claim re-fire on the new connection. Transport reports a dead host only after its disconnect timeout (~30 s default) and the lobby notices the dead player on its own clock, in either order — so one watch loop keys on three signals (host abandoned, host code changed, own link down) and ONE budget clock, and publishes a stage (lost / promoting / rejoining / reconnecting / abandoned) that the HUD renders; the transport layer never hard-cuts scenes. Abandon = the flow owner parks an exit notice, the HUD shows the terminal card and leaves through the normal path, a timed fallback covers a missing HUD. Per-connection "sent once" markers (SeatClaimSent) live on the connection entity, never in a static, so the new connection re-sends for free. Ghosts respawn with NEW entity handles after a reconnect — flush every entity-keyed UI cache on recovery. (a mobile strategy game: the 20 s grace shipped with no guest rejoin path at all.)
HOST-LEAVE-IS-A-MIGRATION — a host that walks out of a live match rides the SAME election path as a host that crashes. A "keep the room for a rematch" path is for a CONCLUDED match only; run it on a mid-match LEAVE and the host stays lobby host, the guests' watch loop sees "own link down while the host lives", and they burn the whole recovery budget rejoining a relay nobody serves before SIEGE ENDED. Correct shape: the leaving host publishes the final migration snapshot, then REMOVES ITSELF from the lobby so the service elects the next player, who promotes exactly as after a crash; bound the hand-off (a few seconds) so a dead service never holds the loading screen, and the pause sheet tells a host with guests what LEAVE does. (a mobile strategy game: the first two-client test was a host LEAVE, and it looked like migration had never been built.)

ONE-WORLD-SPAWN-BREAKS-PREDICTION - a non-ECS spawner must fill EVERY game world

In a multi-world process (Local Host = ClientWorld + ServerWorld), anything that creates simulation-relevant entities from a non-ECS source - colliders, obstacles, scatter, pickups - populates EVERY game world present, deterministically from the same seed/source, or none of them. A spawner that picks "the server world if present, else the client" leaves the predicting client without the bodies the server collides against (and the reverse on a real client + dedicated server pair), so every contact is a misprediction: the client walks through what the server stops it on, then snaps back on correction. Verify with a per-world PhysicsCollider count, never from the spawner's header comment. (a space sandbox game: ScatterEcsSpawner.TryResolveWorld measured ServerWorld 2602 PhysicsColliders against ClientWorld 2 under Local Host, while its header comment claimed both worlds were populated.)

THIN-CLIENT-PUMP-IS-CALLER-DRIVEN - N thin clients need a per-update pump, not a 1 Hz tick

AutomaticThinClientWorldsUtility.NumThinClientsRequested = N creates nothing on its own. The worlds appear only while something calls UpdateAutomaticThinClientWorlds, and in a stock project the ONLY caller is Netcode's own MultiplayerPlayModeWindow - close that window and a soak silently populates zero clients. Worse, when CreationFrequency != 0 the call spawns AT MOST ONE world per invocation (maxAllowedToSpawn = 1, gated again by 1/frequency seconds since the last spawn), so a tool that pumps on its own 1 Hz readout timer reaches N after N seconds - twenty minutes for a 1000-client soak that was supposed to ramp in one. Any tool driving thin clients pumps from EditorApplication.update and keeps its data sampling on a separate, slower timer. Two further traps in the same call: it returns early while EditorGUIUtility.editingTextField is true (typing a population in your own field stalls the ramp you just asked for), and ReferenceWorld null falls back to ServerWorld ?? ClientWorld. (a space sandbox game: a Control Panel soak launcher pumped at 1 Hz; 20 thin clients took 20 s and the bug would have read as "thin clients are slow to connect" at N=1000. After moving the pump to every editor update, 20 worlds and 21 server connections came up in under 5 s.)

SIMULATOR-LIVE-PUSH-NEEDS-JOB-DRAIN - complete tracked jobs before writing driver pipeline params

NetworkSimulatorSettings.RefreshSimulationPipelineParametersLive(in parameters, ref driverStore) writes the transport pipeline's m_StaticInstanceBuffer, which RpcSystem:RpcExecJob reads. Called straight from a button handler or EditorApplication.update it throws "previously scheduled job RpcSystem:RpcExecJob reads from ... you must call JobHandle.Complete". Grabbing the singleton RefRW does NOT drain that job. Drain first - world.EntityManager.CompleteAllTrackedJobs is affordable for a one-shot operator action - and still wrap the push, because a driver built while the simulator was disabled has no simulator stage to modify. (a space sandbox game: a Control Panel latency/jitter/loss preset threw on first click under Local Host; draining tracked jobs on the client world fixed it and the preset then applied without a reconnect.)

PIPELINE-STAGE-BUFFERS-ARE-PER-PIPELINE-AND-FAIL-EMPTY - a missing stage reads as zero traffic

A custom INetworkPipelineStage gets one shared buffer per PIPELINE it was registered into, not one per connection. A driver built the usual way has three - unreliable, reliable, unreliableFragmented - so a stage that counts anything holds three separate running totals for the same connection, and code that reads unreliablePipeline and calls it the connection total silently drops the other two. Worse, NetworkDriver.GetPipelineBuffers(pipeline, stageId, connection, out _, out _, out shared) does not fail when the stage is absent from that pipeline: it assigns default and returns, so shared.Length == 0 and a naive reader sums zero and reports a healthy, plausible "sent nothing". Any readback checks shared.Length >= UnsafeUtility.SizeOf<T> and surfaces the shortfall as its own state - never as a zero. The same call DOES throw InvalidOperationException("Invalid connection") under ENABLE_UNITY_COLLECTIONS_CHECKS for a connection the driver no longer holds, and NetCode's NetworkStreamConnection is ICleanupComponentData, so it OUTLIVES its entity: a query on it alone walks straight into torn-down connections. Filter with WithAll<NetworkId> - ordinary component data, removed with the entity - so only live connections are read. (a space sandbox game: a per-connection bandwidth ledger over a custom byte-meter stage. All three traps were live in one 60-line system - three pipelines to sum, an empty-buffer path that would have reported an unmetered driver as an idle client, and a teardown throw waiting on the first disconnect.)

Amended - a readback flag guards nothing until every display reads it, and the stage must sit on the driver path your numbers come from. A dedicated server built its drivers through a relay path that added the metering stage only behind a command-line flag. The ledger flagged every connection unreadable, correctly. The console ignored the flag and printed the untouched zero totals, so a live client showed 0.0 KB sent and received. Add the stage by default in the process whose numbers are read. A stage with no header changes nothing on the wire, so clients need no update. Every display renders the flagged state as unmetered (JSON null), never as 0. Test the off path as well: one run with the stage disabled must print unmetered.

IPC-IS-NOT-HOST-IDENTITY - the client's transport is picked ONCE at world creation, and the simulator forces sockets

TransportType.IPC looks like a free answer to "is this connection the host's own client", because an in-process client normally gets the IPC driver. It is not an identity signal. DefaultDriverConstructor.ClientUseSocketDriver decides the CLIENT's transport exactly once, while the client world is being created, and never revisits it. It returns socket on the FIRST of:

NetworkSimulatorSettings.Enabled - in the Editor that is the sticky MultiplayerPlayModePreferences.SimulatorEnabled pref, so any tool that toggles the simulator (a latency/loss preset button, a soak harness) silently changes transport for every later session. This branch runs BEFORE the server-world check, so an in-process host client still gets a socket. The pref is the runtime type Unity.NetCode.MultiplayerPlayModePreferences, not Unity.NetCode.Editor.*; an Editor script that names the wrong namespace fails to compile. Restore SimulatorEnabled and PacketDelayMs after every latency test, since the pref outlives Play.
ClientServerBootstrap.RequestedPlayType == Client.
No server world in the process AT THAT INSTANT - creation ORDER decides it. CreateDefaultClientServerWorlds makes the server first, then the client, and gets IPC; a menu "host now" path that adopts a client world created earlier gets a socket.

Consequences:

Any authorisation that reads the driver type is a coin flip on editor prefs. Key "this is the host's own client" on an in-process secret instead: mint a 128-bit nonce (RandomNumberGenerator, not Unity.Mathematics.Random - this is the auth secret) when the server world is created, hold it in a process-wide static, have the in-process client send it in an RPC, compare on the server. A remote process has no such static and sends nothing. Guard the all-zero case explicitly: a process that minted no nonce must not match a client offering zeros.
Toggling the simulator pref mid-session changes nothing - the driver is already built. Restart Play.
A "one attempt per connection" tag added through an EntityCommandBuffer is NOT visible to EntityManager.HasComponent until playback, so a flood inside one frame gets an attempt per packet. Track the connections answered this update in a NativeHashSet<Entity> as well as the tag.
WorldSystemFilterFlags.ClientSimulation does not include ThinClientSimulation (separate bits), so a nonce-sending system marked ClientSimulation alone will not run in a soak's thin clients - which is exactly what stops every thin client claiming to be the host. Marking it both hands them all admin.
RPCs ride the reliable pipeline (RpcSystem only ever calls BeginSend(reliablePipeline, ...)), so a one-shot send survives simulated packet loss; no resend logic needed.

(a space sandbox game: the Local Host client came up driverId=2 type=Socket, the admin grant policy read that as "not the host", and admin flight was unreachable in every session on the only permitted test target. SimulatorEnabled was still true when checked - the project's own capacity tab is what writes it.)

GHOST-QUANTIZATION-IS-AN-INT32-WORLD-BOUND — a quantized position silently teleports past 2^31/scale

[GhostField(Quantization = N)] on a float does not "compress" it — it changes the wire TYPE. The shipped template (Runtime/SourceGenerators/Templates/DefaultTypes/GhostSnapshotValueFloat...additionalfile) declares the snapshot field as public int and writes

  snapshot.Field = (int) math.round(component.Field * __GHOST_QUANTIZE_SCALE__);

so the representable magnitude is a hard 2^31 / N in the field's own units:

Quantization	precision	max magnitude
1000	1 mm	+/- 2,147 km
100	1 cm	+/- 21,474 km
10	10 cm	+/- 214,748 km
0	full float32	unbounded magnitude, but see below

Past that the unchecked float->int cast is not an exception and not a clamp: it produces a garbage int (int.MinValue on x64), so the value TELEPORTS. Nothing warns, at author time or at runtime.

The discriminator is whether the quantity is BOUNDED. Angles, velocities, health, stamina, timers are bounded by physics or design, and PhysicsVelocityVariant ships Quantization = 1000 on Linear/Angular for exactly that reason. Positions and accumulators are not bounded, and quantizing one imposes a world size nobody wrote down.
Unquantized is not the fix at scale, it is the other failure. float32 has a 24-bit mantissa, so its ULP grows with magnitude: ~0.5 m at 5,000 km. A raw float position on a planet-sized world is jittery rather than teleporting.
The fix that survives both is DATUM-RELATIVE state: replicate the offset from a chunk/cell/planet origin the receiver already knows, so the magnitude stays small — then quantization is both safe AND precise. Ghost a world position only when the world is provably smaller than 2^31 / N.
Check your project's transform ghost variant too: whatever quantization it carries applies the same bound to every ghost's position. It is configurable, so read yours rather than assuming.
Reviewing a diff: any NEW quantized [GhostField] whose value is a POSITION, a distance from an origin, or an accumulating total is this bug. Compute 2^31 / N and compare it to the world's radius before approving.

(Found a space sandbox game, in review rather than at runtime: a terrain-hold system latched [GhostField(Quantization = 1000)] public float TerrainHoldHeight — a world-space Y — giving +/-2,147 km on a 5,000 km-radius planet, in a project that had already recorded a planet-frame Y of -4,484,426 m. Latent, because the spawn frame sat near y = -57. It was the only quantized ghost field in that codebase carrying a position; every other one was an angle, a velocity or a survival stat. Developer ruled: no quantization on any world-position value, make it relative to the chunk datum.)

NETCODE-CORRECTION-METRICS-NEED-A-POSITIVE-CONTROL

Validate a prediction-correction recorder with a known server displacement before accepting its zero. In the inspected NetCode source, GhostPredictionDebugSystem compares only when the replay reaches GhostSnapshotLastBackupTick; an applied snapshot newer than that backup can bypass the comparison.
For an applied-snapshot recorder, compare the authoritative state after GhostUpdateSystem with the latest predicted state saved for exactly PredictedGhost.AppliedTick, before applying that snapshot. Refresh the ring on replay too; original-first-prediction samples are a different metric.
Report matching history, snapshots newer than the last predicted tick, and unexplained history gaps separately. No matching tick is no comparison, never zero error. Tick-aligned samples also do not prove that presentation has no discontinuity; camera smoothing is not reconciliation evidence.

Observed on Unity 6000.5.0f1: a +0.5 m authoritative displacement produced about 0.001 m in native prediction metrics, while a validated snapshot/history recorder caught about 0.518 m. Source condition and a positive control are necessary; do not assume the same metric behavior across package versions.

SAME-TICK-CANONICAL-COMPARE - two orbit-space positions are comparable only when evaluated for the same tick

On a planet that orbits at ~28 km/s, one 60 Hz tick moves its surface ~476 m in canonical (star-relative) space. A relevancy rebuild that ran before the system capturing each viewer's canonical position compared a freshly evaluated corpse (tick N) with the viewer's stored position (tick N-1): distance² 226,945 m² for a bag one metre from the feet, so nothing inside the 80 m radius ever reached a client. The same formula run from an editor exec after the frame gave 1 m² and looked correct for an hour of probing - the tick skew only exists inside the frame.

A comparison between a stored canonical position and a freshly evaluated one is a defect unless both carry the same tick. Order the reader after the writer ([UpdateAfter(typeof(TheCaptureSystem))]) or compare in the shared anchor frame (planet-relative pose vs planet-relative pose), which is time-independent.
Guard the order with a test that reads the attribute: an UpdateAfter that goes missing in a refactor reproduces the bug silently.
When an in-system result differs from the same code run outside the frame, instrument the system itself (one throttled log of every operand) instead of re-probing from outside; the outside probe cannot see intra-frame ordering.

CODE-BUILT-GHOST-HAS-NO-REWIND-DELAY - verify timeline tracking explicitly (0.3.85 correction)

Code-built ghosts do not automatically acquire CommandDataInterpolationDelay from ConvertToGhostPrefab. Add it explicitly when using the package delay-tracking path. TrackInterpolationDelay belongs to the authoring baker, not GhostPrefabCreation.Config.

The component exposes the latest connection-reported command/interpolation tick distance and may be stale after a command-target change. Validate/clamp age and use the correct command tick; do not subtract RTT again. Measure render offset.
Inspect installed PhysicsWorldHistorySingleton/LagCompensationConfig behavior and region/world lifetimes before designing separate history. A custom player ring is not universally preferable to collision-world history.
Historical queries account for frame transforms, moving blockers, collider lifetimes and non-physics hit eligibility. Current static geometry is usable only when its identity and transform are valid for the historical query.
Authoritative hit shapes must agree with gameplay and presentation. Animation sampling versus semantic pose is an architecture choice with measured cost and fidelity, not a universal prohibition on server animation.
Test identified shots under latency/loss/jitter, stale/future/replayed input, cover and frame transitions. Compare outcomes, not only totals. Storage history length and maximum accepted rewind age are separate policies.

Evidence: installed NetCode 6.5 delay source and server-rewind sample, https://docs.unity3d.com/Packages/[email protected]/manual/server-rewind.html. This corrects the unmeasured custom-ring prescription introduced in 0.3.84.

REWOUND-HIT-NAMES-ITS-PART - a lag-compensated hit carries the part it met

Shots at a running player landed behind the drawn body: the server traced its current physics world while the shooter's client drew the others the command's interpolation delay earlier.

The package's physics-world history clones the ONE physics world singleton. A server that steps several physics worlds (one per region or frame) gets history for one of them; keep a per-player pose ring instead (tick, pose, frame id, region lifetime, stance; 56 B x 32 ticks = 1.75 KB per player) written after prediction, so entry T is snapshot T, the pose the client interpolated.
Skip live player bodies in the world cast (it still finds the environment stop), then test each rewound body analytically up to that stop; the nearest wins. A frame or region-lifetime mismatch skips the entry, which is how a transfer invalidates the ring without a write.
Name the struck part (band, side) on the rewound pose at trace time and carry it on the hit event. A downstream reaction that re-classifies the point against the current pose names the wrong part for every rewound hit.
A requested age past the accepted maximum is a REJECTION (no player hit, counted), never a clamp to the oldest tick and never the live pose. Storage length and accepted age stay separate settings.

REWIND-FROM-THE-COMMAND-TICK - the reported delay is relative to the command, not the step

Lag-compensated shots at a running target missed at every latency and the rewind depth did not grow with RTT. The client drew the target 4 ticks before the press at 0 ms and 9 at 150 ms; the server read 1 before and 4 after.

CommandDataInterpolationDelay.Delay is filled from the command header, which the client computes at send as the command's tick minus its interpolation tick. Rewinding to serverTick - Delay is short by the command's age whenever the server steps a command late (a common accept window is a few ticks).
Rewind to command.Tick - Delay. Carry the command tick with the aim when the hit resolves in a later system.
Count the accepted-age cap back from the server tick (age plus delay), so a late command cannot reach further into a victim's past than the cap. Show command age beside rewind depth in the diagnostics.
Before tuning depth, log at FIRE the client's predicted and interpolation ticks and the nearest recorded server tick to the drawn pose; compare against the tick the server read. A probe's press frame is not the command's frame: input is sampled a frame or more later.
Amended - the reported delay floors. The client sends ServerTick - InterpolationTick, less 1 when the interpolation fraction exceeds the server fraction, so the integer lags the drawn pose: 5 reported for a target drawn 6.11 ticks behind the command's full tick, and the read landed 1 tick ahead (a running target's miss at 0 ms). Treat Delay as a floor: rewind Delay + 1 and blend the two recorded ticks, or send the fractions; prove it with a test that sets both fractions. Measured once, fix not yet verified live.

A DROPPING SERVER CANNOT BE BOUNDED BY REQUESTS OUTSTANDING (0.3.95)

A per-connection rate budget that destroys an over-budget RPC without a reply makes a throttled request indistinguishable from a lost one. Every client-side bound built on "how many answers am I still waiting for" then fails, because a dropped request is outstanding for good and its slot never comes back. Measured on a streamed-world channel with an 8/s budget: the client got 9 of 220 tiles answered and the count did not move in 95 seconds, so the data those tiles carried never reached the player at all.

Bound the client by asks per second, matched to the server's budget with headroom for whatever else shares it. A count of answers outstanding is not a bound; a permit that is never returned is a leak.
An unanswered request waits the same refresh interval as an answered one. Re-asking it every frame bumps the correlation id past the reply already in flight, so the answer is discarded on arrival and the item never becomes known.
Iterate the work set with a cursor that resumes and wraps, never from the front each update: restarting spends the whole window's budget on whichever items the enumerator reaches first, and the tail is never asked once. The symptom is a count that climbs to a fixed number and stops.
Prefer replying with a throttle result code over dropping silently, so the client can distinguish "later" from "gone" — but design the client to survive a server that does not.

GHOST-BYTE-ACCOUNTING-NEEDS-A-MONITOR-SINGLETON

NetCode gathers per-ghost-type serialized size in every Editor/NETCODE_DEBUG build, but only PUBLISHES it while a GhostMetricsMonitor singleton exists. The send system's per-type EntityCount / SizeInBits writes are inside a plain #if UNITY_EDITOR || NETCODE_DEBUG, so the numbers are already being counted; the stats system's gate is (!hasMonitor && statIndex < 0), where statIndex is only set by an attached network debugger. No monitor, no debugger attached, and the buffers stay empty.
The cost of not knowing is that every byte figure in your own tooling is a description of the PREFAB CONFIG (send rate, importance, which components are server-only) rather than a reading of the traffic. A panel can look thorough and complete and still contain no measurement — found as a hardcoded string reading "achieved per-instance wire bytes: unmeasured" beside three paragraphs of configuration.
Create the monitor yourself: one entity with GhostMetricsMonitor + GhostNames + GhostMetrics, on server and client. Gate the whole file on UNITY_EDITOR || NETCODE_DEBUG, since the component types do not exist otherwise and a shipping build then carries neither the monitor nor its cost.
GhostMetrics.SizeInBits is the WHOLE TYPE's cost in that packet, not one instance's. Per-instance is bits / 8 / InstanceCount; dividing by the number of snapshots reports a herd's cost as one animal's. The buffer is 1:1 with GhostNames by index, and it is CLEARED and refilled every time the monitor's CapturedTick advances, so a reader that samples it directly sees one tick of one type — accumulate over a window and close it into rates.
A tick where a type carried nothing is not a send. Counting those ticks makes every ghost report its send rate as the simulation tick rate, which hides exactly the per-type MaxSendRate tuning you added the measurement to check.
The package's own profiler module DESTROYS a user-created monitor and recreates it (with a warning) when it opens. Read whichever monitor singleton is currently live, by query, rather than holding the entity you created.

ANGLE-SMOOTHING-LERPS-THROUGH-THE-WRAP and COMMAND-REPLICATION-IS-OPT-IN-BY-GHOSTFIELD

SmoothingAction.Interpolate on a wrapped angle whips remote ghosts. The generated float interpolation is a plain math.lerp of the two snapshot values. An angle normalized to +/-180 (or 0..360) that crosses the wrap between snapshots goes the long way: 179 -> -179 renders 89.5, 0, -89.5. That is up to 180 degrees wrong for one snapshot interval, and any delta built from two such fields (aim minus body) inherits the error. Before the change, the default Clamp snapped correctly. Fix it with MaxSmoothingDistance below 360: the generated clamp zeroes that interval's interpolation factor. Alternatives are an unwrapped accumulator (mind the quantization bound above) or a quaternion, which slerps. Prove it on the generated serializer: CopyToSnapshot two components, then CopyFromSnapshot(state, component, .5f, .5f, before, after).
ICommandData server-to-client replication is opt-in by [GhostField]. A command buffer with no [GhostField] is never ghosted. Marking every field turns replication on, and NetCode already defaults command buffers to OnlyPredictedClients + SendToNonOwner. Adding both therefore enables replication; it does not fix a leak. Before claiming a fix, check whether a ...GhostComponentSerializer was generated for the command type. The recipients are non-owners that predict the ghost. With owner-predicted players and no prediction switching, that set is empty.
Command wire cost comes from the packed path, and Quantization does not apply to commands. The command sender writes the newest command with the PACKED serializer against a zero baseline. Each older command follows as a change bit plus a packed delta against the newest. The unpacked Serialize(writer, state, data) overload is not on that path, so its byte count is not the wire size. Command floats go raw whatever the attribute says: WritePackedFloatDelta costs 1 bit unchanged and 33 bits changed. A short is written as an absolute packed int on every command. Compare encodings by running the generated packed serializer over a representative tick sequence in that layout, not by comparing struct sizes.

(Found, NetCode for Entities 6.5, reviewing a batch of "network optimizations". A jitter fix added a turn-around whip. An input "leak fix" enabled replication that reached zero recipients. A four-short to float-quaternion change saved 4-16 bits per packet at rest and cost 45-192 bits per packet while rotating. The same review confirmed a flag-packing change across all 12,288 states.)

REPLICATED-FIELD-NAMES-ITS-CLIENT-READER (HARD, developer)

The check happens BEFORE the attribute is typed, and its five answers go in the report.

Reader. Name the client file and line that reads the value. A field with a server writer and no client reader is pure wire cost. Server-only state takes no [GhostField], or [GhostComponent(PrefabType = GhostPrefabType.Server)] when the whole component is server-side.
Derivable. If the client already receives what it needs to compute the value, do not send it. Check the derivation by measurement, because it cuts both ways: a stride speed derived from ghost position was already what the view used, so the replicated copy was dead; a turn rate derived from interpolated rotation came out 32 deg/s RMS off and had to stay replicated.
Audience. Owner-only values use SendToOwner; predicted-only values use OnlyPredictedClients. "Everyone" is a decision, not the default.
Precision by what is SHOWN. Replay real server samples through snapshot quantization, linear interpolation and the view's own smoothing, then compare against the unquantized value. Hold the shown error under one pixel at the nearest plausible viewing distance. A unit vector on a known hemisphere sends two parts and rebuilds the third (measured: same shown error, 4 bits cheaper).
Price. Price the field per send on real samples with NetCode's three-baseline predictor and the packed-integer cost table (2 bits for a zero delta, then bucket code plus raw bits). The predictor takes a SECOND difference, so a noisy signal costs more than its plain delta: a body-tilt normal at Quantization 1000 cost 32 of a 91-bit update, more than the position.

Narrowing the C# type saves nothing on a ghost field. Small integers are stored as 32-bit snapshot values and sent as packed deltas; byte and int cost the same. Quantization and removal are the levers.
A comment that says why a field is replicated goes stale silently. One header still said "replicated rather than derived" after a later commit switched the client to deriving. When a client reader is removed, remove the replication in the same commit.
One ghost type can carry a cheaper wire form of a shared component. Declare a [GhostComponentVariation(typeof(LocalTransform), "...")] struct with the wanted Quantization and only the fields that change, then pass it per prefab: ConvertToGhostPrefab(manager, prefab, config, overrides) with a ComponentOverride { OverrideType = Variant, Variant = GhostVariantsUtility.UncheckedVariantHashNBC(typeof(TheVariant), ComponentType.ReadWrite<LocalTransform>) }. Other ghost types keep the default variant. Prove it loaded by finding the generated ...TheVariantGhostComponentSerializer type after compile.
Live bytes per instance swing with activity. Compare an old and a new layout by pricing both on the SAME captured samples, and record the live wire row over that same window as the check.
Audit form: list every [GhostField], and for each one find its client reader. Rejected by measurement in one audit, so do not retry blind: deadbands on a noisy normal or turn rate (under 1 bit), writing a terrain normal less often (3 degrees of shown tilt error, because collider normals jump per triangle), and moving a body-local normal to region space (no gain).

(Found, NetCode for Entities 6.5, 48 interpolated animals: one unread float, a three-part normal and default transform precision made up 20% of every update. Removing them took the largest type from 9.2 to 7.4 bytes per instance with the shown pose under a pixel from two metres.)

STORED-TICK-KEEPS-ITS-VALID-BIT

A uint that stores a server tick must hold ONE of NetCode's two forms, and every writer must use the same one. NetworkTick.SerializedData is (index << 1) | 1; the low bit marks the tick valid. TickIndexForValidTick is the bare index. A reader that rebuilds new NetworkTick { SerializedData = stored } from a stored index sees every even index as an INVALID tick: IsValid is false and TicksSince means nothing.
It fails on tick parity, so a run can pass by luck. Measured: a respawn cooldown gated on deathTick.IsValid && now.TicksSince(deathTick) >= 180 refused respawn forever for a death stored at index 1676, while two earlier runs had passed on odd ticks. Three of the five writers of that field stored the index; the original writer and the only reader used SerializedData.
Fix: give the component one static constructor that takes a NetworkTick and stores SerializedData, plus one readonly property that rebuilds the tick. Route every writer through the constructor, using new NetworkTick(index) where only the index is at hand. Test with an even AND an odd index.

(Found, NetCode for Entities on Unity 6000.5, in a death and respawn loop. The locked-out player's corpse reached its loot lifetime and despawned with the inventory before anyone noticed.)

VIEW-QUERY-READS-ONLY-WHAT-INTERPOLATED-GHOSTS-CARRY

A client presentation query that requires a component stripped from interpolated ghosts matches nothing on those ghosts, so the view is never built. No error, no warning: the entity simply does not draw for anyone but its predicting owner.

The trap: a component whose ghost variant is GhostPrefabType.AllPredicted (or PredictedClient) exists only on predicted ghosts. Every remote player, animal or NPC that a client interpolates lacks it. Adding it as a RequireForUpdate/query term to a view system (for example, a death state read so the body can play its death pose) silently drops every interpolated instance.
Measured twice in one codebase, same day: no animal body drew on clients for four days (the view required Health and a death state; animal ghosts are interpolated-only), and no remote player body drew for a day after a death-state term was added to the character view query. Both passed every EditMode test and every single-player Play check, because the local player is predicted.
Rule: before adding a component to a client view query, look up its ghost prefab type. If it is not on interpolated ghosts, read it optionally (ComponentLookup.HasComponent / TryGetComponent) or read a replicated signal that IS on them (a replicated action or state field). Never make it a required term.
Test it with a remote instance: a check with only the local predicted player proves nothing here. Spawn a second, interpolated instance (a second client, a server-owned dummy, an animal) and assert its view exists.
Presence is not replication: a per-prefab GhostPrefabCreation.ComponentOverride with only ComponentOverrideType.PrefabType puts the component on interpolated copies, but the serializer's send mask still comes from the attribute's prefab type, so the value is never sent: the copy holds its default forever. Measured: a server-killed ghost read Dead=0 on the client 3 s later. Set ComponentOverrideType.SendMask with SendMask = GhostSendType.AllClients in the same override, and check the remote copy's VALUE changes, not only that the component exists.

(Found, NetCode for Entities on Unity 6000.5.)

PREDICTED-DATA-IS-PROTOCOL

Authored data that both the owning client's prediction and the server simulation read (a cadence in seconds, a strike arc, a reach) is part of the protocol, even when no RPC, ghost field or command struct changes. Found in review: a melee item's swing time went from 0.95 s to 1.3 s and its strike arc moved. Both are read through one shared item definition by the predicted step and the server step, and a structural wire detector reported no compatibility change. A client on the old data predicts swings the server refuses, so every swing is corrected (expected, not measured; the change rides a compatibility bump).

When a change touches data a predicted system reads, raise the compatibility number with it, whatever a structural detector says.
Name the shared readers in the merge: the definition both sides read and each system that runs on it.
A detector that checks only wire types needs a second list: the authored assets prediction reads.

RECONCILE-ADOPTS-EVERY-GATE

One trigger press predicted two rounds while the server fired one. The owning client's predicted copy steps only first-time ticks (no rollback). At about 12 fps the server snapshot for the press tick arrived before the copy had stepped that tick; reconcile took the server's shot count and magazine but kept the copy's own next-allowed-fire tick, so the copy's next tick found the same held Fire command still inside its age window with the gate open and fired again. The same gate also served melee swings and dry-fire clicks. Server ammo and counters were right, so only the client HUD and effects showed the extra round.

Every value the predicted step reads to decide WHETHER to act (next-allowed tick, cooldown, charge, sequence, ammo, reload state) is adopted from the server as one set. A gate kept local is a second source that reads stale after a reconcile.
The fix is replication (an owner-only predicted field), which is a compatibility change; bump it.
The gate that should have caught it: a test that sends ONE held press through the real step on server and client and asserts the predicted count moves by exactly 1. Reconcile tests that set counts by hand never run the step and cannot see this.

PLAYER-BUILD-HAS-NO-NETWORK-SIMULATOR

A batch multiplayer sitting (dedicated server plus two player-build clients) was planned with latency, jitter and loss rows. NetCode's NetworkSimulatorSettings and the simulator stage that DefaultDriverConstructor adds are compiled only under UNITY_EDITOR || NETCODE_DEBUG. No build profile defined NETCODE_DEBUG, so --loadNetworkSimulatorJsonFile did nothing in a player: the class does not exist there, and no error says so.

Before planning adverse-network checks on builds, check the build's defines for NETCODE_DEBUG. A player without it runs a clean link whatever the command line says.
Either impair at the OS (a WinDivert tool such as clumsy, filtered to one client's UDP port; needs an elevated start), or add NETCODE_DEBUG to a development-only client profile. That define also turns on NetCode's debug systems and their allocations, so measure performance on a build without it.
Gate: confirm the impairment took effect before measuring. The server's per-connection RTT (NetworkSnapshotAck.EstimatedRTT) must move by the injected delay.

JOIN-HASH-COVERS-ONLY-OUTCOME-INPUTS

Found in source before it shipped: ground-paint children (masked splat layers with no height) added to a planet's vendor terrain would have changed the terrain content hash that client and server compare at spawn admission, so every client on the previous build would be refused with a catalogue mismatch. Heights were identical. The captured height recipe hashed the terrain's feature COUNT, and the count includes colour, texture and zero-height layers the server never samples.

A join or compatibility hash covers only the inputs the server's outcome is computed from (here, the height operations). Never hash bookkeeping the simulation does not read: counts of presentation children, names, order.
Test it both ways: a presentation-only edit keeps the hash, and a single outcome edit moves it.
Dropping a field from the hash moves every existing hash once, so it ships with a compatibility bump.
A detector that checks only wire types cannot see authored assets hashed at admission. Bump by hand and name the hash in the merge (see PREDICTED-DATA-IS-PROTOCOL above).

SERVER-SLEEP-WARNING-COUNTS-SLOW-FRAMES

A headless dedicated server logged Expected server to always update once per frame when in sleep mode. Ran N steps about every 10 s, and a source read blamed the boot for never setting Application.targetFrameRate. Wrong: in Sleep mode (the default for a UNITY_SERVER player, NetcodeTimeTracker.ShouldSleep) NetCode writes targetFrameRate itself every frame (AdjustTargetFrameRate: tick rate +/-2 to hold half a step in the accumulator), and NetworkTime calls NetcodeServerRateManager.WillUpdateInternal every frame, which warns, unthrottled, whenever that frame's delta yields other than exactly 1 step.

The warning counts server frames outside about 0.5-1.5 ticks (8-25 ms at 60 Hz): a slow frame runs 2+ steps, a short one runs 0. Boot and load spikes log it too.
Never set targetFrameRate (or apply player graphics settings that set it) on a server; the warning's own text says to remove it, and it fights NetCode's pacing.
Gate: read the warning rate against the server frame-time histogram. Fix the slow frames that are yours; a rare warning at a steady 16.7 ms mean points at sleep jitter, but confirm it in the histogram first.

SERVER-LOG-STACKS-FEED-SLOW-FRAMES

The sleep-mode warning (SERVER-SLEEP-WARNING-COUNTS-SLOW-FRAMES above) feeds itself on a dedicated server while Log and Warning keep their default stack traces: each warning captures and writes a ~20-line stack inside the slow frame (measured 43-46 ms charged to UpdateNetworkTimeSystem), so the next frame runs 2+ steps and warns again. One idle run wrote 1648 of them (4.7 MB).

In the dedicated server process only, call Application.SetStackTraceLogType with StackTraceLogType.None for LogType.Log and LogType.Warning at boot; errors and exceptions keep their traces.
The warning stays: it is the long-frame symptom. Fix the frames it counts.

CONSOLE-DRIVER-LIVES-IN-THE-LOCAL-WORLD

A dedicated server's operator console (log feed, heartbeat, typed commands) moved off a MonoBehaviour Update into ECS could not live in the NetCode server world: a Relay-hosted bootstrap creates only the local world at process start, the server world arrives later from the Relay allocation, the session stop disposes it while the console still prints the shutdown, and a failed boot keeps the window open with no server world at all.

Put the driver in the local world the bootstrap creates first: a [DisableAutoCreation] managed system made with CreateSystemManaged, added to that world's SimulationSystemGroup when the console opens and removed with RemoveSystemFromUpdateList in the console's Dispose. Removal is deferred to the next sort, so it is safe from inside the system's own update (an exit command).
Create the server-world queries once when that world is ready and read them only while it IsCreated; the world owns and frees them. Operator-rate queries (typed commands) may stay per call.

PREDICTED-QUERY-BEFORE-THE-BUILD-READS-THE-FUTURE

A client standing up from prone flickered Standing/Prone/Standing over 4 frames (the hands jumped 72-96 cm) while the server stood once. The predicted stance system ran before this tick's physics world build and grew the capsule only if the last build was not newer than the tick it simulated. On the first tick replayed after a rollback the world still held the build of the newest predicted tick, so that guard refused a stand-up the server had accepted; the client stood a tick late and was corrected.

A predicted query that runs before this tick's build reads the PREVIOUS build. In a replay that build is the future, so any "world is not newer than this tick" guard fires on the client only.
Decide the outcome after this tick's build (after the physics build, before the character update) against the build stamped for this tick. Keep before the build only the restores the build itself needs.
A test that pins "a future build must not authorize the change" pins the bug. Rewrite it as "the replayed first tick decides like the server".
A collider resized after the build shows the new size at the old bounds for one tick; check same-tick queries by other systems before moving the decision.
Gate: a test that rewinds to a tick, replays it with a newer build present, and asserts the server's outcome. Found from a measured flicker plus a source read; the fix was compile-only when this was written.

Request channels and presentation edges

SILENT-DROP-CHANNEL-IS-BOUNDED-BY-RATE. A server that drops requests past a rate cap (8/s) without a reply breaks a client that re-asks. The client (a) re-asked every frame, bumping its request id past the reply in flight, (b) capped outstanding asks, which a dropped ask holds forever, and (c) restarted its scan at the front each update, so the tail was never asked: 9 of 220 tiles answered. Bound asks per second instead of outstanding count and resume the scan where it stopped: 220/220.
QUANTIZED-VALUE-EDGE-FIRES-PER-QUANTUM. A presentation effect edge-triggered on a replicated value fires once per quantization step: a hurt cry on every health drop played about 100 times in a row while drowning, because health replicates in 0.1 HP steps (226 drops). Drive the effect from a cadence over time (226 drops -> 3 cries) and test it with a slow continuous drain.
PREDICTED-INPUT-EDGE-CONSUMED-ONCE. A predicted system that consumes an input edge (an interact press) on every predicted tick replays it on each rollback: entering a vehicle ran twice. Consume a one-shot edge only on the first prediction of a tick (NetworkTime.IsFirstTimeFullyPredictingTick) or make the effect idempotent by state.
CLIENT-GHOST-CANNOT-HOLD-A-SERVER-ENTITY. A client ghost drew at the frame origin because its region membership was an Entity bound on the server; that Entity means nothing in the client world. Replicate a stable id (the frame id) and let each world resolve its own Entity and physics world index from it.

PREDICTED-CUE-RECONCILES-BY-RESULT - a server echo with the same event id can carry a different result

An owner's predicted impact cue skipped the server's echo whose shot number matched its own. When the local trace and the server trace disagreed (local: ground, server: body) the shooter saw a dirt puff and no body cue; the reverse left false blood and no hit marker. - Keep two tracks per event id: the predicted result (what was drawn, plus a pooled-effect handle with a generation so a reused slot is never faded by mistake) and the confirmed result. On the echo compare results: agree, redraw nothing; disagree, fade the predicted effect out fast (about 0.15 s) and draw the server's at the server's point. A predicted effect with no echo after about 1.5 x RTT + 0.1 s (clamped 0.15..1 s) fades out.
Sound and confirm markers play on the server's word only; the predicted marker is dim and silent. Count confirmed hits that showed a cue, so the share is measured before and after, not argued.

INTERPOLATE-THE-AIRBORNE-TICKS - presentation interpolation gated on "grounded" steps every jump

An owner view interpolated completed controller ticks only while Grounded was set. In the air the camera took the raw simulation-tick staircase: at 60 Hz sim and 144 fps render, one position every 2-3 frames. Switching interpolation off at takeoff and on at landing moved the view by one tick of delay each time, a visible snap at both ends of every jump.

Interpolate completed ticks in every motion state. The controller skips partial ticks either way, so the gate bought nothing.
Check the jump's per-frame speed spread (coefficient of variation) against walking. A staircase shows as a spread several times larger, and a toggle shows as a one-frame step at the transition.

DEATH-FALL-STARTS-FROM-THE-DRAWN-POSE - a killed body vanished, then reappeared lying elsewhere

The living view hid the frame the death flag arrived (predicted early on the owning client, at interpolated time on the others), but the server corpse ghost spawned later and its ragdoll started from a standing rest pose on the corpse's spot. Every viewer saw an empty gap, then a body in a different pose.

Start the ragdoll from the hidden view's last drawn bones on the frame it hides, in the viewer's frame, with the body's velocity and the killing push.
When the corpse ghost arrives, re-express the running bodies in its frame (same on-screen pose, clock carried on); a pin pulls the hips toward the server's rest spot, so the corpse and its loot stay authoritative.
Release a continued fall that never gets its corpse after a bounded window (2 s).
Measure it: ms between the hide and the first ragdoll frame (target 0), hips jump at the handoff (target under 5 cm).

LATIOS-BOOTSTRAP-STATIC-NEW — route world creation through LatiosClientServerBootstrap