Map Replication

A client never assumes it already has the map a server is running. Instead, the server advertises the current map’s identity and content hash, and the client either hash-skips straight to loading a copy it already has, or requests a chunked transfer of the pallet bytes over the network. This is protocol v19 (NetProtocol.Version) — the version that introduced it; the current wire version is higher.

Transport diagnostics

External hosts can opt into LiteNetLibTransport(diagnostics: writer) for bounded connection troubleshooting. The host-owned writer receives at most 32 lifecycle/socket events plus a limit notice, including dial endpoints, local ports, network errors and disconnect reason/socket codes. No packet payloads are logged; the default remains silent. Socket codes are evidence, not a diagnosis of firewall or local-network privacy settings.

Advertising the map

The server names its current map as a (name, palletHash, closureHash) triple everywhere a client needs to know what to load:

On join, it rides serverInfo as MapName, MapPalletHash and MapClosureHash — no separate round trip is needed before a fresh client can start resolving the map.
On a live map switch, the server broadcasts a MapLoad message (id 12) carrying the same triple to every already-connected session, so a client mid-session reloads to match without disconnecting.

Both hashes are 128-bit (ContentHash128), and nothing in this path ever truncates one back to 64.

MapName is always the installed map’s barcode — palletId:assets/path.dh-map — whether it was chosen with --map, switched into live, or simply the map the world opened on. It has to be, because it is the only thing that says which map of a multi-map pallet the server is simulating. The one world that names no asset is a genuinely content-less one, which falls back to the procedural gray-box map and advertises no content at all, so nothing is ever asked to open that name.

MapPalletHash is the map pallet’s own fingerprint — “which file” — and is what a transfer or a cache lookup keys on. It is not computed at map-switch time: every .pallet carries it in its fixed header, stamped by dh build (see PalletContentHash). Content-addressed, not name-addressed, so two maps that happen to share a name are never confused and a byte-for-byte-identical map recompiled elsewhere fingerprints the same.

MapClosureHash is “which content set”: the map name, the map pallet’s fingerprint and every dependency’s, combined in pallet-id order. That is the identity a client compares against the map it already has meshed, so an edit confined to a dependency still forces a reload where the map pallet’s own hash alone would not.

The fingerprint is a compile-time fact

A pallet’s fingerprint is computed once, while the compiler is already streaming every entry, and stored in the pallet’s fixed header. Serving a map therefore reads header bytes only, never the whole dependency closure (hundreds of megabytes on a large map), to learn these few values.

What is hashed, with XxHash128, in this order: the formula version; the pallet’s canonical text (the normalized manifest, its used-dependency ids, and every entry’s path, size and content checksum, in ordinal path order); then every entry’s stored bytes, again in ordinal path order. The build timestamp and the compiler version are deliberately excluded, so a pallet rebuilt from unchanged inputs fingerprints identically and every peer that has it hash-skips.

Storing the hash is a pallet format version bump, v1 → v2. There is no fallback rehash: a v1 pallet is refused by name — Pallet format v1 cannot be read by this build (expected v2) - recompile the pallet with 'dh build' — because silently re-deriving a fingerprint the format promises to carry is exactly the cost this change removes.

Hash-skip vs. chunked transfer

On receiving either advert, the client resolves the named map the same way in both cases:

Deterministic maps the client can build locally from its own content (the built-in core pallet, for instance) resolve immediately with no network round trip at all.
Content the client already has by that exact hash loads straight from where it is — a hash-skip. “Has” means its on-disk cache, a bundled pallet, or any pallet this machine’s own discovery found (the workspace’s compiled pallets on the desktop): a pallet whose header carries the advertised fingerprint is used in place, never copied. That is what makes a join to a server on the same machine — the embedded server, or a dedicated one reading the same workspace — transfer nothing at all. The match is on the fingerprint alone, never the file name or pallet id, so a local pallet with the same id and other bytes is ignored and the server’s is fetched.
Content the client does not have triggers a request: the client sends BlobRequest (id 13) naming the kind mapPallet and the content hash it needs, and the server answers by streaming the pallet as a sequence of reliable-ordered BlobChunk messages (id 14).

A chunked transfer carries the pallet bytes in fixed BlobChunkPayloadBytes (1024-byte) pieces over the reliable channel — small enough that a whole chunk packet, header included, stays under the engine’s 1200-byte unreliable-packet budget, even though the reliable channel itself could fragment a larger message. The client’s assembler accepts chunks as they arrive (out of a total chunk count carried in each chunk), so assembly never blocks the client’s poll loop.

Neither end ever holds a pallet whole. The server opens the pallet file as a stream and reads a chunk only when the transport has room for it: each poll it tops the peer up to server.net.blobChunkWindow (default 512) reliable packets waiting in the transport, so a pallet bigger than memory, or than one array, costs a window of packets and no more. One blob per peer moves at a time, in the order asked; a new map pallet asked for supersedes any other still queued for that peer. The client writes every chunk straight into its partial file (below). The total size in each chunk is 64-bit (totalBytes u64), so a 5.5 GB pallet travels the same way a 5 MB one does; an older peer that wrote or reads the 32-bit field is matched at its own width through the schema plan, and a server never streams a pallet past 2 GiB to a client whose schema predates the refusal below, telling it why instead.

A pallet the server holds but cannot read — the file locked by a build, deleted under it, cut short mid-stream — is answered with blobRefused (50): the kind, the hash, and the reason, with the server’s own directories taken out of it. The client ends a map join with that reason (“the server couldn’t send pallet ‘com.newblood.ultrakill’: …”) and an avatar as the placeholder, instead of waiting on chunks that are never coming. The server never drops the peer over it. A client too old to read blobRefused is disconnected with the same words for a map pallet, and simply never answered for anything else. Once every chunk has arrived, the client re-derives the fingerprint from the assembled container — header and entry bytes both — and checks it against the hash the file itself stores and against the one that was advertised, before trusting any of it. A stored fingerprint is only a claim until the bytes behind it are hashed.

The map’s dependency pallets ride along

A map is rarely one pallet. A pallet.dh may declare dependencies — a shared materials pallet, a texture pallet, a model pallet — and the map reads palletId:path references through them. Those pallets are replicated too, on this same pipeline.

Before each advert — ahead of serverInfo on a join, ahead of MapLoad on a live switch, on the same reliable-ordered channel — the server sends MapDependencies (id 22): the closure hash the set belongs to, then a (palletId, contentHash) pair per pallet, dependencies first. Binding the set to that hash is what keeps a set from a superseded switch off the map that superseded it; the client discards a set whose hash does not match the advert it is acting on.

The set is the transitive closure, not the declaration. A map that declares one materials pallet, which itself reads through a texture pallet, is advertised with both — a client that only received what the map named would still have a hole. The engine core pallet is deliberately never in it: every client launches with it.

Each entry is then resolved independently and in order, through the exact machinery the map’s own pallet uses:

A pallet whose content hash the client already has (cache or bundle) is a hash-skip — no bytes cross the wire for it, even if every other pallet in the set has to be fetched.
A pallet the client lacks is fetched with the same BlobRequest/BlobChunk pair the map is, one transfer at a time, at the same 1024-byte chunk pacing. BlobRequest names a content hash, which is why it needed no second message: a dependency was always expressible in it.

Only when the whole set — dependencies, then the map — is resolved does the client load anything. Delivery is all-or-nothing on purpose: the host installs the delivered pallets as the lookup that palletId: references resolve through and then opens the map, rather than discovering a missing pallet halfway into a load. The delivered pallets take precedence over any same-id pallet the client has locally, so what a client renders is the content the server’s map was authored against. The advertised MapName is what picks the map inside the pallet — a pallet holds several — and a name that does not resolve there, or is malformed, fails the load loudly rather than silently opening whichever map the pallet happens to list first.

Because the transfer is per-pallet and content-addressed, rebuilding one dependency re-transfers only that one: everything else in the set hash-skips.

This bumped NetProtocol.Version 35 → 36. A later bump, 39 → 40, widened every hash in this path from 64 to 128 bits and split the join-time advert into MapPalletHash and MapClosureHash. There is no compatibility shim for either bump.

The bulk channel

A chunk pipeline paced by a reliable-UDP window is the right shape for a map of a few megabytes and the wrong one for a texture library of 840 MB: LiteNetLib’s reliable window is 64 packets per channel, every chunk is one packet, and cutting the whole pallet into packets on the tick thread would block the first one from leaving. So a server that binds a real socket also opens a TCP listener on the same port number, and every peer that connects is told about it ahead of serverInfo by a BlobChannelOffer (id 33) carrying the port and a per-peer ticket — a random 128-bit value the server minted for that session, never anything the client asserted. The offer rides ahead of the handshake reply because the dependency set and the advert do too: the client must already know the pipe when the advert makes it ask.

A client that received an offer answers a not cached verdict by opening one TCP connection instead of sending BlobRequest: it names the ticket, the kind, the hash and the offset it already holds, and the server answers with the total and the bytes from there, then closes. The connection is served entirely on the thread pool — the tick thread never sees it — and the bytes land straight in the pallet store’s partial file (below), so a connection that drops is retried from the partial’s length and a join that is abandoned resumes on the next one from wherever it stopped. The server honors a ticket only once its peer is admitted (a request that races the admission over UDP waits briefly rather than being refused) and revokes it when the peer leaves. The server resolves map pallets and the members of avatar closures it holds; anything else is answered notFound.

The channel also runs backwards, for the one blob whose bytes start on a player’s machine. An upload opens with the magic DHBU instead of DHBL and carries the blob’s total where a fetch carries its offset; the server answers with the offset it already holds of that blob (a resume, like a fetch’s), the client sends the rest, and the server answers once more after the body is on disk. A server takes exactly one upload per peer at a time, and only the blob its tick thread just asked that peer for with BlobRequest: the ask registers the expectation (kind, hash and the bytes still left under server.avatarUploadMaxBytes) before it is sent, so the push it provokes can never arrive ahead of the permission to land. A total over that limit is answered tooLarge (status 4, with the limit) and the tick thread refuses the closure with the same notice the chunk path gives. The body lands on the thread pool and is verified there, the whole file re-hashed against the hash asked for, before the second answer; bytes that fail are discarded and answered refused. The tick thread only moves the finished, verified file into the store and reads its header. Only one transfer of a hash is ever open in the server’s store: an upload connection whose bytes another transfer is still writing waits for it to close, and one whose expectation is withdrawn (a swap, a leaver, a newer ask) stops writing at once, closing its file for whoever is asked next. See one sender per pallet. A server older than uploads reads DHBU as a stray connection and refuses it, and the client streams the same pallet as BlobChunks instead for the rest of that session — the frame version did not change, and neither did NetProtocol.Version.

server.net.blobConnectionsPerPeer (default 4) bounds how many of these one peer holds at once: a join can overlap a map fetch, an avatar fetch and an avatar upload.

A server that never opened the channel — the loopback compositions, a listen server whose socket is closed, an older build — sends no offer, and the client asks with BlobRequest exactly as before. Both pipes end in the same place: the partial is verified in place and moved into the cache.

Disk cache

A transfer lands on disk as it arrives, in a partial file beside the cache entry it will become (<hash>.partial), whichever pipe carries it: nothing the size of the pallet is allocated in memory, which is what lets a texture library land on a tablet at all. The verifier reads the partial back a piece at a time through the same digest walk the compiler used, and a commit is a rename. Closing a transfer keeps the partial, so the next attempt has a head start; only a failed verification discards it.

A verified transfer is written to a local cache keyed by its content hash, at <AppContext.BaseDirectory>/cache/maps/<hash>.pallet. A later join or switch that advertises the same hash — from this server or any other — hash-skips against that cached copy instead of re-requesting it. The cache is content-addressed, so nothing needs to track which server a given map came from.

It is the same store on both ends of every kind: a server installing an uploaded avatar files it under the engine cache directory’s avatars/<hash>.pallet, and because it is content-addressed the closure survives a restart — the first re-advert afterward hash-skips every member and installs without a byte on the wire.

The public DiskPalletContentStore, with IPalletContentStore and IPalletTransfer, lives in Core (DigitalHeaven.Core.Transport), not in the engine: it is the one store every host shares, the engine’s client and server and the game mods’ pallet transfers alike, so a fix to it reaches every game at once. Hosts inject the writable, engine-owned cache directory, the paths of bundled pallets, and optionally a live list of the pallet files their own discovery holds (the desktop host passes its workspace catalog). A pallet found that way is resolved where it is and never copied into the cache or deleted by a sweep. The desktop host keeps the install-relative path above; an external iOS host can supply a sandbox-writable directory without duplicating the store or changing the transfer protocol. Commit trusts the caller’s prior verification (NetClient verifies assembled transfers, and the server verifies every upload before committing it); it does not rehash the payload itself. A commit whose partial another transfer of the same hash already moved into place returns the installed path instead of throwing. TryBeginTransfer opens a transfer as the only writer of its hash, refused while any other transfer of that hash is open in the store; BeginTransfer opens one regardless and still counts as open, so the server’s exclusive opens see it.

Answering “do I have this?” costs a fixed-header read per candidate and nothing more: the file’s fingerprint was stamped at compile time and its bytes were verified against it when the transfer landed, so re-reading whole pallets to resolve a cache hit would put back exactly the cost the store exists to avoid. Locally bundled pallets are scanned the same way, and so are the pallets local discovery found: each discovered file’s fingerprint is remembered with its length and write time, so a later miss re-reads only a header that changed, and a pallet compiled mid-session counts on the next advert.

Fresh join vs. live switch

On a fresh join, the client’s ClientReady (which asks the server for a pawn) is gated until map resolution completes — hash-skip or full chunk transfer — so a slow transfer over a bad link can never race a pawn spawning into a map the client hasn’t loaded yet.
On a live switch, an already-connected, already-playing client reloads in place when it receives the MapLoad broadcast, running through the identical hash-skip/request/chunk path.

The server does not stall to switch

A switch is two phases on the server, for the same reason the client’s load is: the physics backend’s weld-and-BVH cook is one indivisible native call that takes seconds on a large map, and running it on the tick thread meant a listen server emitted no snapshots for its whole duration — a hitch every connected player saw.

The prepare runs on a worker (DevMapServer.PrepareSwitch): validate the target, reopen its pallet, resolve its looks, and cook its collision mesh — through the same on-disk cook cache the client’s load consults, so a map cooked once is a read afterward. It touches no world state at all. The old map stays installed and authoritative throughout, so the tick loop keeps stepping and keeps publishing snapshots.
The commit runs on the tick thread as one atomic action: uninstall the old map, install the cooked one, respawn pawns for a deliberate map switch, and only then update the advertised map content and broadcast MapLoad.

That ordering is the safety property. Nothing about the new map — no geometry, no respawn, no advert — is observable before the commit, so a client that joins or readies while the cook is running resolves the old map, whole and consistent, and can never be spawned into a world whose collision has not been installed. One switch prepares at a time; a request arriving while another is still cooking is refused rather than queued.

Embedded host vs. remote clients

The embedded loopback host (singleplayer, and the local player of a listen server) reloads its map in-process on a live switch, because it already holds the exact bytes the server is running — which is why a server broadcasting MapLoad explicitly skips it.

On a join it resolves the advert like anyone else. It has to: a client can switch servers at runtime, so the map it has meshed may be some other server’s, and the only honest way to know what the world it is joining is running is the content hash that world advertises. Joining the map already on screen hash-matches and costs no re-mesh (the boot map’s own pallet hash is seeded at startup for exactly that); anything else re-meshes, from the local cache when the bytes are already here and from a transfer when they are not. A client-side live switch clears that hash to zero — what is meshed came from in-process, not from any advert — so the next join always re-resolves rather than trusting a hash describing a map two switches ago.

Map hot reload

You do not have to restart the engine — or even leave the map — to see a map edit. Rebuild the map’s pallet with dh build while the engine is running and the loaded map reloads for everyone on the server.

Or rebuild anything the map is made of. A map’s materials, textures and models usually live in other pallets, which its pallet.dh declares as dependencies — and a pallet full of textures contains no map at all. Rebuilding one of those reloads the map just the same, because the reload is what re-reads them. The registry walks the declared dependency graph (transitively, and including the engine core pallet, which every map reads through whether or not it declares it), so a rebuild anywhere in it that the loaded map can reach is a rebuild of that map. The log names the pallet that actually moved.

A rebuild is a revision

A map load is the difference from what is standing, so a rebuild of the map you are in brings in only what the build changed. There is one load path, not a reload tier beside it: a first load is the difference from nothing, a change of map is the difference from another map, and a rebuild is the difference from the last build of the same one. What changed is decided by identity, never by position — a pallet entry by its path and checksum, an entity by its authored id, a geometry node by its manifest id (MapContentDiff).

The server revises its installation in place. Every part of the map’s collision — each geometry node, the brushes, the colliders no node owns, the spawn points — carries a fingerprint of what it is built from. A rebuild cooks and builds only the parts whose fingerprint moved and retires only what they replace, inside the same atomic seam a switch uses (World.ReviseMap). A one-object edit of a large map costs one object’s cook.
The generation stays. A rebuild is announced as a revision (MapRevision, id 44) under the generation the room already holds, so nothing map-scoped is retired: every replica keeps its id, a light the build moved is the same replica at its new position, the snapshots never stop, and the edit model is kept and rebased by identity — an object the build restated starts from its new file, every other edit is carried, and so is the selection.
The client swaps quietly. A client brings the revision in through the same job as any load, as the difference from the world on screen: no loading box, no gate on play, and the prediction world is revised object by object in the live world rather than rebuilt (past client.load.revisePartsMax changed parts it is staged whole instead, still swapping in one frame). Undo survives a rebuild that renumbered nothing it recorded against.
Nobody moves. A rebuild is a content refresh of the map you are standing in, not a change of venue, so pawns are not respawned (a map switch to a different map still respawns them) and spawned things stay in it. Geometry can move under a player; the character controller’s depenetration recovers them, and respawn is the escape hatch.
A client that cannot take a revision still gets the old full reload. Whether it can is read off its wire schema: a client whose schema lists mapRevision (id 44) takes revisions, and one whose schema does not is a build from before them. The generation is one number for the whole room, so while any admitted client cannot, a rebuild is announced the way a change of map is — a generation bump and a MapLoad — for everybody. The capability is the schema itself, so no message says it and nothing bumped the protocol.

(The one content swap the engine performed in place before any of this, the lightmap atlas, is a bake product rather than a pallet asset, and is still guarded by its own UV hash.)

Nothing locks the pallet. The engine holds a map’s compiled .pallet open for the whole life of that map, so the compiler publishes a rebuild by writing a sibling scratch file and then replacing the target, rather than truncating it in place. Every pallet reader opens with FileShare.Delete so that replace is permitted. A reader holding the previous build keeps seeing a consistent view of it (the old file is unlinked, not overwritten) until it reopens.

The server is the authority. Detection rides the same pallet watchers that keep the maps list current: when a pallet the currently loaded map is made of is republished, the embedded host brings the map up through the identical path a typed map command uses — client load as a revision, then the server’s authoritative revision, then a MapRevision advert to every remote client (which hash-skip or stream the new bytes as above). Clients never watch their own copy of a pallet; if they did they would drift out of sync with the server’s map.

Debounced and coalesced. A compiler writes in bursts, so notifications are collapsed into one rebuild; a rebuild that lands while another is still coming in becomes a single follow-up, never a queue. A dh build that republishes a map and its dependencies is one burst, so it is also one rebuild.
Atomic. The server independently loads and validates the rebuilt map before touching anything. A broken rebuild leaves the previous map fully installed, broadcasts nothing, and reports the error — clients are never left on a half-installed map.
Everyone is told what changed. A quiet swap opens no box, so the one thing it says is a ClientNotice toast the server broadcasts to every session once the rebuild lands — Map rebuilt: changed entry dev.pitr:models/room.glb, entity streetLamp, with a count past the first few of each — or Map rebuild failed: <reason>. These appear in the DigitalHeaven overlay whether or not it is open.

Pass --noMapReload to turn this off. It applies only while this client is the server: a client that launched into a remote one — or left its own for one — takes its map from that server, and a dedicated server watches no map registry.

The same pipeline carries more than pallets

Protocol v70 generalized this transfer into a blob pipeline. Ask by (kind, contentHash), receive uniform chunks, verify the whole: the pallet stream above is the kind mapPallet, and five more kinds ride the identical machinery — two operator-facing, one that runs it backwards, and two an avatar picker asks for.

mapCatalog — every map the server could switch to, as the rows a picker draws (barcode, pallet id, pallet display name, built-in flag, thumbnail fingerprint). Asked for with no hash, because a client cannot fingerprint a list it has never seen, and answered under whatever fingerprint the bytes have.
mapThumbnail — one map’s picture, named by the fingerprint its catalog row carried. Two maps sharing a picture cost one transfer.
avatarCatalog — every avatar the server already holds, as the rows the avatar picker draws: its own pallets’ avatars plus the closures players uploaded to it, deduped by barcode. Asked for with no hash, like the map catalog.
avatarThumbnail — one avatar’s picture, named by the fingerprint its catalog row carried.
avatarPallet — one pallet of a player’s avatar closure, and the one kind that travels both ways. A wearer offers its closure with wornPallets (32) and the server sends the BlobRequest, which the client answers with BlobChunk — the direction of this whole page, reversed. The installed closure is then served back out the ordinary way to every peer that has to draw that player.

The catalog kinds are computed on the server, by exactly the code a local player’s own list is computed by: one IAssetCatalogSource over a pallet root set, with local implementations over this process’s own maps and avatars and a synced one mirroring what a server sent. The picker, its categories and its filtering never learn which of the two answered. A catalog row carries what it is — AssetCatalogKind is map or avatar — so one codec, one blob path and one thumbnail path serve both, and AssetCatalogBlobSource is the single place a request is routed to the half that can answer it.

The two map kinds are gated on world authority — the host peer or an identity in server.operators, the same gate the map verb passes. A request from anyone else is refused in silence: nothing is sent back and the server logs it at debug, so a probing client learns nothing about the maps a server holds. One transfer per kind is in flight at a time. The three avatar kinds are deliberately ungated: avatarPallet in both directions, because an avatar is content other players have to draw and a server that hid it would just draw boxes; avatarCatalog and avatarThumbnail because what a server will let you wear is every admitted player’s business, not an operator’s secret. They are sibling kinds rather than widened map ones for exactly this reason — the authority gate is read from the kind alone, so there is no second place where a map’s privacy could be spelled onto an avatar’s list.

The client’s mirror holds thumbnails in memory for the session only — nothing about another machine’s pictures belongs on this disk. client.ui.remoteThumbnailCacheBytes bounds it (least-recently-drawn evicted first), client.ui.remoteThumbnailRequests bounds how many asks are outstanding at once, and client.ui.remoteThumbnailTimeout is how long an unanswered ask holds its slot.

Wire summary

Message id	Direction	Reliability	Carries
`MapLoad` (12)	Server → client	Reliable	`(name, palletHash, closureHash)` — broadcast on a live map switch; the join-time equivalent rides `serverInfo` instead
`MapDependencies` (22)	Server → client	Reliable, ordered	The advertised set’s closure hash, then `(palletId, contentHash)` per dependency pallet, dependencies first — sent immediately before the advert it describes
`WornPallets` (32)	Both ways	Reliable, ordered	An avatar barcode, then `(palletId, contentHash)` per pallet of its closure, dependencies first — an offer going up, an advert coming back down
`BlobRequest` (13)	Both ways	Reliable	A kind byte, then the content hash needed within that kind — a map pallet (the map’s own, or one of the dependency pallets), the server’s map or avatar catalog, one map or avatar thumbnail, or (server → client) one pallet of a worn avatar’s closure
`BlobChunk` (14)	Both ways	Reliable, ordered	The kind and hash the transfer belongs to, the blob’s 64-bit total, plus one `1024`-byte slice of the blob, its index and the transfer’s total chunk count — client → server only for `avatarPallet`
`BlobRefused` (50)	Server → client	Reliable, ordered	The kind and hash of a `BlobRequest` the server could not read, and why — the client ends that transfer with the reason
`BlobChannelOffer` (33)	Server → client	Reliable, ordered	The bulk channel’s TCP port and this peer’s ticket on it — sent ahead of `serverInfo` by a server that opened the channel; a client that got one fetches map and avatar pallets over TCP from the offset it already holds, and answers the server’s avatar `BlobRequest` by uploading over it
`ClientNotice` (16)	Server → client	Reliable	`(severity, message)` — an operator-facing overlay toast, broadcast to every session including the local host

See Engine Assets & Audio for what happens to the pallet bytes once they’re loaded, and Engine Overview → Listen Server for how this path is shared with a player-hosted server.