VR IK and full-body tracking

Research Nothing here is built. This page records the research of 2026-10-02 and the decisions mltn has made so far, so the session that builds it starts from them.

The goal: one VR body for all of DigitalHeaven. A head and two controllers, plus any trackers, drive a DigitalHeaven avatar the same way in the engine, in BONELAB, in Beat Saber and in any game where we only get tracked points.

Decisions

Decision	Why
We write our own IK, clean-room. No FinalIK/VRIK dependency, and nobody reads FinalIK’s source or the local VRChat decompile	FinalIK is a paid RootMotion package; BeatMods’ redistribution of its DLL is a gray area
One shared IK, in Core. The engine, BONELAB, Beat Saber and the other mods all use it	DigitalHeaven is the standardizer. Nothing is written per game except a thin adapter
Crouching with three points is solver-driven: a hip hinge and a spine hunch, not an animation blend	mltn’s call
Trackers come only from the runtime: OpenXR first, SteamVR/OpenVR where OpenXR does not expose them	SlimeVR and similar products already appear in SteamVR as ordinary trackers. No brand-specific receivers
Full-body tracking in BONELAB is built on the same shared layer	Not a BONELAB-only feature
Calibration ownership is not decided yet	Before proposing one, the builder briefs mltn on how VRChat, Resonite, ChilloutVR and CustomAvatars calibrate, with “mostly automated” as the aim

Architecture

There are two layers in Core, both pure System.Numerics and both testable headless. This follows the shape of the mouth pipeline: shared logic in Core, with a thin host adapter at each end.

1. Solver. Pure math:

   • analytic two-bone arms, with a dynamic elbow pole and hysteresis
   • a shoulder model
   • a spine chain that spreads the bend across the spine and neck and hinges at the hips
   • foot geometry

   It outputs the existing HumanoidPose: rotations plus the hips position. Rotation only: bone lengths never change. A hand out of reach stops short, and the solver reports the error.

2. Body behavior. Decides what to ask the solver:

   • calibration
   • scale policy (by height or by arm span)
   • body yaw
   • posture (standing, crouching, prone)
   • the foot planner
   • the handoff to locomotion animation
   • tracker bookkeeping: dropout, fallback, freezing a lost tracker
   • per-region weights

   Its tuning values are client.vr.body.* preferences.

The seams:

• Input. Role-tagged tracker poses in play space, through one Core pose-source interface that every runtime reader implements. Plus the play-space floor, the avatar’s bone map and rest pose, and the user’s measurements.
• Host-owned hands. A per-hand flag. The hand pose becomes a hard end effector with no reach clamp. BONELAB’s physics hands, which hold items, and Beat Saber’s sabers both need this.
• Calibration results. Height, inseam and wingspan are exposed, so a host can set its own player measurements.
• Unity. A HumanoidPose-to-skeleton driver lives in DigitalHeaven.Unity and is shared by every Unity mod.

What other platforms do

Most of the “feel” lives in the behavior layer, not the solver.

Platform	Solver	Where it differs
VRChat (IK 2.0)	Unconfirmed: it ships FinalIK for creators but has never said the body uses it	IK 2.0’s improvements were mostly policy: a calibration saved once and remapped to every avatar, the choice of height versus arm span (fixing the arm-span scaling bug ended the “arm pull” complaints), Lock Hip / Lock Head, auto-footsteps, crouch and prone picked from head height, tracker prediction off by default
ChilloutVR	A modified FinalIK 1.9, stated publicly	Calibration UX, the locomotion setup
Resonite / Neos	A patched FinalIK port; its own solver is in progress	Calibration in two layers: one for the user, saved to the account, and one per avatar
Beat Saber CustomAvatars	FinalIK VRIK with patches	Its source (LGPL) reads as a list of what breaks on a VRIK-style solver: pelvis follow, the shoulder rest assumption, foot noise. It places the pelvis at 4/7 of eye height

Where the solver math itself shows is the shoulders and how the spine and neck share a bend. VRChat still gets complaints about both, and Resonite names both as open problems.

References

Reference	License	Use
V-Sekai RenIK	MIT	Design reference, not a dependency. It is maintained by hobbyists, has an open elbow-flip bug, and nothing ships on it. Worth learning from: the dynamic elbow pole, the spine influence curves, the foot-placement structure
SlimeVR Server	Apache-2.0 / MIT	The full-body stage: floor and skating correction, height calibration, auto proportions (LegTweaks)
FABRIK (Aristidou and Lasenby, 2011)	Paper	An option for the spine chain
AvatarPoser and other learned body estimators	Code MIT, training data non-commercial	Research only

The oracle idea. mltn would buy FinalIK and keep it in an isolated Unity sandbox that agents never open. A batchmode harness calls only VRIK’s public API with recorded tracker fixtures and writes the solved bones to JSON. Core tests then compare our solver against that output with tolerances. It is a yardstick, not a target to copy, and RenIK serves as a third reference.

BONELAB already has a body path

BONELAB ships a dormant 7-point (“hepta”) full-body path written by SLZ:

• OpenControllerRig.UpdateHeptaBody
• XRApi.Trackers, with ten roles
• the rig’s tracker* and tracked* transforms
• AssignTrackingPucks

All of it is switched off:

• The OpenXR vive-tracker feature is not enabled.
• trackedPelvis, trackedKnee* and trackedFoot* are null in the prefab.
• Nothing calls AssignTrackingPucks.

It is less of a tracker path than it looks. AssignTrackingPucks and RefreshBodyTrack copy Meta body tracking bones (XRApi.Body) into the tracker transforms, and the tracked spine (chest and pelvis) runs only while that body is tracking. With Vive trackers, SLZ’s solve reads the feet, knees, elbows and shoulders once their targets are wired and CaptureTrackedOffsets has run; on its own, a waist puck leaves the pelvis procedural. The tracked spine itself reads only transforms (the pelvis target, its puck and the head), so the BONELAB full-body tracking probe wires the feet as they are and opens the spine for the waist by passing the body gate for the length of the solve (FbtHips). It blends from SLZ’s own procedural pelvis by a weight and a dropout fade, and keeps the locomotion’s feet center where the head-driven body puts it. The blend, the fade and the yaw-only fallback are behavior-layer rules that belong in Core’s shared solver once it exists.

The BONELAB adapter now gates every tracker body write on successful calibration for the current rig/avatar. Clear calibration restores the saved targets and releases owned foot physics. Native torso capture uses a fixed Meta quaternion, so the adapter measures the pelvis/chest offset against SLZ’s procedural frame instead; recorded-pose tests reproduce the procedural pelvis. Hips remain off pending VR validation. Foot hold measures tracker rise from the standing calibration, planted legs stay kinematic, and collider clearance is applied before IK with all contacts preserved. This is a native-path experiment, not the shared solver described above. See the platform page for its six visible controls and remaining reach limitation.

The avatar’s art bones are copied from the physics rig in ArtRig.ArtOutputLateUpdate, so there is no IK pass on the avatar. The plan has two stages:

1. Visual. The Core solver writes the legs and hips in a postfix on ArtOutputLateUpdate. It is safe; the physics feet do not follow.
2. Native. Feed SLZ’s tracker transforms, wire the null fields and call SLZ’s own calibration, so the physics body follows the trackers.

LabFusion does not carry body poses, so remote players need a module of their own.

Gaps in the code today

• HumanoidPoseBone has no shoulder, upper-chest or toe slots. HumanoidPoseBones.Count is (int)RightFoot + 1, so a slot appended after RightFoot is silently left out of the count. The recommendation is to add the slots and accept the renumbering.
• HumanoidMeasures holds only the hips and legs. It needs to become a full rest body.
• DigitalHeaven.Unity and the BONELAB mod reference Core only. The swing-to-neutral math in Content’s HumanoidPoseRig has to move into Core.
• The engine’s VR path tracks the head only. There is no hand-pose plumbing yet.

Suggested order

1. Calibration and scale, arms with the reach clamp and elbow pole, and the head
2. Body yaw and the spine, including the solver-driven crouch
3. Feet and stepping
4. Trackers and full-body tracking, starting with BONELAB’s visual stage

Each stage is tested headless against recorded fixtures.

Open decisions

1. Who owns calibration, and how much of it is automatic. The comparison brief comes first.
2. For remote VR players, whether to replicate the solved bones, or the head, hands and trackers and solve on the receiver. The recommendation is the second.
3. The default measurement: height, or arm span for exact reach. The recommendation is height, with arm span measured during calibration.
4. Whether to buy FinalIK for the oracle.