PRIYN VR: immersive hands from a normal webcam
The hands-first alpha now combines finger tracking, live rooms, proximity voice, and physical interaction inside a shared 3D world.
PRIYN VR now renders anatomically assigned skeletal gloves from local tracking and adds an optional phone-and-watch motion path alongside spatial rooms and touch interactions.
Notes
The goal is embodied presence without a headset: a normal camera observes movement locally and drives a character inside a three-dimensional Priyn space.
This first build intentionally shows a diagnostic skeleton. It lets us see orientation mistakes, hand identity swaps, jitter, and latency instead of hiding them inside a finished character.
Keyboard locomotion and webcam motion are separate. WASD moves the character through the world while shoulders, elbows, wrists, palms, and fingers follow the person in front of the camera.
Small multiplayer rooms now use WebRTC with a Priyn-managed TURN relay whenever a direct connection cannot be established. Camera video is never part of the avatar stream.
The existing PRIYN app now has an optional right-phone and left-watch motion path. Camera landmarks still own positions and fingers; fresh inertial data only helps rapid wrist movement and brief occlusion, so the keyboard and camera experience remains complete without wearables.
The normal view now shows two immersive hand meshes and short forearm cuffs rather than a full-body stick figure. A thumb-and-index pinch can grab test objects in the world.
Alpha 0.4 adds authenticated six-person rooms, remote hands, floating name tags, HRTF proximity voice, adaptive pose updates, collision boundaries, and reliable push impulses.
Alpha 0.5 adds connection-aware host election and seamless host migration. The first entrant starts as host; a consistently stronger peer can take over, and host departure promotes the best remaining participant without closing the room.
Alpha 0.6 locks camera movement to local embodiment: walking sideways or toward the webcam no longer slides the player through the world. Jump height and torso articulation remain live, while uncertain joints hold their last trusted pose and then settle to neutral.
Alpha 0.7 separates calibration from play with a true first-person camera, mouse look, and an F-key view switch. Brief occlusion uses velocity prediction; hidden legs follow a deterministic standing, torso-turn, gait, crouch, and jump prior rather than unstable landmark guesses.
Alpha 0.8 makes the camera optional. Mouse-only users enter a default first-person playground without a camera request, while tracked users gain explicit fist/boxing-guard recognition and calibrated head nod/tilt layered beneath mouse or phone look.
Alpha 0.9 freezes a hand at its last trusted pose when rapid movement leaves the frame, separates landmark diagnostics from skinned hands, and constrains calibrated body proportions so vigorous movement cannot repeatedly stretch, shrink, or hover the avatar.
Alpha 0.10 anchors articulated hands and forearms inside the first-person view, provides neutral fallbacks when hands are unseen, hides every overlay during immersion, clears nearby view-blocking props, and makes keyboard travel drive matching forward, backward, and strafe leg animation.
Alpha 0.11 corrects the first-person horizontal hand projection so physical left and right hand motion appears on the matching side while preserving finger and palm tracking.
Alpha 0.12 replaces independently frozen limb points with fixed-length arm and leg chains, reconstructs missing elbows and knees from trusted neighbors, carries hidden feet with visible knees, and adds lightweight hand identity, spike, and finger-length stabilization.
Alpha 0.13 enters first person level and facing forward, teaches keyboard movement with a brief overlay, preserves play look separately from calibration, and separates stable eye-line tilt from eye-to-mouth nod so nodding produces pitch instead of noisy camera roll.
Alpha 0.14 separates first-person wrist placement from hand orientation: outward motion stays outward, hand anatomy flips locally around its wrist, and palm-facing detection remains physically correct.
Alpha 0.15 fuses pose-wrist depth with palm-to-body apparent scale so reaching toward the webcam extends the hand from the torso, adds pose-guided occlusion fallback, and replaces normal-view blobs with solid articulated procedural hands.
Alpha 0.16 corrects the vertical head-pitch direction and records a primary-source research roadmap based on Apple Vision Pro, ARKit, MediaPipe Holistic, monocular hand reconstruction, and sparse-sensor avatar motion.
Alpha 0.17 corrects MediaPipe's mirrored handedness assumption and first-person palm orientation, replaces debug geometry with hierarchical Valve skeletal gloves, and adds timestamped phone/watch sensor streaming with a local, latest-only fusion bridge.
Alpha 0.18 hardens the production room path: CSP-safe no-store HTML, the self-hosted Socket.IO client, live data-channel diagnostics, reliable peer hello checks, and verified bidirectional two-browser pose/event traffic with seamless host departure.
Checks
- On-device body and two-hand landmark inference
- Palm-front, palm-back, and edge-on diagnostics
- Adaptive motion smoothing and hand-crossing correction
- Keyboard movement with a following world camera
- Accuracy benchmark before avatar or multiplayer work
- Downloaded, license-recorded Valve and WebXR hand assets
- Pinch-to-grab interaction prototype
Implementation Flow
- 01 Camera
- 02 Local landmarks
- 03 Calibrated skeleton
- 04 Keyboard locomotion
- 05 Avatar next
