Research Scientist, Hand Tracking & Manipulation

Location: New York, Toronto or Vancouver (on-site)

Type: Full time · Openings: 1

Salary: US$170K – $210K equity The role

Architect and train proprietary foundation models from scratch for 3D hand tracking and articulated pose estimation. This role targets a critical bottleneck in embodied AI: dexterous manipulation and hand-object interaction. You'll build in-house equivalents to leading 3D hand and mesh recovery architectures, plus robust interaction models for heavily occluded egocentric manipulation. You'll train on a large, continuous stream of proprietary ground-truth manipulation data.

What you'll do

- Design, implement and train state-of-the-art networks for 3D hand pose estimation, dense mesh recovery and kinematic tracking
- Scale multi-view and temporal architectures across multi-GPU clusters on massive datasets of hands in action
- Develop novel loss functions that enforce biomechanical constraints, temporal smoothness and physical plausibility
- Build models robust to motion blur, severe self-occlusion and rapid rotation in first-person manipulation
- Track objects through complex grasps, segment tools from hands, and map contact points and forces for downstream robotics models
- Rapidly prototype models for tactile-visual fusion,



fine-grained action segmentation and new hardware sensors
- Feed tracking outputs into pipelines that reason about contact surfaces and object affordances, bridging human video and robot control policies

What you bring

- Deep expertise in deep learning, 3D computer vision and articulated tracking / hand pose estimation
- Proven experience training large-scale vision models from scratch, beyond inference or fine-tuning existing checkpoints
- Strong grasp of parametric hand models (e.g., MANO), inverse kinematics and dense mesh estimation
- Mastery of PyTorch and deep learning scaling frameworks
- Experience curating multi-terabyte image and video training datasets
- Comfort in a fast-paced environment where priorities shift with new research or hardware

Strong signals

- First-author publications at CVPR, ICCV, ECCV or NeurIPS on 3D hand tracking, hand-object interaction, dexterous manipulation or human mesh recovery
- Work with egocentric manipulation datasets (e.g., Ego4D, Ego-Exo4D, DexYCB, EPIC-KITCHENS)
- Custom CUDA kernels for 3D operations, differentiable mesh rendering or collision/contact computation

Please note: Research Scientist roles require highly specific expertise. Apply to only one research role; you may also apply to non-research roles.

📌 Research Scientist, Hand Tracking & Manipulation - ON-SITE (Canada)
🏢 Targeted Talent
📍 Canada