Papers
Topics
Authors
Recent
Search
2000 character limit reached

From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans

Published 8 Sep 2025 in cs.GR and cs.CV | (2509.06607v1)

Abstract: Great progress has been made in estimating 3D human pose and shape from images and video by training neural networks to directly regress the parameters of parametric human models like SMPL. However, existing body models have simplified kinematic structures that do not correspond to the true joint locations and articulations in the human skeletal system, limiting their potential use in biomechanics. On the other hand, methods for estimating biomechanically accurate skeletal motion typically rely on complex motion capture systems and expensive optimization methods. What is needed is a parametric 3D human model with a biomechanically accurate skeletal structure that can be easily posed. To that end, we develop SKEL, which re-rigs the SMPL body model with a biomechanics skeleton. To enable this, we need training data of skeletons inside SMPL meshes in diverse poses. We build such a dataset by optimizing biomechanically accurate skeletons inside SMPL meshes from AMASS sequences. We then learn a regressor from SMPL mesh vertices to the optimized joint locations and bone rotations. Finally, we re-parametrize the SMPL mesh with the new kinematic parameters. The resulting SKEL model is animatable like SMPL but with fewer, and biomechanically-realistic, degrees of freedom. We show that SKEL has more biomechanically accurate joint locations than SMPL, and the bones fit inside the body surface better than previous methods. By fitting SKEL to SMPL meshes we are able to "upgrade" existing human pose and shape datasets to include biomechanical parameters. SKEL provides a new tool to enable biomechanics in the wild, while also providing vision and graphics researchers with a better constrained and more realistic model of human articulation. The model, code, and data are available for research at https://skel.is.tue.mpg.de..

Summary

  • The paper introduces SKEL, a model that integrates biomechanical skeletons with SMPL, reducing joint positioning errors to sub-centimeter levels.
  • The paper proposes a novel methodology that leverages the BioAMASS dataset and BSM for anatomically accurate kinematic representations in diverse human poses.
  • The paper demonstrates practical implications for motion capture enhancements and clinical biomechanics through upgraded datasets and realistic 3D digital human models.

From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans

Introduction

The paper "From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans" (2509.06607) presents an innovative approach to creating 3D human body models that accurately represent the biomechanics of human skeletons. The authors introduce SKEL, a parametric model that integrates biomechanical skeleton structures with the widely used SMPL model to improve anatomical realism in applications across vision, graphics, and biomechanical research. This integration addresses the limitations of existing models by providing a more anatomically consistent representation of human joints and articulations, which is crucial for applications in disease diagnosis and realistic animation.

Methodology

Biomechanical Skeleton Model (BSM)

Central to the creation of SKEL is the Biomechanical Skeleton Model (BSM), which offers a more realistic kinematic representation compared to traditional models like SMPL. BSM incorporates anatomical constraints on joint movements, such as the constant curvature model for the spine and detailed modeling of the shoulder blades and forearm pronation. These adaptations ensure that the skeletal structure within the body meshes accurately reflects real-world human anatomy.

SKEL Model Construction

To create SKEL, the authors map biomechanical skeletons onto existing SMPL body models, regressing anatomical joint locations from the posed SMPL mesh vertices. This involves optimizing the placement and orientation of bones within the 3D body space, using data captured from diverse poses in the AMASS dataset. The SKEL model integrates both body surface and skeleton rigging in a unified kinematic tree driven by biomechanical pose parameters, enabling synchronized animation of both skin and bone structures. Figure 1

Figure 1

Figure 1

Figure 1

Figure 1

Figure 1

Figure 1

Figure 1

Figure 1

Figure 1

Figure 1

Figure 1

Figure 1: A visual representation of SMPL meshes from the Total Capture dataset [totalcapture].

BioAMASS Dataset

The creation of the BioAMASS dataset is instrumental in developing SKEL. BioAMASS comprises paired datasets of SMPL body meshes and optimized skeletal structures across various poses. Using a combination of synthetic markers and BSM alignment procedures, this dataset enables learning the regressors needed to morph SMPL meshes into anatomically accurate skeletal models. The paired data allows for training models that predict anatomical joint locations from 3D body scans, thereby enhancing biomechanical applications.

Experimental Results

Joint Accuracy and Fit

The paper reports notable improvements in anatomical joint positioning within the SKEL model compared to the SMPL baseline. On evaluating datasets, SKEL demonstrates superior alignment of joints such as the humerus and femur, with errors reduced to sub-centimeter levels compared to pseudo-ground-truth BSM data. This accuracy is highlighted through comparisons of femur and tibia joint locations, emphasizing SKEL's biomechanical consistency.

Applications and Upgrades

By fitting SKEL to SMPL datasets like 3DPW and BEDLAM, the authors effectively upgrade these datasets with biomechanical parameters. This enhances the datasets' applicability in biomechanics by providing more realistic skeletal configurations for existing motion capture data. SKEL's ability to infer skeletal structures from SMPL meshes allows for novel applications in visualizing mocap data and diagnosing biomechanical conditions.

Implications and Future Work

The integration of biomechanical accuracy in 3D human modeling has significant implications. It enables more precise analysis and visualization tools for biomechanics, expanding the use of parametric models like SKEL in fields such as sports medicine and ergonomic design. Future developments may focus on enhancing SKEL's pose-dependent deformations and integrating muscle and skin dynamics to further refine the model's anatomical fidelity. The authors also suggest leveraging SKEL's framework with new biomechanical models for clinical validation and improved datasets.

Conclusion

The "From Skin to Skeleton" paper presents a significant advancement in 3D digital human modeling by combining the anatomical realism of biomechanical skeletons with the established SMPL framework. SKEL emerges as a powerful tool for both vision and biomechanics communities, offering a more constrained and realistic model of human articulation. As research continues, SKEL lays the foundation for integrating sophisticated biomechanical data with parametric body models, enhancing the accuracy and utility of digital human representations in diverse applications.

Paper to Video (Beta)

To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video.

All Videos Create Your Own

Whiteboard

Explain it Like I'm 14

What this paper is about (brief overview)

This paper builds a new kind of 3D digital human called SKEL. Think of a digital human as a puppet: the “skin” is the outside surface you see, and the “skeleton” is the inner structure that makes it move. Many popular 3D body models today have great-looking skin but very simplified skeletons. That’s fine for animation, but not accurate enough for science and medicine. SKEL combines a realistic outside (skin) with a more anatomically correct inside (skeleton), so the body moves more like a real human.

What the researchers wanted to find out

In simple terms, they asked:

  • Can we put a biomechanically accurate skeleton inside a popular 3D body model so both move together correctly?
  • Can we learn where real joints should be inside the body surface (like where the hip joint actually sits under the skin)?
  • Can we make a model that is easy to use like existing ones, but with motions that match real human anatomy?

How they did it (methods explained simply)

To explain the approach, imagine fitting a good-quality costume (the skin) onto a mannequin with realistic bones (the skeleton) so they line up and bend correctly.

Here’s the process they followed:

  • Built a realistic skeleton model (called BSM): This skeleton uses joint rules from biomechanics—how the spine curves, how shoulder blades slide over ribs, and how the forearm twists by the radius rotating around the ulna—rather than just using simple “ball joints.”
  • Created training data that pairs skin and skeleton (BioAMASS): They took many moving 3D body surfaces from a big dataset (AMASS, based on the SMPL body model) and placed virtual “motion-capture markers” on them (like dots you’d stick to someone’s skin). Then they used a tool called AddBiomechanics to fit the realistic skeleton (BSM) inside each moving body, matching those dots. This produced lots of examples where both the skin and the skeleton positions are known at the same time.
  • Smoothed out distances between bones and skin: Because people have different body shapes (for example, more body fat means bones are farther from the skin), they used another tool (OSSO) to estimate how far markers should sit from bones for each person. This kept the bones from being “stretched” unrealistically during fitting.
  • Learned to “see” joints from the skin: With the paired examples, they trained a regressor (a simple prediction model) that looks at the skin’s 3D shape and predicts the true anatomical joint locations and bone rotations inside.
  • Re-rigged the body model to make SKEL: They re-wired the popular SMPL body so its skin and the new skeleton move together, using biomechanical pose controls. SKEL keeps SMPL’s flexibility in body shape (tall, short, slim, etc.) but uses more realistic joint motions and fewer, more meaningful movement controls (degrees of freedom).

Key ideas in everyday terms:

  • “Rigging” is like setting up the hinges in a puppet so they bend the right way.
  • “Degrees of freedom” are the ways a joint can move (like a hinge that bends in one direction versus a ball joint that rotates in many).
  • “Regression” here means learning a rule that maps from the outside (skin) to the inside (joint positions).

What they found and why it matters

Main results:

  • More accurate joints: SKEL’s joint locations are closer to where real human joints are compared with the old model (SMPL). This makes analyses (like how the knee moves) more trustworthy.
  • Bones fit inside better: The skeleton fits properly inside the skin without poking through or being misaligned, and the joints (like shoulders and forearms) move in more realistic ways (e.g., shoulder blades sliding on the ribs, forearm twisting correctly).
  • “Upgrading” old datasets: Because many datasets already use SMPL, they can fit SKEL to those and add biomechanical details—so existing videos or animations can now include realistic skeleton motion without re-capturing anything.
  • Dual use: It works both ways—you can put a skeleton into a skin, or put a skin onto a skeleton—helpful for visualization and research.

Why this is important:

  • Better science from everyday videos: Researchers in biomechanics can use more anatomically faithful skeleton data from regular video-based methods, not just expensive lab motion capture.
  • Improved training for AI: Joint detectors or motion estimators can be trained on more realistic joint positions, likely improving accuracy in the wild.
  • More realistic animation and analysis: Game, film, and medical applications benefit from motions that respect how real bodies move.

What this could lead to (implications)

SKEL brings “biomechanics in the wild” closer to reality. That means:

  • Coaches, clinicians, and researchers could analyze movement more accurately using normal footage.
  • Computer vision and graphics systems get a better, more realistic human model, reducing guesswork and errors.
  • Future tools could directly predict biomechanical parameters from video, helping with injury prevention, rehab, sports performance, ergonomics, and realistic animation.

In short, SKEL is like giving a high-quality puppet a proper set of bones and hinges, so it moves the way a real human does—making both science and storytelling more accurate and powerful.

Ai Sparkles Streamline Icon: https://streamlinehq.com View Paper Prompt  List Streamline Icon: https://streamlinehq.com View All Prompts 

Knowledge Gaps

Knowledge gaps, limitations, and open questions

Below is a single, concrete list of what remains missing, uncertain, or unexplored in the paper, targeted to guide future research.

  • Lack of validation against true anatomical ground truth: all “ground truth” skeletons come from AddBiomechanics fits to synthetic markers on SMPL (i.e., pseudo-ground truth). There is no quantitative validation against radiological or instrumented datasets where bone positions/rotations are known, especially in motion.
  • Dependence on SMPL accuracy and bias: SKEL’s learning and evaluation pipelines assume accurate SMPL fits. Errors and biases in SMPL reconstructions (e.g., due to clothing, occlusion, or unusual body shapes) propagate directly into SKEL joint regressors and skeleton placement, but this sensitivity is unquantified.
  • Pose-dependent deformations are not learned for SKEL: SKEL inherits SMPL’s pose blend shapes via heuristic DOF mapping, and falls back to plain LBS where no SMPL equivalent exists, leading to visible artifacts in extreme poses. A SKEL-specific learned pose deformation model is left for future work.
  • Subject-specific bone torsion and geometry are not modeled: the bone rotation around the bone axis (R_i^{base}) is learned as a fixed consensus across the dataset, not individualized (e.g., femoral anteversion, tibial torsion). Bone meshes are scaled along axes but do not capture subject-specific shape/curvature variations.
  • Knee biomechanics are simplified: although BSM uses a more realistic model (Walker knee), SKEL applies a simplified hinge knee for rigging. The impact of this simplification on downstream biomechanical analyses (e.g., joint contact, kinematics) is not quantified, and replacing it with a realistic knee in SKEL is left open.
  • Spine and scapula models are generic, not personalized: the constant-curvature spine (three sections) and scapulothoracic ellipsoid are parameterized generically. How to personalize these models per subject (e.g., thorax shape, vertebral geometry) and validate against measured kinematics remains open.
  • Forearm supination/pronation axis personalization is missing: the single-DoF model rotates the radius around an axis defined by ulna/humerus points, but inter-subject variability in the forearm rotation axis is not modeled or validated.
  • ROM constraints are hand-specified and not individualized: ranges of motion for shoulder blades, spine, knee, etc., are generic. Methods to learn or calibrate subject-specific ROM (including pathological cases) and enforce them during posing are not explored.
  • OSSO-derived marker offsets may be biased by lying-pose training: personalized offsets for soft markers rely on OSSO predictions in a supine pose. The validity of those offsets for diverse standing/dynamic poses, and their sensitivity to OSSO errors, is not analyzed.
  • Uniform-density assumption for estimating height/weight from SMPL is unvalidated: body mass properties are inferred from SMPL shape via uniform density, then used to regularize bone scales in AddBiomechanics. The accuracy of this assumption across body types (e.g., high BMI, muscular builds) is unknown.
  • Soft tissue motion is not modeled: markers are assumed rigidly attached or lightly down-weighted (“soft markers”), but no explicit model of soft tissue artifact is included. How soft tissue motion affects joint regressors and skeleton placement is not quantified.
  • No explicit guarantees of bone–skin non-interpenetration across poses: while qualitative results suggest improved bone fit inside the surface, there is no collision analysis or guarantees that bones do not penetrate skin in extreme or rare poses.
  • Limited anatomical scope: hands, fingers, feet, and complex joint models (e.g., subtalar, mid-foot, detailed wrist) are not described or evaluated. Extending SKEL to more distal segments with anatomically realistic DOFs remains open.
  • Dataset diversity and generalization: BioAMASS is built from AMASS (mostly adult subjects, specific motion regimes). Generalization to pediatric, geriatric, high-BMI, and pathological anatomies, and to highly dynamic sports motions, is not addressed.
  • Direct regression of SKEL parameters from images/video is not demonstrated: the paper proposes upgrading datasets by fitting SKEL to SMPL but does not train or evaluate networks that regress SKEL q (and shape) directly from monocular or multi-view inputs.
  • Uncertainty quantification is missing: regressors from skin to joints (and from SMPL to SKEL) have no uncertainty estimates. Methods for propagating and reporting confidence in anatomical joint locations and bone orientations are needed.
  • Interoperability with physics-based simulation (OpenSim) not evaluated: although BSM is built in OpenSim, it is unclear how SKEL parameters map to dynamic simulations (e.g., inverse dynamics, muscle analyses), including numerical stability and force/torque consistency.
  • Computational performance and scalability are not reported: the runtime and resource requirements for fitting SKEL to SMPL meshes (vertex-to-vertex optimization) and for training/use of regressors at scale (e.g., upgrading large datasets) are unspecified.
  • Evaluation breadth is limited: most metrics reported are marker fitting MAE and anatomical joint regression errors (partially truncated). No quantitative comparison of skeleton–skin alignment quality versus OSSO/BOSS under varied shapes/poses or ablation studies (e.g., with/without OSSO offsets) is provided.
  • Potential failure modes are not cataloged: cases with bone over-stretching despite personalized offsets, extreme ROM violations, spine/scapula misalignments, or forearm axis misplacement are not systematically analyzed.
  • Mapping between SKEL and SMPL DOFs is ad hoc: the DOF correspondence used to transfer SMPL pose correctives may cause unintended deformations. Formalizing the mapping and learning it end-to-end (or learning SKEL-native pose correctives) is an open direction.
  • Ethical and licensing constraints for medical validation are unaddressed: using radiological or surgical datasets to validate bones-in-motion will require careful handling; plans and protocols for such validation are not discussed.
Ai Sparkles Streamline Icon: https://streamlinehq.com View Paper Prompt  List Streamline Icon: https://streamlinehq.com View All Prompts 

Practical Applications

Overview

This paper introduces SKEL, a parametric 3D human model that synchronizes an anatomically realistic skeleton with a statistically learned body surface (SMPL). It also presents BioAMASS, a paired dataset of SMPL meshes and biomechanical skeletons, and a learned regressor that maps skin vertices to anatomical joint locations and bone rotations. These advances enable “biomechanics in the wild” by upgrading existing SMPL-based datasets and workflows to output biomechanically meaningful parameters and by providing a co-rigged skin–skeleton model with realistic degrees of freedom (spine curvature, scapular sliding, forearm pronation/supination).

Below are practical applications grouped by deployment horizon.

Immediate Applications

The following applications can be built and piloted now using the released SKEL model, BioAMASS data, and the described workflows.

  • Sector: Software/Graphics (VFX, gaming, AR/VR)
    • Action: Integrate SKEL as a rigging option for SMPL assets to improve shoulder, spine, and forearm articulation in avatars and digital humans.
    • Workflow: Import SMPL assets → fit SKEL to the SMPL mesh (minimize vertex-to-vertex error) → export co-rigged skin+skeletal mesh to DCC tools (Blender/Maya) or engines (Unreal/Unity).
    • Potential tools/products: “SMPL→SKEL” converter plugin, SKEL FBX exporter with anatomical DOFs.
    • Assumptions/Dependencies: Accurate SMPL fits to input motion; minor pose-corrective artifacts in extreme poses until SKEL-specific pose correctives are learned.
  • Sector: Motion Capture/Markerless Biomechanics (industry & academia)
    • Action: Upgrade existing SMPL-based datasets (e.g., 3DPW, BEDLAM) and markerless pipelines (e.g., OpenCap) to output biomechanical joint centers and rotations via SKEL.
    • Workflow: Video → SMPL regression → SKEL fitting → extract anatomical joint locations/angles; optionally use BioAMASS-trained regressor to directly estimate joints from SMPL vertices.
    • Potential tools/products: Add-on for OpenCap/OpenSim that ingests SMPL and returns SKEL-based biomechanical parameters.
    • Assumptions/Dependencies: Pseudo-ground-truth from AddBiomechanics; accuracy depends on SMPL regression quality and multi-view coverage.
  • Sector: Computer Vision (academia/industry)
    • Action: Train 2D/3D joint detectors and pose-estimation models with anatomically correct labels derived from SKEL instead of artist-defined SMPL joints.
    • Workflow: Use BioAMASS regressors to compute anatomical joints from SMPL meshes → project to image space → retrain detectors.
    • Potential tools/products: Improved joint detection datasets; benchmark suite comparing “SMPL joints” vs “SKEL joints.”
    • Assumptions/Dependencies: Domain shift from training data (AMASS-derived) to in-the-wild images; detector performance hinges on label consistency.
  • Sector: Healthcare/Sports Science (visualization, patient communication)
    • Action: Add a plausible skin to biomechanical skeletons fitted from mocap for better visualization in clinics, sports labs, and rehabilitation sessions.
    • Workflow: Mocap → OpenSim/AddBiomechanics fit → SKEL skin generation constrained by subject’s weight/shape.
    • Potential tools/products: Clinical viewer showing synchronized skin+skeleton motion (with scapula sliding and forearm pronation).
    • Assumptions/Dependencies: Skin is plausible but not diagnostic; requires basic anthropometric inputs (height, weight).
  • Sector: Education (anatomy, biomechanics)
    • Action: Interactive teaching modules illustrating realistic DOFs (constant-curvature spine, scapulothoracic motion, pronation/supination) within a skin envelope.
    • Workflow: SKEL model embedded in web apps/VR experiences for anatomy courses and coaching certifications.
    • Potential tools/products: SKEL-based educational content; anatomically accurate avatar demos.
    • Assumptions/Dependencies: Non-clinical; relies on SKEL’s current DOFs and bones placement learned from BioAMASS.
  • Sector: Robotics/HRI and Simulation
    • Action: Use SKEL to simulate human movement with anatomically realistic DOFs for controller testing, ergonomics, and human–robot interaction planning.
    • Workflow: Import SKEL into physics simulators; run inverse kinematics/retargeting with reduced DOF constraints to produce safer, more realistic motions.
    • Potential tools/products: SKEL module for MuJoCo/Bullet/Drake; HRI scenario libraries with anatomically constrained avatars.
    • Assumptions/Dependencies: SKEL is kinematic; no muscles or ground reaction forces; coupling with dynamics requires additional models.
  • Sector: Ergonomics/Occupational Health (pilot use)
    • Action: Rapid screening of joint angles and postures from workplace videos using SKEL-enhanced markerless capture to flag risky movements.
    • Workflow: Video → SMPL → SKEL → compute joint angle ranges and posture metrics.
    • Potential tools/products: Pilot ergonomic assessment tool (non-diagnostic).
    • Assumptions/Dependencies: Not yet clinically validated; limited accuracy under loose clothing/occlusion; privacy/compliance requirements for workplace video.
  • Sector: Research Tooling (inverse problems, optimization)
    • Action: Use SKEL’s more realistic DOFs to constrain parameter spaces in pose/shape optimization, reducing over-parameterization and improving stability.
    • Workflow: Replace SMPL’s ball joints with SKEL’s DOFs in optimization pipelines for motion fitting.
    • Potential tools/products: SKEL-based optimization libraries; comparative studies.
    • Assumptions/Dependencies: Integration effort in existing codebases; performance depends on solver tuning.

Long-Term Applications

These applications are promising but require additional validation, scaling, and/or extensions (e.g., muscles, dynamics, clinical studies).

  • Sector: Clinical Gait Analysis and Orthopedics (healthcare, policy)
    • Action: Clinical-grade, markerless biomechanical assessment from smartphone or clinic cameras to estimate joint centers/angles for diagnosis and monitoring.
    • Workflow: Video → SMPL → SKEL → clinical metrics (e.g., knee valgus, spine curvature) → clinician review.
    • Potential tools/products: Telemedicine gait apps; hospital-grade analysis suites; guidelines for markerless biomechanical assessment.
    • Assumptions/Dependencies: Regulatory approval; extensive clinical validation across demographics and pathologies; robust clothing/occlusion handling.
  • Sector: Rehabilitation and “Biomechanical Digital Twins”
    • Action: Patient-specific SKEL-based digital twins used to plan and track rehabilitation; personalize exercises based on anatomically accurate motion.
    • Workflow: Initial assessment → SKEL fit → longitudinal tracking → adaptive exercise protocols.
    • Potential tools/products: Rehab platforms with SKEL avatars; compliance monitoring tools.
    • Assumptions/Dependencies: Longitudinal accuracy; integration with wearables; outcome studies demonstrating benefit.
  • Sector: Prosthetics/Implants Design
    • Action: Use SKEL joint centers and orientations to inform personalized prosthetic alignment or implant planning.
    • Workflow: Imaging + video → SKEL alignment → mechanical design constraints for devices.
    • Potential tools/products: Pre-operative planning tools; prosthetic alignment software.
    • Assumptions/Dependencies: Fusion with CT/MRI; regulatory standards; mechanical validation; population-scale anthropometric calibration.
  • Sector: Exoskeletons and Assistive Robotics
    • Action: SKEL-informed controllers and fitment protocols that respect human joint limits and coupled motions (e.g., scapular kinematics).
    • Workflow: Human motion capture → SKEL features → controller design and safety envelopes.
    • Potential tools/products: Exoskeleton fitment assistants; simulation-first design pipelines.
    • Assumptions/Dependencies: Dynamics and load modeling (muscles/forces); safety certification; individualized anatomy mapping.
  • Sector: Sports Performance and Injury Prevention
    • Action: Large-scale athlete monitoring with anatomically meaningful metrics (e.g., pronation ranges, scapular dynamics) for technique optimization and risk reduction.
    • Workflow: Training video → SMPL/SKEL → biomechanical dashboards → coach feedback loops.
    • Potential tools/products: Team analytics platforms; performance scoring systems.
    • Assumptions/Dependencies: Robustness to sport-specific attire and environments; link metrics to outcomes via longitudinal studies.
  • Sector: Population Health and Policy
    • Action: Aggregate, anonymized biomechanical indicators from public or workplace footage to inform ergonomic standards and public health interventions.
    • Workflow: Privacy-preserving analytics → SKEL metrics at scale → policy recommendations (e.g., lifting/posture guidelines).
    • Potential tools/products: National/enterprise risk dashboards; standards updates.
    • Assumptions/Dependencies: Privacy/ethics frameworks; bias mitigation; representativeness across populations.
  • Sector: Content Creation and Retargeting (advanced workflows)
    • Action: Automatic motion retargeting that preserves anatomical constraints across characters with different shapes, reducing artifacts in animation.
    • Workflow: Source motion → SKEL-constrained retargeting → target character animation.
    • Potential tools/products: SKEL-constrained retargeting plugins for DCC and game engines.
    • Assumptions/Dependencies: Real-time efficiency; extension of SKEL pose correctives; widespread toolchain support.
  • Sector: Extended Model Ecosystem (research & industry)
    • Action: Expand SKEL with muscles, tendons, and soft tissue models; learn SKEL-specific pose-dependent deformations; unify with dynamics for full-body simulation.
    • Workflow: Combine BioAMASS with medical datasets; learn muscle activation/soft tissue; integrate physics.
    • Potential tools/products: Full digital human biomechanics stack for simulation and analysis.
    • Assumptions/Dependencies: Access to medical data; compute-intensive training; cross-disciplinary validation.

Cross-cutting assumptions and dependencies

  • Pseudo-ground-truth: Current joint accuracy is referenced to AddBiomechanics fits; clinical-grade truth requires medical validation.
  • Input quality: All downstream accuracy depends on SMPL estimation quality from video; multi-view and high-fidelity capture improve results.
  • Demographic coverage: Training data (AMASS-derived) may underrepresent certain body types, ages (e.g., children, elderly), and pathologies.
  • Real-time constraints: Some applications require optimized implementations and GPU acceleration to meet latency targets.
  • Licensing and ethics: Dataset and video usage must comply with consent, privacy, and licensing requirements; clinical use requires regulatory approvals.
Ai Sparkles Streamline Icon: https://streamlinehq.com View Paper Prompt  List Streamline Icon: https://streamlinehq.com View All Prompts 

Glossary

  • 3DPW: A real-world dataset of 3D human poses and shapes in the wild. "We apply this process to archival datasets such as 3DPW \cite{von2018recovering} and BEDLAM \cite{bedlam}."
  • Active set method: An optimization technique for solving constrained least squares problems. "solving it with an active set method \cite{lawson1995solving}."
  • AddBiomechanics: A biomechanical optimization framework used to fit skeletal models to motion capture markers. "We use AddBiomechanics \cite{werling2022addbio} to obtain the BSM scale parameters ss and the NfN_f poses {qf}\{q_f\}."
  • AMASS: A large dataset unifying motion capture sequences into SMPL parameters. "we take sequences of 3D bodies from the AMASS dataset \cite{mahmood2019amass} that cover a wide range of body shapes and challenging poses."
  • Axis-aligned frame of reference: Joint coordinate frames aligned with global axes. "all joint orientations are defined in a global T-pose space with an axis-aligned frame of reference for each joint"
  • Axis-angle representation: A rotation parameterization defined by an axis and an angle. "Each joint is parameterized by three degrees of freedom in an axis-angle representation."
  • Ball joint: A joint allowing three rotational degrees of freedom. "the knee, spine, or the shoulder."
  • BEDLAM: A large-scale dataset of synthetic humans with diverse motions. "We apply this process to archival datasets such as 3DPW \cite{von2018recovering} and BEDLAM \cite{bedlam}."
  • BioAMASS: A paired dataset of SMPL bodies and biomechanical skeletons in motion. "The paired SMPL meshes and BSM skeletons form the BioAMASS dataset."
  • Biomechanical Skeleton Model (BSM): A custom anatomical skeletal model with realistic degrees of freedom. "we unify the SMPL body model with BSM, a new Biomechanical Skeleton Model."
  • BMI: Body Mass Index, used to reference subject body shape/size. "On high BMI subjects, a shape-agnostic marker definition for all subjects yields over-stretched bones (red)."
  • BOSS: A learned model combining skin, bones, and organs from medical data. "The recent BOSS model \cite{shetty2023boss} improves on OSSO by learning a skin-bone-organs model from segmented 3D medical data."
  • Constant curvature (joint): A custom joint model for spine bending with fixed arc length. "we call 'constant curvature', to model the spine bending with a constant length."
  • Degrees of freedom: The number of independent parameters defining motion. "the pose parameters q∈R46q \in \mathbb{R}^{46} represent the 46 degrees of freedom of the articulated model."
  • DFAUST: A dataset of high-resolution dynamic human scans. "Fitting SKEL to DFAUST scans \cite{bogo2017dfaust} results in SKEL's scapula sliding (c) and the forearms twisting appropriately (d)."
  • Ellipsoid: A geometric surface used to constrain scapula motion around the thorax. "that slides along an ellipsoid defined around the thorax."
  • Euler angles: A rotation parameterization using ordered axis rotations. "represented as Euler-angles in XZY."
  • Femur head: The spherical proximal end of the femur forming the hip joint center. "Predicting the location of joints, such as the femur head, from 2D images of clothed individuals is inherently ill-posed because the joint location is not directly observed."
  • Forearm pronation and supination: Rotational motions of the forearm around the longitudinal axis. "drive forearm pronation and supination."
  • Forward kinematics: Computing joint positions and markers from pose parameters. "Using forward kinematics, these functions output the skeleton joint locations, BSMJ(s,q)BSM^J(s, q), the bone meshes vertices, BSMv(s,q)BSM^v(s, q), and the posed marker locations, BSMm(s,q,m0)BSM^m(s, q, m_0)."
  • Hinge joint: A single-axis rotational joint type. "The ulna is linked to the humerus through a hinge joint, enabling the elbow flexion."
  • Humerus: The upper arm bone connecting shoulder and elbow. "The ulna is linked to the humerus through a hinge joint, enabling the elbow flexion."
  • Inverse kinematics: Solving for joint parameters that best match target positions. "inverse-kinematics with these scales yields poses {qf}\{q_f\}"
  • Joint regressor: A learned mapping from surface vertices to anatomical joint locations. "we learn a joint regressor JJ that takes as input the SMPL mesh vertices v∈R6890×3v \in \mathbb{R}^{6890 \times 3} and predicts the new anatomic joints J∈R24×3J \in \mathbb{R}^{24 \times 3}."
  • Kinematic tree: The hierarchical structure describing joint connectivity and transformations. "The SMPL kinematic tree is artist-defined and only approximately corresponds to the human anatomy."
  • Linear blend skinning (LBS): A standard technique to pose mesh vertices using weighted joint transforms. "Then linear blend skinning (LBS) is used to pose the vertices and produce the body mesh vertices."
  • Marker-based motion capture (mocap): Tracking movement using physical markers and cameras. "marker-based motion capture (mocap) systems."
  • Markerless motion capture: Estimating motion without physical markers, often from video. "markerless video-based 3D motion capture is catching up with marker-based techniques."
  • MoSh: A method fitting statistical body models to mocap markers to reconstruct surface motion. "MoSh \cite{Loper:SIGASIA:2014,mahmood2019amass} unifies mocap and statistical body models by fitting the parameters of the model to match the marker data."
  • OpenCap: A system for biomechanical analysis from smartphone videos. "For example, OpenCap \cite{uhlrich2022opencap} enables biomechanics from smartphone videos."
  • OpenPose: A multi-person 2D pose estimator used to detect joint locations in images. "They use OpenPose \cite{cao2017openpose} to detect the subject's 2D joint locations in several camera views and reconstruct their 3D locations."
  • OpenSim: An open-source platform for musculoskeletal modeling and simulation. "we create BSM, a custom skeleton model using the OpenSim framework \cite{delp2007opensim}"
  • OSSO: A model predicting internal skeletal geometry from body surface meshes. "In contrast, OSSO \cite{keller2022osso} learns to predict the geometry of the bones from a SMPL body mesh."
  • Parametric body model: A compact, controllable human shape model driven by pose and shape parameters. "parametric body models like SMPL \cite{loper2015smpl}."
  • Proximal and distal ends: Anatomical terms for ends of a bone closer to or farther from the torso. "the 'bone axis' as the axis passing through the bone's proximal and distal ends."
  • Pseudo-ground-truth: Approximate labels treated as ground truth for evaluation. "we take the joint locations estimated by AddBiomechanics as pseudo-ground-truth."
  • Radius and ulna: The two forearm bones enabling pronation and supination. "The forearm is made of two bones: the radius and the ulna."
  • Regression (regressor): Learning a mapping from inputs to outputs, e.g., predicting joints from vertices. "we train a regressor to estimate the 3D anatomical BSM joint locations of the body given a posed SMPL mesh."
  • Rigid transformation: A rotation and translation applied without deformation. "we use the composition of rigid transformations T(R,t)T(R, t) defined by a rotation matrix RR and a translation tt"
  • SCAPE: A statistical body model used to envelop biomechanical skeletons. "BASH \cite{schleicher2021bash} uses the SCAPE body model \cite{anguelov2005scape} to envelop a biomechanical skeletal and muscle model"
  • Scapula: The shoulder blade bone that slides along the rib cage during arm motion. "making the scapula slide along the ribs."
  • SKEL: A parametric model unifying skin and anatomical skeleton driven by biomechanical pose. "we develop SKEL, which re-rigs the SMPL body model with a biomechanics skeleton."
  • SMPL: A widely used statistical human body model with learned shape and pose-dependent deformations. "Specifically, we focus on advances in computer vision that infer the 3D pose and shape of the human body in the form of parametric body models like SMPL \cite{loper2015smpl}."
  • SMPL-X: An extension of SMPL including hands, face, and more joints. "one can fit, or regress, SMPL or SMPL-X body model parameters \cite{Kanazawa:CVPR:2018,Pavlakos2019smplx,Kocabas2020vibe}."
  • Spline: A smooth curve used to model complex spinal motion. "the spine in SKEL is modeled by a spline derived from biomechanics."
  • Thorax: The chest region; used to define scapula motion constraints. "defined around the thorax"
  • Virtual markers: Synthetic markers placed on the mesh surface to simulate mocap input. "we place virtual motion capture markers on the body surface."
Ai Sparkles Streamline Icon: https://streamlinehq.com View Paper Prompt  List Streamline Icon: https://streamlinehq.com View All Prompts 

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 4 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free