Abstract:Accurate real-time perception of human muscle force is a core prerequisite for achieving "assistance-on-demand" control in lower-limb exoskeletons. To address the vulnerability of traditional surface electromyography (sEMG)based motion intention recognition to failure under challenging real-world conditions such as muscle fatigue, electrode displacement, and skin perspiration, this paper proposes a non-invasive lower-limb muscle force prediction and control method based on the multimodal fusion of depth vision and inertial measurement unit (IMU). First, a multimodal visual and kinematic dataset was constructed involving five healthy subjects across five typical daily terrains, including level ground and sloped walking at various gradients. The internal muscle forces of the quadriceps and hamstrings were calculated offline using an sEMG-driven Hill-type musculoskeletal model to serve as the supervised ground truth. Second, a dual-branch spatiotemporal feature fusion deep network was designed, which extracts spatial-geometric features from depth image sequences and temporal dynamics from inertial motion data through parallel channels. By incorporating an adaptive attention mechanism for the intelligent, heterogeneous weighted fusion of these features, the proposed framework completely eliminates the reliance on sEMG sensors during the practical application phase, thereby bypassing signal degradation issues. To bridge the gap from perception to closed-loop control, an adaptive impedance control framework driven by real-time predicted muscle force was established, continuously mapping variations in muscle force to the assistance stiffness of the exoskeleton. Experimental results demonstrate that the proposed method significantly outperforms unimodal approaches, yielding a robust cross-subject average coefficient of determination (R2) exceeding 0.81 and a root-mean-square error (RMSE) of less than 6 N. Furthermore, the predicted force curves maintain highly consistent profiles under simulated muscle fatigue interference, exhibiting superior robustness against physiological disturbances. Finally, platform validation on a commercial lower-limb exoskeleton system achieved high temporal phase synchronization and low lateral drift, providing a promising and viable new paradigm for the highly robust, assistance-on-demand control of exoskeletons in complex, dynamic environments.