设计理念源自一个洞察:人的动作不是孤立的点,而是连续的、有节奏与方向的流动。我从舞蹈理论家 Rudolph Laban 的动作符号系统、以及运动摄影先驱 Gjon Mili 的轨迹影像中获得启发——身体语言能否转化为视觉、听觉、触觉,反过来辅助学习?据此搭建技术链路:动作捕捉(YOLOv8-Pose / MediaPipe 提取人体关键点)→ 轨迹生成(Python 处理时序数据)→ 正确轨迹比对(DTW 计算与标准动作的偏差,量化肘、肩、腕的关节角度)→ 多模态反馈(视觉粒子、听觉音效、触觉震动)。硬件层用 ESP32 + MPU6050 六轴传感器实测挥拍力度,超过阈值即触发差异化震动。
The concept came from one insight: movement isn't isolated points but continuous, rhythmic flow. Drawing on Rudolph Laban's movement notation and Gjon Mili's motion photography, I built a pipeline: capture (YOLOv8-Pose/MediaPipe) → trajectory (Python) → comparison (DTW, quantifying joint angles) → multimodal feedback (particles, sound, haptics), with an ESP32 + MPU6050 measuring real swing force.