Fig. 9

The neural network architecture combining vision and kinematics data³. The performance of this network (named as ‘Optical Flow’) is compared with the optical flow features zeroed out (named as ‘No Flow’). We find that using egocentric optical flow helps improve prediction of knee and ankle joint ankles from the rest of the body. The prediction performance is measured in terms of the root mean squared error (RMSE) between the predicted trajectory and the actual measured trajectory using motion capture.