Disclosed are various embodiments for computer-aided tracking and motion analysis with ultrasound. A computing device is employed to access an ultrasound video generated by at least one ultrasonic imaging device and/or at least one motion sensing camera. A target patch embodied is tracked throughout frames of the ultrasound video by compressing target patches of individual ones of the frames of the ultrasound video into vectors generating a space partitioning data structure for each of the frames of the ultrasound video and identifying an image intensity feature for each frame utilizing a corresponding one of the space partitioning data structures generated for each frame of the ultrasound video. Optimized tracking locations may be determined for a sequence of the ultrasound video using the image intensity feature identified for each frame.