To improve the overall navigation process for minimally invasive repair of heart valve leaflets, an augmented reality technique capable of providing a robust three-dimensional context for transesophogeal echocardiography data has been developed. In the context of various embodiment of the invention, augmented reality essentially refers to a system in which the primary environment is virtual but the environment is augmented by real elements. In this real- time environment, the surgeon can easily and intuitively identify the tool, surgical targets, and high risk areas, and view tool trajectories and orientations.
