A monitoring system tracks the non-visible structure of a body in three dimensions. A tracker obtains image information of the object and instruments in its vicinity, all bearing 3D tracking markers with at least one pattern segment. A controller spatially relates image information with previously obtained scan data of the object revealing non-visible structure of the object. For the scan a fiducial reference detectable in the scan is removably attached to a location on the object. Scan data and image information is used by the controller to provide the user with real time information on relative 3D locations and orientations of instruments and the non-visible structure of the body. In a further embodiment the monitoring system may be used to model and track changes in the body itself. In other embodiments, a model of the body and instruments is used to track contemplated actions and warn about possibly inappropriate instrument procedures.
