A system for assisting a surgeon to perform an operation on a patient where surgeon employs a first surgical instrument that has a shaft and a tip. The system include a video capture device configured to capture real-time video data showing the first surgical instrument while the first surgical instrument is being used by the surgeon to perform the operation. The system also includes a display device and a computer processor. The computer processor is configured to receive real-time video data from the video capture device as a sequence of digital image frames and, for each image frame, analyze the image frame to determine the location in the image frame of the shaft of the first surgical instrument, process the image frame to overlay received overlay information on the shaft of the first surgical instrument as it appears in the image frame, and display the processed image frame on the display device.
