A robot system and a method to control the robot system are used to detect position information of a surgical instrument in an incised region, thereby improving the safety of robotic surgery. A surgical instrument may be inserted in a through-hole of a trocar inserted into an incised region of a patient. The medical robotic system includes a surgical instrument position detection apparatus to detect position information of the surgical instrument in the through-hole of the trocar, when the surgical instrument is inserted into the through-hole. The medical robotic system further includes a console to control an operation of a surgical robot having the surgical instrument, based on the detected position information of the surgical instrument.