A position detection system includes: a capsule medical device having therein a magnetic field generator for generating a magnetic field; magnetic field detectors for detecting the magnetic field and outputting detection signals; and a processor including hardware. The processor is configured to: calculate a position of the capsule medical device using at least one of the detection signals; determine whether or not proper position detection of the capsule medical device based on the detection signals is possible using a threshold that is a value based on at least one of output values of the detection signals output by the magnetic field detectors under a predetermined condition; decide a determination value using at least one of the output values of the detection signals; and determine that the proper position detection of the capsule medical device is not possible when the determination value is smaller than the threshold.