Provided is an image processing apparatus for fitting a model to image data. The image processing apparatus determines a final parameter set and a final inlier scale of a first model by iteratively performing a model estimation process and an inlier scale estimation process, wherein the model estimation process determines a model parameter set, and the inlier scale estimation process determines an inlier scale by using the determined model parameter set.