Described is a system for anomaly detection to detect an anomalous object in an image, such as a concealed object beneath a persons clothing. The system is configured to generate a subspace model for a normal class using training images. The normal class represents normal objects in a common class. The system receives a novel image having an object in the common class. A set of geometric landmarks are identified in the object in the novel image for use in registering the image. The novel image is registered by warping the image so that the geometric landmarks coincide in the novel image and the training images, resulting in a warped novel image having an object. Thereafter, the system determines if the object in the warped novel image is anomalous by measuring the distance of the warped novel image from the subspace model. Finally, if anomalous, an operator is notified accordingly.
