An audio source location, tracking and isolation system, particularly suited for use with person-mounted microphone arrays. The system increases capabilities by reducing resources required for certain functions so those resources can be utilized for result enhancing processes. A wide area scan may be utilized to identify the general vicinity of an audio source and a narrow scan to locate pinpoint positions may be initiated in the general vicinity identified by the wide area scan. Subsequent locations may be anticipated by compensating for motion of the sensor array and anticipated changes in source location by trajectory. Identification may use two or more sets of characterizations and rules. The characterizations may use computationally less intense analyses to characterize audio and only perform computationally higher intensity analysis if needed. Rule sets may be used to eliminate the need to track audio sources that emit audio to be eliminated from an audio output.