A system and method for selecting follicular units for hair harvesting using imaging and processing techniques are provided. The method of selecting an order of harvesting follicular units is based on a combination of one or more policies and filters that are generally designed to improve a speed, quality and efficacy of the harvesting process. The method of the present invention may be implemented with various hair harvesting and transplantation systems, including manual, partially automated and fully automated systems.