A system for residue detection and implement control is disclosed. The system comprising: an agricultural implement (10) with access to a source of environmental data (220, 222, 236, 234, 238, 310) having an indication of environmental factors; a sensor (54, 58, 74, 82) that obtains image data of a field (14); a data store (210) containing image processing methods to detect residue; a controller (70) operatively coupled to the sensor (54, 58, 74, 82) and to the source of environmental data (220, 222, 236, 234, 238, 310) and to the data store (210), the controller (70) processing the image data according to one or more of the image processing methods; wherein the controller (70) is configured to: select one or more of the image processing methods based on the environmental data (220, 222, 236, 234, 238, 310); process the image data using the selected one or more of the image processing methods (500) to determine a value of residue coverage in the imaged area of the field (14); and generate one or more control signals that includes an adjustment to the agricultural implement (10) based on the value of residue coverage. Further, a method (400, 500) to detect residue and control an implement (14) is disclosed.