In one aspect, the present subject matter is directed to a system for controlling the operation of a residue removal device of a seed-planting implement. The system may include a residue removal device configured to remove residue from a path of the seed-planting implement. The system may also include a furrow closing assembly including at least one closing disc, with the furrow closing assembly configured to close a furrow formed in the soil by the seed-planting implement. Furthermore, the system may include a sensor configured to capture data indicative of an operational parameter of the furrow closing assembly and a controller communicatively coupled to the sensor. The controller may be configured to monitor the operational parameter based on the data received from the sensor, the controller further configured to control an operation of the residue removal device based on the monitored operational parameter.