In one aspect, a system for controlling the operation of agricultural sprayersmay a vision-based sensor configured to capture data indicative of one or morefeatures within a field across which an agricultural sprayer is traveling. Acontrollerof the system may be configured to identify the one or more features withinthe fieldbased on the data received from the vision-based sensor and determine aposition of aboom component of the agricultural sprayer relative to the one or moreidentifiedfeatures. Furthermore, the controller may also be configured to initiate acontrolaction to adjust the position of the boom component when it is determined thattherelative position between the boom component and the one or more identifiedfeaturesis offset from a predetermined positional relationship defined for the boomcomponent.
