In one aspect, a system for monitoring field profiles may include a vision-based sensor configured to capture vision data indicative of a profile of a field, with the profile being at least of a crop canopy profile of the field or a soil surface profile of the field. The system may also include a secondary sensor configured to capture secondary data indicative of the profile of the field. Furthermore, a controller of the disclosed system may be configured to receive the vision data from the vision-based sensor and the secondary data from the secondary sensor. Moreover, the controller may be configured to determine a quality parameter associated with the vision data. Additionally, when the quality parameter falls below a minimum threshold, the controller may be configured to determine at least a portion of the profile of the field based on the secondary data.
