In one aspect, a system for monitoring an orientation of an agricultural implement during an agricultural operation is disclosed. The system may include a work vehicle and an agricultural implement coupled to the work vehicle and configured to be towed by the work vehicle. The system may include a vision-based sensor coupled to the work vehicle and a controller communicatively coupled to the vision-based sensor. The controller may include a processor and associated memory. The memory may store instructions that, when executed by the processor, configure the controller to receive image data from the vision-based sensor； determine an orientation parameter based on the received image data； and initiate a corrective action based on the orientation parameter of the agricultural implement while the agricultural implement is being towed by the work vehicle. The orientation parameter may describe an orientation of the agricultural implement relative to the work vehicle.
