In one aspect, a system for controlling the speed of an agricultural implement may include a work vehicle including a vehicle-based controller, with the vehicle-based controller being configured to adjust a drive parameter of the work vehicle. The system may also include an agricultural implement configured to be towed by the work vehicle. The implement may include a sensor configured to detect an operational parameter indicative of an orientation of the implement relative to the work vehicle. The implement may further include an implement-based controller supported on the implement and being communicatively coupled to the sensor. The implement-based controller may be configured to initiate control of the drive parameter of the work vehicle based on sensor data received from the sensor in a manner that adjusts the speed of the implement.
