In one aspect, a system for controlling the position of row units of an agricultural implement as the implement is moved across a field may include a row unit. The system may also include an actuator configured to adjust a position of the row unit between an operating position relative to the ground and a raised position relative to the ground. Additionally, the system may also include a controller configured to monitor a current location of the row unit relative to a boundary defined between a processed portion of the field and an unprocessed portion of the field. Furthermore, the controller may be configured to control an operation of the actuator to initiate adjustment the position of the row unit between the operating and raised positions when it is determined that the row unit is about to traverse across the boundary.