In one aspect, a system for controlling the operation of an agricultural implement may include an agricultural implement configured to perform an agricultural operation on a field as the implement is moved across the field. A controller of the system configured to receive data indicative of a plurality of field characteristics of the field. The controller may also be configured to determine first and second soil moisture content values of the soil within the field at first and second depths below a field surface of the field based on the received data, respectively. Additionally, the controller may be configured to initiate adjustment of an operating parameter of the agricultural implement based on the determined the first soil moisture content value and the determined second soil moisture content value.