What is disclosed is a system and method for estimating a vector of skin parameters from in-vivo color measurements obtained from a video. In one embodiment, a video of exposed skin is received which comprises a plurality of time-sequential image frames acquired over time t. A vector of in-vivo color measurements is obtained on a per-frame basis from at least one imaging channel of a video imaging device used to capture the video. In a manner more fully disclosed herein, an intermediate vector of estimated skin parameters is determined based on an initial vector of estimated skin parameters and the in-vivo color measurements of all image frames averaged over time t. A final vector of estimated skin parameters is then determined for each image frame of the video based on the intermediate vector. The temporally successive final vectors are used to predict changes in time-varying skin parameters for the subject.