The statistical counterpart or analogue of the functional expression, in ordinary mathematics, of one variable in terms of others.
A random variable is seldom uniquely determined by any other variables, but it may assume a unique mean value for a prescribed set of values of any other variables. The variate y is statistically dependent upon other variates x1, x2, · · ·, xn when it has different probability distributions for different sets of values of the xs. In that case its mean value, called its conditional mean, corresponding to given values of the xs will ordinarily be a function of the xs. The regression function Y of y with respect to x1, x2, · · ·, xn is the functional expression, in terms of the xs, of the conditional mean of y. This is the basis of statistical estimation or prediction of y for known values of the xs. From the definition of the regression function, we may deduce the following fundamental properties:
where σ2(w) denotes the variance of any variate w, and E(w) denotes the expected value of w. The variate y is called the regressand, and the associated variates x1, x2, · · ·, xn are called regressors; or, alternatively, y is called the predictand, and the xs are called predictors. When it is necessary to resort to an approximation Y′ of the true regression function Y, the approximating function is usually expanded as a series of terms Y1, Y2, · · ·, Ym, each of which may involve one or more of the basic variates x1, x2, · · ·, xn. By extension of the original definitions, the component functions Y1, Y2, · · ·, Ym are then called regressors or predictors. Various quantities associated with regression are referred to by the following technical terms: The variance σ2(y) of the regressand is called the total variance. The quantity y -Y is variously termed the residual, the error, the error of estimate. Its variance σ2(y -Y) is called the unexplained variance, the residual variance, the mean-square error; and its positive square root σ(y -Y) is called the residual standard deviation, the standard error of estimate, the standard error, the root-mean-square error. The variance σ2(Y) of the regression function is called the explained variance or the variance reduction; the ratio σ2(Y)/σ2(y) of explained to total variance is called the relative reduction, or, expressed in percent, the percent reduction.
