设训练数据为（Xi,Yi）,i=1,2,...,n.
回归模型为 Y = aX + b + W.
a,b为待定系数.w为模型误差.
Yi = aXi + b + Wi,
Wi = Yi - aXi - b,i = 1,2,...,n
一般的回归准则是使得
[W1]^2 + [W2]^2 + ...+ [Wn]^2 = [Y1-aX1-b]^2 + [Y2-aX2-b]^2 + ...+ [Yn-aXn-b]^2
= a^2{[X1]^2 + ...+ [Xn]^2} + nb^2 + 2ab[X1 + ...+ Xn] - 2a[Y1X1 + ...+ YnXn] - 2b[Y1 + ...+ Yn] + [Y1]^2 + ...+ [Yn]^2
达到最小.
记
f(a,b) = [W1]^2 + [W2]^2 + ...+ [Wn]^2
则
df/da = 2a{[X1]^2 + ...+ [Xn]^2} + 2b[X1 + ...+ Xn] - 2[Y1X1 + ...+ YnXn]
df/db = 2nb + 2a[X1 + ...+ Xn] - 2[Y1 + ...+ Yn]
令
df/da = df/db = 0可以解出a,b.
线性回归法的误差 一般定义为Wi的平方的平均值的平方根【均方根误差】,
或者Wi的绝对值的平均值,或Wi的绝对值的最大值,等等.