Derive a Least-Squares Solution 

Fit a line to data using the method of least squares.

Define the sum of squares of the vertical deviations for a given set of data.

In[1]:=

X squareDeviations = Inactivate[Sum[(y[[x]] - (a x + b))^2, {x, 1, n}], Sum | Part]

Out[1]=

Set up the equations for minimum deviation.

In[2]:=

X eqna = D[squareDeviations, a] == 0

Out[2]=

In[3]:=

X eqnb = D[squareDeviations, b] == 0

Out[3]=

Generate some data.

In[4]:=

X n = 100; y = Table[5 x + 78 + RandomReal[{-50, 50}], {x, 1, n}];

Solve the least-squares problem for this data.

In[5]:=

X sol = a x + b /. Solve[Activate[{eqna, eqnb}], {a, b}][[1]]

Out[5]=

Plot the solution along with the data and verify they match.

In[6]:=

X n = 100; y = Table[5 x + 78 + RandomReal[{-80, 80}], {x, 1, n}]; Show[ Plot[sol, {x, 1, 100}, PlotTheme -> "Scientific"], ListPlot[y], PlotRange -> All, ImageSize -> Medium]

Out[6]=