Key takeaway: The least-squares regression line is the straight line through a scatterplot that makes the sum of the squared vertical distances from the points as small as possible, and it is used to predict y from x.
Grade 12 – AP StatisticsAP Statistics Unit 5Statistics and Data
Definition of Least-Squares Regression Line
The least-squares regression line summarizes a linear trend in paired data. Every candidate line leaves a vertical gap, called a residual, between each point and the line. Squaring the gaps and adding them gives one number, and the least-squares line is the one with the smallest such number.
The line always passes through the point of means, \((\,\overline{x},\,\overline{y})\). Its slope tells the predicted change in \(y\) for each one-unit increase in \(x\). It is meant for prediction inside the range of the data, not far outside.
Formula and notation
The fitted line is \(\hat{y}=a+bx\), where
\[ b=r\,\frac{s_y}{s_x},\qquad a=\,\overline{y}-b\,\,\overline{x}. \]
Here \(r\) is the correlation, \(s_x\) and \(s_y\) are the standard deviations, and \(\hat{y}\) (y-hat) is a predicted value. The residual is \(y-\hat{y}\).
Key facts
- The residuals of a least-squares line always add to zero.
- The slope has units of y per unit of x; the intercept is the prediction at x = 0.
- Predicting far outside the observed x values, called extrapolation, is unreliable.
- A residual plot with no pattern supports using a least-squares regression line.
Least-Squares Regression Line example
Five students studied 2, 4, 6, 8 and 10 hours a week and scored 5, 9, 10, 15 and 16 on a 20-point quiz. Find the least-squares line and predict the score for 7 hours.
-
Means: \(\,\overline{x}=6\) and \(\,\overline{y}=11\). Deviation products: \((-4)(-6)+(-2)(-2)+0+(2)(4)+(4)(5)=56\), and \(\sum(x-\,\overline{x})^2=40\).
-
Slope: \(b=\frac{56}{40}=1.4\). Intercept: \(a=11-1.4\cdot 6=2.6\).
-
The line is \(\hat{y}=2.6+1.4x\), so for 7 hours \(\hat{y}=2.6+9.8=12.4\).
The regression line is \(\hat{y}=2.6+1.4x\), predicting a score of 12.4 for 7 hours of study.
Common mistake
Watch out
A student uses the line to predict the score for 30 hours of study and reports 44.6 points.
Why it is wrong
The quiz is out of 20 points, and the data only cover 2 to 10 hours. Using the line at 30 hours is extrapolation, and the linear trend almost surely does not continue that far. Restrict the least-squares regression line to predictions inside the range of observed x values.
Related terms and lessons
Browse the glossary: A–Z list · AP Statistics vocabulary · Statistics and Data