1The idea in one paragraph

The regression line of y on x is built to predict y. It is the line that makes the vertical distances from the points to the line as small as possible, because the vertical distance is exactly the error you make when you predict a y-value. If instead you know y and want x, the error that matters is the horizontal distance, and the line that makes those as small as possible is a different line: the regression line of x on y, written x = cy + d. The two lines are not rearrangements of each other. They cross at the mean point (x̄, ȳ), they are close together when the correlation is strong and far apart when it is weak, and each one is only for its own job. The rule to learn is short: use the line whose left-hand side is the variable you want.

2Why one line is not enough

In 4.4 the regression line of y on x was found by the least squares method. For every data point, measure the vertical distance to the line; that is the residual, the amount by which the line misses the point when it predicts y. Square those distances and add them. The regression line of y on x is the line that makes that total as small as it can be. Figure 1, left, shows the vertical misses for five points.

Now turn the job round. Suppose you know a y-value and want to estimate x. Your error is now how far left or right you land, which is a horizontal distance. The line that makes the sum of squared horizontal distances smallest is the regression line of x on y, shown on the right of Figure 1. For those five points it is x = 0.752y − 0.205, while the y on x line is y = 0.83x + 1.77. Rearranging the first gives y = 1.33x + 0.273 (to 3 s.f.), a much steeper line than y = 0.83x + 1.77. They are genuinely different lines, because they answer different questions.

The names tell you which is which. "y on x" means y depends on x: y = ax + b, with y on the left. "x on y" means x depends on y: x = cy + d, with x on the left. The variable before "on" is the one the line predicts.

To predict y, use y on x. To predict x, use x on y. Never rearrange one line to get the other.

3Finding the x on y line with technology

The GDC has only one linear regression function, and it always fits "second list = gradient × first list + intercept". So to get x on y, give it the lists the other way round: the y-values as the first (independent) list and the x-values as the second.

• On a TI-84 Plus: put x in L1 and y in L2, then run LinReg(ax+b) with Xlist set to L2 and Ylist set to L1.
• On a Casio: in the regression settings, set XList to the list holding the y-data and YList to the list holding the x-data.
• On a TI-Nspire: in the linear regression dialog, choose the y-data column as the X List and the x-data column as the Y List.

The calculator will still print its answer as "y = ax + b", because it does not know you swapped the lists. You must read it as x = (a)y + (b) and write it that way. Writing the output with the letters unchanged is the commonest way to lose the marks here.

Worked example 1: trees. A forester measures the height y (m) and the trunk diameter x (cm), at chest height, of eight trees of one species in a plantation.

From the GDC, as in 4.4:

Now swap the lists:

A drone survey can measure the height of a tree from above but not its diameter, so the forester wants to estimate the diameter of a tree 19 m tall. The known value is a height, y; the wanted value is a diameter, x. That is a job for x on y.

Figure 2 shows both lines with the data. Rearranging the y on x line instead, x = (19 − 7.41) ÷ 0.286, would give 40.5 cm, marked grey on the figure. That is a two-centimetre difference caused only by using the wrong line.

The reverse job has its own line. To estimate the height of a tree with diameter 30 cm, use y on x: y = 0.28615 × 30 + 7.4068 = 16.0 m. Rearranging the x on y line would give 15.5 m, which is again wrong.

Interpreting c and d. In x = 2.44y − 7.82, the gradient c = 2.44 means that for each extra metre of height, the predicted trunk diameter increases by 2.44 cm. The intercept d = −7.82 would be the predicted diameter of a tree of height 0 m, which is meaningless: no tree has height 0, and it is far outside the data. As in 4.4, an intercept often has no sensible interpretation, and saying so is the right answer.

4Why rearranging gives the wrong answer

The two lines disagree because the correlation is not perfect. Look at the tree that is 19 m tall. It is 1.9 m taller than the mean height of 17.1 m. The y on x line, rearranged, assumes that a tree's diameter is exactly as far above average as its height is, in the line's units, and gives 40.5 cm. But with r = 0.835, height does not fix diameter. Some tall trees are thin. The best estimate of the diameter of a tall tree is above average, but not by as much as that, so the x on y line pulls the estimate back towards the mean diameter, to 38.5 cm.

Each line pulls its predictions towards the mean of the variable it predicts. The y on x line pulls predicted heights towards ȳ; the x on y line pulls predicted diameters towards x̄. That is why each line is the right one only for its own variable.

Figure 3 shows how much the two lines differ as the strength of correlation changes. When r is weak, they open like scissors: one is nearly flat and the other nearly vertical, and predictions from the wrong one are wildly out. As |r| grows the scissors close, and when r = ±1 every point lies on one line and the two regression lines are the same line. That is the only case in which rearranging one gives the other.

A fact that is not on the syllabus but makes a good check: when both lines are written with their own variable on the left, y = ax + b and x = cy + d, the product of the gradients is a × c = r². For the trees, 0.28615 × 2.43810 = 0.698, and 0.835² = 0.698. If your two gradients multiply to more than 1, one of them is wrong.

5Both lines pass through the mean point

Every least squares regression line passes through the mean point (x̄, ȳ), the point whose coordinates are the mean of the x-values and the mean of the y-values. That was true of y on x in 4.4, and it is true of x on y too. So the two lines always cross at the mean point, as the cross in Figure 2 shows.

That gives Paper 1 a question that needs no data at all.

Worked example 2 (no calculator). For a set of bivariate data, the regression line of y on x is y = 0.8x + 2 and the regression line of x on y is x = 1.1y − 1.3. Find x̄ and ȳ.

The mean point lies on both lines, so solve them simultaneously. Figure 4 shows the crossing.

So x̄ = 7.5 and ȳ = 8.

A shorter version gives one line and one mean. If the regression line of x on y is x = 2.5y − 4 and ȳ = 6, then the mean point lies on the line, so x̄ = 2.5 × 6 − 4 = 11. Notice that this works only because (x̄, ȳ) is on the line. You cannot find x̄ by putting some other value of y into the line.

The same Paper 1 question often goes on to ask for a prediction, with both lines printed. Choosing the right one is the whole mark: to estimate y when x = 10, use y = 0.8x + 2 and get 10; to estimate x when y = 10, use x = 1.1y − 1.3 and get 9.7.

6When a prediction can be trusted

Choosing the right line is necessary, but it does not make the prediction good. The same two checks from 4.4 apply to x on y, and Figure 5 puts the whole decision in one place.

1. Is the correlation strong? A line fitted to weakly correlated data predicts poorly in either direction. With r = 0.835 the tree predictions are reasonable estimates; with r = 0.3 they would be little better than guessing the mean.
2. Is the given value inside the range of the data? Predicting inside the range of the observed values is interpolation, and is reasonable when r is strong. Predicting outside it is extrapolation, and is unreliable, because nothing tells you the linear pattern continues. For the x on y line, the range that matters is the range of the y-values, because y is the value you are given. The trees' heights run from 11.8 m to 22.8 m. A prediction for a 19 m tree is interpolation; a prediction for a 30 m tree is extrapolation, and the line would give 2.4381 × 30 − 7.8165 = 65.3 cm, a number with no evidence behind it.

A third point decides which line is sensible at all. The guide's warning, that you cannot always reliably predict y from x with an x on y line, is the same warning as 4.4's in reverse. A question may give both equations and ask you to explain your choice. The answer is always of this shape: "y is the known value and x the value to be estimated, so the regression line of x on y is used; the y on x line minimises vertical, not horizontal, distances, so it does not give the best estimate of x."

Correlation is still not causation. A strong x on y line lets you estimate a tree's diameter from its height; it does not say that height causes diameter. Both grow with the tree's age.