Seeing History
CorrelationCorrelation has a common meaning: how well things match up with each other. In statistics, the concept acquires greater precision, and also greater interpretive power. It works like this: Of two measurable quantities associated with n different objects, correlation tells us how closely changes in one quantity are predictable from changes in the other quantity. If we measure the height and the annual income of 18 people, we can calculate how far income is a function of height in that sample. Take another sample:
You are the Mayor of Islington. The Mothers of Islington come to you in a body, demanding that more open space be created, since out of 18 districts of London, Islington ranks at the bottom for percentage of open space (variable x, Sp), but near the top for percentage of accidents that are accidents to children (variable y, Ac). They thrust in your face these figures, for Islington and the other 17 districts.
District Bermondsey Camberwell Deptford Finsbury Fulham Hammersmith Hampstead Islington Marylebone Paddington Poplar Shoreditch Southwark Stepney Stoke Newington Wandsworth Westminster Woolwich 
How to make sense of them? You mentally calculate the average value, or mean, of the x column, and then subtract that average (mx, the mean of x) from each individual value of x to make the high and low values more obvious. The result is a column of (x-mx) figures. You then do the same to y. You have:
District Bermondsey Camberwell Deptford Finsbury Fulham Hammersmith Hampstead Islington Marylebone Paddington Poplar Shoreditch Southwark Stepney Stoke Newington Wandsworth Westminster Woolwich SUM (S) mean (S/18)
Hmm, you say to yourself, look at that zigzag pattern. The minus or below-average values of x do seem to match the plus or above-average values of y. There is clearly something going on here.
We now want to define the relationship more exactly. To do this, we could calculate what is called the correlation coefficient, which is not intuitively meaningful, or we could go for the coefficient of determination (D), which is intuitively meaningful. Let's go for D. To get it, we first multiply together each pair of (x-mx) and (y-my) terms, and put the result in column 6. Then we square each of those terms, and put the results in columns 7 and 8. The result, accomplished by an unobtrusive secretary while you distract the Mothers of Islington with small talk, would look like this:
District Bermondsey Camberwell Deptford Finsbury Fulham Hammersmith Hampstead Islington Marylebone Paddington Poplar Shoreditch Southwark Stepney Stoke Newington Wandsworth Westminster Woolwich SUM (S) mean (S/18)
The boxes are now all filled in (note that mx means "the mean of the x values" and so on), and we can proceed to calculate our answer. It's very easy. All we need are the three sums here called a, b, and c. We substitute them into the following formula:
D = a² / bc
getting
D = 0.014029454 / (0.099146)(0.170173) = 0.014029454 / 0.016871972 = 0.8315
And we can then prefix the sign of "a" (which here is minus) to the final result, to remind us that the correlation is inverse: as one quantity goes up, the other tends to go down. Thus, finally,
D = -0.8315
The D value tells us, as a fraction, how much one of our categories can be predicted from other category. In this case, it is 83%. Then the proportion of accidents which are accidents to children is 83% associated with the amount of open space. It is a pretty high level, but still, it leaves some room for politics. Here come the politics:
"Ladies," you then say, "I congratulate you on gathering these figures; they do indeed show a relationship between open space and proportion of accidents to children. About five-sixths of accidents to children can be accounted for by the proportion of open space. But one-sixth cannot be so explained, and surely (eyeing briefly every sixth mother in the room) we would not wish to abandon one-sixth of our children to their fate. Perhaps we should look at it even more closely. If you will ask the Police Commander for a map of the borough showing where accidents to children have occurred, and I will have my secretary request his cooperation, we can pinpoint the exact locations of these lamentable accidents, and take precise and effective steps to prevent them."
And as they head for the doorway, you add silently, Or you could move to Westminster.
Just for the record, if you want to get instead the classical correlation coefficient C in the example above, it is the square root of D, which in this case would be:
C = 0.9119
The standard textbooks will tell you, by means of a series of tables in the back, how to interpret this.
Readings
- M J Moroney. Facts From Figures. The Islington problem (with its out of date data) is at p288. It there gets a correlation coefficient solution; the coefficient of determination is not mentioned. Moroney reaches a correlation coefficient of -0.92. This is a little high, due to rounding error.
- Islington Police Station. Note the program for safety in the schools.
17 Dec 2006 / Contact The Project / Exit to Outline Index Page
