Reconsidering the value of pupil attitudes to studying post-16: a caution for Paul Croll



Intention to stay
on

Intention to leave

Total

‘Observed’ to stay on

519"

84^

603^^

‘Observed’ to leave

206^

34

240^

Total

725

118

843^

Just because most pupils say they will stay on and most pupils do stay on, using
intention to ‘predict’ participation will be 66% ([519+34]/843) accurate even though
in my analysis the two things are designed to be completely unrelated. This means
that Croll’s figure for the accuracy of intentions (see above) is only 10% better than
pure chance. Presumably it is possible to imagine around 10% of each cohort already
knowingly on a trajectory towards school avoidance or long-term academic success.
This would mean that Croll is wrong to say that expressed intentions are good
predictors of later participation more generally. Intentions are rather weak predictors,
if indeed they are predictors at all (see below).

Looked at another way, by how much can using pupil year 7 intentions help to make a
more accurate prediction of whether they stayed on post-16 than we could make
without using them? Since 72% of pupils stay on in fact, if we predicted that all year 7
pupils would stay on then we would be right in around 72% of cases. This is even
better than the 66% chance model above, and makes Croll’s accuracy of 76% look
less impressive again. Using intentions would only improve the base prediction by 34
pupils (4% of 843). If this were a logistic regression model, with a binary outcome
variable of participation or not, it would fail because both the outcome and the
predictor are so skewed. It is a shame that Croll did not attempt such a model, or
similar, and that the journal reviewers for BJES did not request it, since it would
quickly reveal the flaw in the claims about intentions being 76% accurate. Intentions
improve the accuracy of predictions by 4% at best.

Croll describes a standard set of potential background determinants associated with
post-16 participation like prior attainment, sex, and family background. These could
have been used in a model to predict participation, and would probably improve
predictions by more than the 4% increase on the baseline due to using intentions (see



More intriguing information

1. The name is absent
2. Climate change, mitigation and adaptation: the case of the Murray–Darling Basin in Australia
3. The name is absent
4. Innovation in commercialization of pelagic fish: the example of "Srdela Snack" Franchise
5. Handling the measurement error problem by means of panel data: Moment methods applied on firm data
6. The name is absent
7. Analyzing the Agricultural Trade Impacts of the Canada-Chile Free Trade Agreement
8. Spectral density bandwith choice and prewightening in the estimation of heteroskadasticity and autocorrelation consistent covariance matrices in panel data models
9. A parametric approach to the estimation of cointegration vectors in panel data
10. Foreign Direct Investment and Unequal Regional Economic Growth in China