STATISTICAL ANALYSIS or SOCIAL ENGINEERING?
A Critical Review of a Statistics Canada
"Factors Associated with Bicycle Helmet Use"
by Avery Burdett
BACKGROUND
Statistics Canada, a federal government department has published an article
titled "Factors Associated with Bicycle Helmet Use" in the Autumn 1997
edition of Health Reports (vol. 9 no. 2). This article has as its stated
objective "to examine characteristics associated with the use of bicycle
helmets by children and adults." and uses data from the 1994/95 National
Population Health Survey to determine the use of bicycles and bicycle
helmet use. However, based on a reading of the discussion of results of the
statistical analysis and on a critical examination of sources of information
cited, it appears the authors had much more in mind than to merely look at
bicycle helmet usage. The last section of the report entitled "Implications"
reveals the real intent of the authors, Statistics Canada analyst Wayne
J. Millar, a self-professed helmet enthusiast, and Ivan B. Pless of the
Montreal Children's Hospital. Buried in this section is a trojan horse
plea for universal mandatory bicycle helmet legislation. Although an area
of public policy outside the scope of Statistics Canada's mandate, this might
have been slightly more palatable to experienced and knowledgable cyclists
had the discussion been objective and the sources of research current and
credible, but as the following review demonstrates, they are anything but.
EXTRAORDINARY STATEMENTS AND CLAIMS
The Efficacy of Bicycle Helmets
The Statistics Canada article claims several studies have demonstrated
that head injuries can be prevented through the use of helmet (p31). But
several other studies have concluded the opposite also. No reduction in
the rate of head injuries, serious head injuries or deaths in jurisdictions
(New Zealand, Australia and US) where helmet use has been heavily promoted
and mandated. Why were these other sources ignored?
Associating General Injuries with Helmet Purchase
The article states that since parents from the lowest income groups
were less likely to perceive injuries as the leading cause of death
among children, they may have been less likely to consider a bicycle
helmet a necessary purchase (p34).
This statement is difficult to fathom since it refers not to cycling
injuries which result in a small percentage of accidental deaths of
children but to deaths suffered by children in all types of accidents.
The association with bicycle helmets lacks an obvious cause and effect.
Misleading Use of Statistics
The article cites a Kingston, Ontario study where 8% of head
injuries to those under the age of 20 involve cycling (p36). Since
the authors don't say, this begs the question what proportion of
these were wearing helmets? By 1996 the year of the study, helmet
use had increased significantly as a result of the helmet law in
Ontario (applying to those under 18). In about half of the fatalities
in Ottawa over the past three years, the victims were wearing helmets.
If helmeted children had been overrepresented among the injured
what would that say about the efficacy of a bicycle helmet?
It raises contextual questions too. Given the amount of time children
spend on bicycles, 8% of child head injuries doesn't seem excessive.
What activities were the victims involved in which make up the other
92%? What percentage was from motor vehicle use? In 1993 according to
the Ministry of Transportation of Ontario's Road Safety Annual Report,
virtually all road deaths of those under 20 involved motor vehicles.
There was no involvement of a cyclist in 93% of them. Motor vehicle
use accounted for over 78%, and 15% were pedestrians. This shows
that those under 20 year of age are eleven times more likely to be
killed inside a car and twice as likely to get killed while walking.
Since the majority of these deaths involve head injuries, one wonders
why cyclists are singled out for special treatment. By focussing on
the effect of accidents involving a small minority of road users, the
authors bypass important issues - the cause of accidents and their
prevention among all road users.
Citing a 1983 Florida study, the report states "research suggests 9
out of 10 fatalities involve injuries to neck and head" (p37). Under
the caption "Cycling Deaths" it states "almost two-thirds of fatalities
involve head injuries". The authors quote these figures without
adjusting them to remove numbers of fatalities where a helmet would
have played no role. For example, the first figure includes neck
injuries, injuries which cannot be prevented by a helmet. But the
report does not say how many out of 9 were neck injuries. Was it 8
or was it 1? Also the use of the word "involve" implies that head
injuries were not the sole factor or cause of all fatalities.
Furthermore, head injuries include injuries to ears, face, jaw and
other parts which are not protected by a helmet. The only relevant
figure in this discussion is the number of fatalities among bareheaded
cyclists whose sole cause of death was a head injury for which a
helmet purports to offer protection. Clearly both the 9 out of 10 and
the two thirds figures are misleading, yet the authors made no effort
to adjust it or caution readers on its interpretation. One must ask
why? Could it be that the authors need to overstate the problem?
Another example of selective use of figures of doubtful relevancy is
the claim "Helmets reduce the risk of serious head injuries by up to
85%." This is taken from Thompson, R., Rivara, F., & Thompson, D., A
Case Control Study of Effectiveness of Bicycle Safety Helmets, New
England Journal of Medicine, 320: 1361-67, 1989 which claimed helmets
reduce the risk of head injuries, not serious head injuries as claimed
by Millar and Pless, by 85%. This report is a study of cyclists received
at emergency rooms at Seattle area hospitals. It has been widely
distributed because it claimed the highest effectiveness rate of any
helmet study. Other studies have suggested much lower risk reduction
(as low as zero). Concern has been expressed by experts over Thompson
et al's selection of cases and controls, and methods of adjusting for
age group disparities. Also, the nature of injuries typically received at
the ER are minor and not representative of the serious head injuries which
are popularly, but wrongly, perceived to be prevented by helmets. There
have been studies showing no helmet effect in severe impacts. "Serious and
unsurvivable head injuries do not differ in relative frequency between
helmet wearers and bareheaded cyclists (McDermott, FT et al, Effectiveness
of Bicyclist Helmets: a study of 1710 casualties, 1993, Journal of Trauma,
vol 11:6 pp 834-45). The authors of this study looked at the Thompson et
al data and concluded the rate of effectiveness to be much lower. Despite
the abundance of contrary evidence, Statistics Canada adopted the highest
risk reduction claim available, and then compounded the distortion by
misquoting the article to make the claimed efficacy of bicycle helmets
even greater. Was this just sloppy work or a deliberate attempt to
mislead?
The authors take the 85% figure from Thompson et al and improperly link
it to another figure in the following quote "... yet just 4% of persons
involved in fatal bicycle-related accidents in Ontario between 1986 and 1991
were wearing helmets". The Thompson et al study was not a study of fatalities
but, as mentioned above, injuries received for emergency room treatment.
So they discuss helmets and fatalities but cite an analysis of mostly
minor injuries. In addition, context would have been helpful to
put the figure in perspective since helmet use was very rare in the 1980's.
If the helmet wearing rate among the population was less than 4%, then
a professional statistician would have asked why helmeted cyclists were
overrepresented in fatalities. In NSW, Australia (1992) 82.4% of cyclists
killed were wearing helmets. The helmet wearing rate at the time was
81.7%. For the period 1992 to 1994 the rate of helmet wearing was matched
by the rate of helmet wearing among fatality injured cyclists (data from
Road Transport Authority, NSW). In a total of 11 years of child helmet
legislation among 8 states in the US, no statistically significant drop
in fatalities has been recorded (data from US National Highway Traffic
Safety Authority). Why didn't the authors examine such data? These results
were well known among cycling groups who oppose helmet legislation, and
freely available over the Internet.
But Figures Show Kids Need Skills
Had the authors done a thorough analysis of their own data, they might
have obtained an insight into the real issue. P37 shows percentages
child cyclist deaths among the population between 1980 and 1994, a period
of no or low helmet use. Fatalities represented 4% of children under
10 years, 10% of those 10 to 14 years, and 2% of those 15 to 19 years.
The large differences between age groups should have been scrutinized.
Fatalities bulge in the middle (10 - 14) age group. The most likely
explanation is that the skills of cyclists improve with age with the
very youngest cyclists receiving close parental supervision.
Studies show that the more persons ride bicycles, the better their skills
get and the fewer accidents they are involved in. The decline in the rate
of deaths among older children suggests skills, and therefore educational
approaches, are more relevant in strategies to reduce cyclist fatalities.
The Selling of Helmet Legislation
The last section is where the real motives behind Statistics Canada
article can be found. It says "Although one study suggests this [reduction
in head injuries] is partly attributable to a reduced number of riders,
the net effect [of helmet legislation] is clearly beneficial. For example,
in Victoria [Australia], after legislation made bicycle helmets mandatory
there was a marked increase in the rates of use between 70% and 90% within
a year and a substantial decline in head injuries and mortality." (P38).
This is a carefully crafted statement which masks the truth. The rate,
or percentage, of cyclists wearing helmets can increase while at the same
time cycling declines. In Australia, that appears to have happened. There
is an abundant amount of evidence indicating declines in the numbers of
Australians cycling, as is conceded by the authors. Regardless of helmet
usage, a decline in head injuries and mortality was to be expected if
participation in cycling declined. But no explanation of this is offered by
the authors even though it would be fundamental in carrying out a statistical
analysis. Robinson, D. L., Head Injuries & Bicycle Helmet Laws, Accident
Analysis Prevention, Vol 28, pp 463-475, 1996, a study of Australian road
victim accidents, states "the most remarkable aspect of the data both from
Victoria and New South Wales is that the decrease in numbers of cyclists was
at least as large as the decrease in numbers of head injuries and larger than
the total decrease in injuries. This suggests cyclists are now worse off than
before the law." From the same source, it states that following the helmet
law in New South Wales, the cyclist proportion of deaths and serious injuries
among child road users actually increased. In the two years prior to the law
in New South Wales, 19 (1989) and 20 (1990) cycling fatalities occurred. In
1994 following legislation 23 cyclist fatalities occurred with a 76% helmet
use rate and a 30% to 40% decline in cycling. Why didn't the authors track down
these facts? A thorough and unbiased search of available research would have
uncovered them.
The Statistics Canada report doesn't just limit its efforts to analyses
of data. It moves into another field of competence - behavioural science
- by stating "However, current research suggests that education alone is
not sufficient to reach and maintain high levels of helmet use and that
legislation is needed.", "However, making laws applicable to the entire
population would reinforce their importance, and peer approval might be
reduced", and "More important from a public health perspective, .....
bicycle-associated injuries creates a health burden regardless of age".
All three opinions are unsubstantiated. All three are highly questionable,
And all three trespass into policy areas where Statistics Canada has no
business.
CONCLUSIONS
So what does this tell us?
1. A report which has Statistics Canada stamped on it immediately is
accorded credibility which will tend to limit rational discussion. But
had the authors involved experts in the field of cycling - professionals
and experienced club cyclists - different conclusions might have been
forthcoming.
2. There is a plethora of evidence from countries with mass helmet use
that helmets have little effect on the rate of head injury among the population.
Furthermore, cycling is a life extending activity. By exaggerating the problem
of cycling accidents, cycling is falsely made to appear dangerous. This has the
effect of discouraging some from cycling, as happened in Australia. Considering
the enormous cost to Canada of heart disease - it causes premature death of
1,000 Canadians for every premature death of a cyclist - we can ill afford to
implement a strategy which negatively impacts the fitness of Canadians.
3. Even in provinces with helmet laws, teenagers are reluctant to wear
helmets. When they do, as many as a half are improperly fitted. Police don't
have the desire or resources to chase after kids, so children learn very quickly
to ignore the law. This sends the wrong message.
4. It has been shown that virtually all road deaths involve a motor
vehicle and young folks are 13 times more likely to be killed as car users
or pedestrians than they are as cyclists. An approach which treats the
symptoms not the cause, benefits no one. It would be far more useful to
parents if they were made aware of the much greater danger motor vehicles
pose to their children while the kids are walking or passengers in a car.
By forcing a small segment of road users to be responsible for protecting
themselves amounts to victim blaming. This is not only a disservice to young
Canadians and their parents but also a grave injustice.
5. Statistics Canada is a government agency whose expertise is in the
field of gathering, assembling, and analysis of data. It is improper to
take public policy positions or to endorse predetermined biases, regardless
of how politically correct they appear. Sometimes it is necessary to present
factual data which are unpopular or politically unwelcome. Even so, based on
the contents of this report, it is difficult to find any objectivity. It looks
as though there has been a deliberate attempt to distort facts. If that is not
the case, then the report represents an extremely sloppy piece of work.
6. Individual officers of the agency should not be pursuing subjects
because of personal interest (which it was in this case). It amounts to
creating "make work" projects, a pastime Canadian taxpayers can ill afford.
7. Most important, Statistics Canada has no mandate to develop social policy,
yet this document explicitly calls for the adoption of policy positions in
the areas of health, law and education. Worse still, the policies, if
implemented, infringe on the fundamental rights and freedoms of millions
of Canadians.
Statistics Canada owes Canadians an explanation.