(Misleading) graph from NPR.

The headline on an NPR blog post Friday blared, “Brain Injuries Rose In Cities After Bike-Sharing Rolled Out.” It sounded horrible! A tiny graph showed a pretty clear trend, suggesting hard data behind this conclusion. As it turns out, while the authors of the study and news stories sensationalize the issue, the data show the exact opposite.

The paper’s lead author even said that this supports the conclusion she expected all along. Only it doesn’t. Instead, the data seem to show that in cities which launched bike sharing systems, bike-related traumatic injuries decreased, including head injuries.

What’s wrong with the analysis

The researchers (Janessa Graves, Barry Pless, Lynne Moore, Avery Nathens, Garth Hunte, and Frederick Rivara) obtained data from trauma center records in cities that implemented public bike sharing programs (PBSPs) and control cities that did not.

The best analysis would compare the number of bike-related head injuries to the total amount of bicycling. That’s because if more people bike but bicycling gets safer, you might see more injuries even if every individual cyclist’s risk goes down.

If they didn’t have that information, it would still be useful to just look at the total numbers of bike-related injuries or bike-related head injuries. Instead, the authors made the odd decision to look at the percentage of bike-related injuries that are head injuries.

This percentage can increase for two reasons:

  1. There are more head injuries
  2. There are fewer non-head injuries (and thus the head injuries make up a larger share)

Clearly, (1) is bad. But (2) is not.

In the paper, the authors get around this limitation by being careful to state their results in an exact fashion:

In this international study, implementation of a PBSP was associated with 14% greater risk of head injury among patients admitted to trauma centers for bicycle-related injuries.

As often happens, news stories shortened this to the very different headline, “Brain Injuries Rose In Cities After Bike-Sharing Rolled Out.” But it’s not just the media. The authors have also been less careful in public statements since writing the article.

NPR quoted lead author Graves saying, “Public bike-share initiatives are great wellness initiatives. But without providing helmets, we were concerned that we would see an increase in head injuries. And we did.”

Only that’s not what they really would see if they looked at the data more clearly.

What the numbers seem to really say

I couldn’t perform an in-depth analysis similar to the study’s because I don’t have their dataset. However, the paper includes a table (Table 2) which lists the total numbers of head and non-head injuries in PBSP cities and control cities before and after the bike sharing program began.

That makes it possible to graph just the raw numbers of injuries for these time periods. (I adjusted the numbers for the fact that the paper’s pre-implementation period spans two years and the post-implementation period only spans one.)

The overall trend is clear. For the control cities, there isn’t much change in total injuries (an increase of 2%) or head injuries (4% decrease). But for the PBSP cities, head injuries decreased by 14% and total injuries decreased by 28%.

Weaknesses in the data

There are plenty of caveats to both the original study and my analysis, many based on limitations in the data.

Since the data on injuries come from emergency room records, the study only captures injuries that involve visits to the ER. The decision to go to an ER could be totally nonrandom for a whole range of injuries, and we could imagine that a large bike share program might affect whether people go or not.

Moreover, the introduction of a bike sharing program is not a discrete event. The authors had to pick a date for each city, and it’s unclear how sensitive the analysis is to pushing the dates a bit in either direction. Plus, some of their dates may simply be incorrect. At least in DC, as Darren Buck pointed out, the paper lists bike sharing as starting in May 2010, when it really launched in late September 2010.

Finally, the dataset just doesn’t contain a ton of cases. The authors end up with two years pre-implementation and one year post-implementation for each PBSP city, and the same length of time for control cities. They only have 5 PBSP cities and 5 control cities.

Since bike-related injuries are not incredibly common and this isn’t a very long time period, it’s going to be hard to identify gradual trends in the data. Bike advocates tend to focus on the need to change culture over the long term, with bike sharing as one element of that. One year after the introduction of bike sharing simply may not be long enough to see much of an effect.

Finally, the dataset suffers from the problem that it’s not actually experimental data. The authors make a nice effort to deal with this by finding a matched control city for each PBSP city, but there are plenty of other differences between the paired cities other than the presence of bike sharing.

For example, the PBSP cities already had a lower incidence of total bike-related injuries before they implemented bike sharing, so it’s hard to argue that they’re identical to the control cities. The drop in injuries could be due to some other change in PBSP cities at the same time (DC built its first cycletracks in the same year it launched Capital Bikeshare, for example).

We have no way of knowing with this sort of analysis. And since I don’t have the full dataset, I can’t drill down to individual cities. But the overall trend is pretty dramatic: control cities had virtually unchanged injury rates, while PBSP cities had large drops in total bike-related injuries as well as smaller drops in head injury rates.

Conclusions

Graves has said that the authors started the study expecting to find that because bikeshare riders often don’t use helmets as often as other cyclists, bikesharing systems put people at greater risk of head injuries. If we take their data at face value, instead we have evidence that bikesharing may instead decrease serious bike-related injuries, including head injuries.

That would be a big deal, because many people expected that perhaps injuries would increase, but the numbers of cyclists would increase faster, meaning bicycling got safer. The paper’s data might not be correct, but if it is, then it’s even better news for bike sharing: cycling went up, injuries per cyclist went down, but on top of that, total injuries went down, too.

Even if it’s too soon to draw that conclusion given all the caveats listed above, it’s clear that the study’s authors have not shown a strong and direct relationship between new bike sharing systems and increases in bike-related head injuries. It’s too bad that they have trumpeted an incorrect interpretation of their data, and that press reports have spread a false and sensationalized conclusion far and wide.