Tuesday, December 20, 2011

On reducing risk.

Currently, there's multiple public panics which neatly cross lines dividing partisan and political allegiances, over things including registering guns, vaccines, wi-fi (in schools), and (placement of) wind turbines. One common thread shot through all of them is that we need to "reduce risk". In what sense these policies reduce risk is totally opaque. Risk assessment is tricky, but that's no excuse for not knowing how to do it -- at least approximately.

Here's the idea. First, you need to know what risk you're trying to reduce. This is not just because you need to know what you're trying to target for reduction, but also because what's being reduced has to be able to be reduced.

Take registering guns, for example. If it "reduces risk", we need to, first, determine what risk we're trying to reduce. Plausibly, it's the risk of violent crime involving guns. And we can measure that -- we can tell how many violent gun crimes there were before registration, and how many after. (Of course, it's a little more complex than that as violent gun crime is multicausal and the analysis would have to isolate the effect of registration. But, it's at least possible to do so.)

Similarly for having wi-fi, and let's presume it's in schools. Again, what's the risk at stake? It can't just be "complaints" or "pain" or anything else that's subjective and difficult to measure, as then it's impossible to tell whether eliminating wi-fi has any effect, positive or negative. If you can't measure it, then notion of "reduction" is meaningless. The risk we're trying to reduce has to be something we can find and figure out what effect the policy change has had on it, if any.
Wind turbine placement has a similar problem, as do silicone breast implants, for that matter. The supposed effects that are being targeted for reduction are so vague and/or subjective that they can't really be measured. And, again: if you can't measure it, you can't reduce it.

So, first possible error: trying to reduce a risk that can't, as a matter of logic, actually be reduced.

Once that's figured out, the second step is to actually collect the data and try to determine what effect the policy change has had. In some cases, this is already well known. It is very well-established that vaccines reduce the incidence of disease, while causing a much smaller number of deaths due to adverse reaction to the vaccine. If the risk at stake is the risk of death and disease, it's a classic no-brainer: vaccination wins over non-vaccination.

When it comes to wi-fi, wind turbines, breast implants and the like, if we restrict our attention to something we can actually measure, then the answer is equally clear. They don't cause anything that can be objectively measured. So, there's nothing to reduce.

With regards to registering guns, it's a little less clear, as most of the data available are irrelevant to the issue. ("The police check the registry thousands of times!" "...and?") We would need to compare circumstances where guns are not registered to circumstances where they are, controlling for all other variables. This is the only way, as far as I can tell, to see whether registering guns reduces risk or not. And it is at least plausible that the reduction in violent gun crime since registration was imposed in Canada can be attributed to factors other than registration.

So, second possible error: assuming that one's favoured policy change reduces risk, rather than performing the necessary data comparison to show it does so.

I'm not saying there aren't other reasons to favour policy changes; but if you're going to rest your argument on reducing risk, is it really so much to ask that you do the legwork?

No comments: