Anecdotes, levels of evidence and why scientists may seem obsessed with controls, placebos, and blinding.
It can be difficult, especially for non-scientists to understand why scientists get worked up over “proper” evidence. Take the discussions on static magnets. Lots of people “believe” they work. That’s not the same as having “evidence they work”. Uncontrolled, unorganised, personal experience is classed as anecdotal. It’s the lowest form of evidence and some don’t even consider it to be a form of evidence at all. The problem with anecdotes is that they are biased and do not take into account multiple other potentially relevant factors.
Take those who “insist” static magnets, hologram discs, ceramic impregnated or titanium impregnated materials have a significant effect. There are multiple problems with this as evidence:
1) The observer is not neutral but is biased. They have already invested in the “treatment”.
2) There is no control – the effect they “witnessed” could have occurred in any case without the “treatment”. But it’s impossible to know as you have nothing else to compare to know what would have happened in any case.
3) The observed “improvement” may not have been due to the intervention as perceived by the owner – e.g. a magnetic boot may have had an effect due to compression, insulation (heating), altered limb sensation (proprioception), greater induced movement, and altered owner-horse dynamic i.e. absolutely nothing to do with the magnet.
So, please excuse and indulge us, scientists, when we get a little obsessive, adamant or intransigent when it comes to anecdotes and evidence and why we seem fixated on controls, placebos, and blinding.
