It’s been a week to be reminded of the limitations of surveys.
As a vegan (but one with a Stilton and port problem), I managed to find an article of interest in Vegan News (stay with me on this one). A government survey had found that around 0.4% of the UK population was vegan, while one authorised by the Vegan Society had found a much higher number, approaching 1.2%.
What was interesting was how differently the surveys were set up – the former asked whether the subject considered themselves to belong to a category, such as vegan, lactovegetarian and so on. The latter simply asked how often they ate meat, fish and dairy and classified those responding ‘never’ as vegan.
Next day, Glasgow University published a report critical of a survey of the number of times people had sex. Their findings strongly implied that men tended to offer a far higher figure based on a perceived kudos that such a figure would imply.
Questionnaire design is a major challenge. A simple change of inflexion or word order can bias any answer. As can patient beliefs of what they feel should be the correct answer or not. Let alone the influence of any incentive (either material or reflected).
For example, Dr Robin Chalmers showed how, if you ask a contact lens patient whether they have dry eye symptoms, you get a much lower positive response than if you asked the question ‘At what time of the day do your eyes feel dry?’ Both are valid and neither is aimed at tricking the patient.
Last week, a study from Australia following up the Blue Mountain study cohort over the last 15 years suggested that an orange a day may decrease the risk of AMD by as much as 60 per cent. I can see at least three significant concerns with this sort of statement…