If you had exactly 10 years left to live but could trade some of them in return for a total cure for persistent and troubling symptomatic floaters, how much life would you trade?
Readers will be more than used to reading papers in refereed journals and recognising the various analyses used to identify data relevance. There is a myriad of options, ranging from defined diagnoses rates, probability indicators, and statistical assessment of differences between groupings through to subjective ratings and question response analyses. Even within studies applying similar analytical methods, one has to be aware of influences such as sample size, incentive biases (is this free lens better than the old ones you pay for?), questionnaire subtleties, the make up of the sample (AREDS2 was criticised for using subjects of a more, shall we say, nutritionally aware lifestyle to name but a few).
Often, methods seem to follow certain trends. Post-AREDS, for example, odds ratios came to the fore. The recent controversial paper by Griffiths et al rely heavily on the Cochrane Risk of Bias tool, a relatively new insight likely to become more familiar in future. The ‘time trade off’ analysis above is a new one on me. It seems to be asking whether someone willing to give up more life years might be more likely to be selected for treatment if resources are scarce as their desire is greater– scary stuff.
At the same conference, we heard how subjective comfort ratings by the same subjects may differ simply by how they are scored. Scoring comfort, for example by stating a number from one (comfortable) to 10 (agony), is less repeatable and accurate than asking someone to mark a point on a line drawn between one and 10, which has much greater repeatability.
Look out for a forthcoming CET article explaining how best to interpret research findings – remember the Vic Reeves quote that 82% of statistics are unreliable.