As the standard of each finalist for the 2016 CooperVision Student of the Year was exceptionally high, and the voting excitingly close, we have decided this year that each merits the spotlight and will are publishing a report on each over the coming Contact Lens Monthly issues. This month we look at the finalist from Anglia Ruskin.
Finalist – Max Clemens
Project – Inter-observer agreement of soft contact lens fit assessment
Supervisor – Dr Marta Vianya, Dr Yvonne Norgett, Robert Conway
A clinical grading system, such as that described by Bailey et al,1 uses a percentage scoring to describe changes sensitively. However, inter-user variation is likely to be significant and perhaps therefore problematic. A more simplified three-point scaling system, as proposed by Wolffsohn,2 is likely to show good concordance but might not adequately reflect changes on eye. Added to this, some studies have shown that the experience of the practitioner is an influence upon grading accuracy,3 and others have recommended an objective approach as more useful.4
Student clinics
The project by Clemens aimed to look at two different grading systems and see which might be more appropriate for the student clinics at Anglia Ruskin. Clemens explains, ‘Towards the end of the second semester we invited second year students to assess the fit of three different contact lenses each. We asked them to use two randomly assigned grading scales; the traditional and the three-point scale with one scale repeated. They were asked to use these scales to assess the blink movement in up-gaze, horizontal lag and push up displacement before making an overall judgement.’
Figure 1 shows the forms participants used in the trial to assess the contact lens wearer appearance. As Clemens explained, ‘The one on top is our example of a traditional scale. Participants were asked to estimate the blink movement in up-gaze and horizontal lag to the nearest 0.1mm and estimate push up displacement as a percentage (100% being tight/stuck and 0% being loose/falling out). Participants were asked to judge the overall fit by ticking one of the boxes at the bottom that had a range of loose reject, loose accept, optimal, tight accept and tight reject. The three-point scale is shown below the traditional scale, each of the same measurements were to be taken on a different lens but recording was graded E for excessive, G for good or I for inadequate. As with the other scale, participants were asked to judge the overall fit by ticking one of the five boxes at the bottom.’
Figure 2 shows how the six lenses chosen for the trial varied in their overall fitting. ‘On the y-axis you can see the numbers one to five. Five was deemed a tight fit that would be rejected, four an acceptable tight fit, three an optimal fit, two an acceptable loose fit and one a loose fit that would be rejected.
Figure 2: Fitting ratings for six trial lenses
‘We randomly paired observations of the same eye made by participants, using this method we could look at the concordance of the results, how often those pairs of measurements agreed with one another. Looking at the graph in figure 3, the x and y axes represent observers A and B and each point on the graph represents their evaluation of the measure taken. If those evaluations are the same, they are said to be in concordance and will sit along the mid line. Because there is a greater range of possible recordings with the traditional scale we expect concordance to be lower with this scale.
Figure 3: Paired observer ratings
‘Intra-class correlations coefficients gives us a measure of the reliability of different observers’ measurements, but is affected by the range of possible outcomes, with the three-point scale offering only three outcomes we expect the ICC to be lower than that of the traditional scale measurements.’
Results
‘The traditional scale had narrower 95% confidence intervals which gives the student a greater ability to detect change but with the trade-off of lower concordance. However, agreement between students was found to be higher when applying the intra-class correlation coefficient. Because our data did show more agreement between students using the ICC statistic and lower 95% confidence intervals giving greater decision making powers, we must conclude that the traditional scale is the more appropriate for training student optometrists.’
The judges were impressed by the design of this small scale study and felt it well highlighted the issues as to which grading scale is best suited to any particular clinical discipline.
Read more
Award winning pupil project in the spotlight
Researchers of the future – part 3
References
1 Bailey, et al (1991), Clinical grading and the effects of scaling, Investigative Ophthalmology & Visual Science, Vol 32, No 2, Pages 422-432.
2 Wolffsohn, et al (2009), Simplified recording of soft contact lens fit, Contact Lens & Anterior Eye 32, Pages 37-42.
3 Cardona & Serés (2009), Grading contact lens complications: the effect of knowledge on grading accuracy, Current Eye Research, No 34(12), Pages 1,074-1,081.
4 Belda-Salmerón, et al (2015), Objective analysis of contact lens fit, Contact Lens & Anterior Eye 38, Pages 163-167.