This UK NEQAS service is for the assessment of clinical advice/comments and of patient investigation/management. The service is provided at a charge of £60 a year.
Eligibility to participate:
The scheme is designed primarily for medical practitioners and clinical scientists who are authorised to undertake independent practice in medical microbiology and virology in the UK. Other participants (non-UK specialists, trainees) will be identified in a way which allows the Scheme Organiser to analyse individual results separately. Please note that confidential, individual codes are provided to all participants so that performance is known only to the individual participant.
Aims and objectives of the scheme:
- To assess interpretive aspects of clinical practice in medical microbiology and virology
- To provide high quality, relevant and effective CPD
- To promote competence and consistency of medical advice
- To provide supporting evidence for appraisal, revalidation and laboratory accreditation
How the scheme works:
On the first working day of each month, a new distribution will appear on the website. Cases will reflect a range of complexity and include both general and specialist issues encountered by medical microbiologists and virologists. Participants will be alerted by email that the scenario is available.
Scenarios will typically represent a case study based on a real patient. Participants will be invited to suggest a diagnosis or a differential diagnosis, to recommend diagnostic interventions (both microbiological and non-microbiological, where appropriate), to interpret any results that are provided, to advise on therapeutic interventions and to identify any infection control and/or public health implications. The responses will be presented in a multiple-choice format. The scenarios will alternate monthly between those of a virological nature and those of a general bacteriological nature. Parasitology or mycology scenarios will be provided periodically; these will be at a level that is appropriate to microbiologists who are not specialists in these fields. There will also be occasional scenarios that specifically concern antibiotic susceptibility testing or antibiotic assays.
Participants will have one month to respond and website entries will close no earlier than 23:59 GMT on the last day of each calendar month. Responses should be from individuals. Although consultation with colleagues is an important part of routine diagnostic practice, responses must be made without conferring as the maximum educational benefit is obtained when interpretive EQA schemes provide feedback on the opinions of individual participants, rather than those of their colleagues. Participants may choose not to respond to a particular scenario if it concerns a sub-speciality in which they do not practice. Participants will therefore be able to declare a case to be outside their normal practice (and should be prepared to justify this). To do this, a formal ‘out of scope’ response must be entered on the website within the normal response period; it cannot be entered retrospectively once the distribution has been closed.
On the day after the deadline, the intended answers (representing a consensus of members of the scheme panel), will be posted on the website. Individual reports, together with the anonymised responses of all of the participants and an educational commentary, will be available on the website approximately two weeks later. Participants will be alerted by email that the report is available.
Participants are expected to respond annually to at least 66% of distributions that they deem to be within the scope of their practice. (UK NEQAS should be informed of any long term absence from work leading to lower participation). An annual record sheet will be available to participants who respond to at least 66% of distributions. This may be used as evidence of participation, e.g. for laboratory accreditation.
Scoring and performance assessment:
In order to be eligible for scoring, answers to a question must achieve a consensus of 80% of participants (and there must also be no evidence that the popular response is incorrect). Individual reports will be produced for each participant following each distribution. These reports will show scores for individual questions and provide charts of performance rating and total scores from previous distributions (as provided for laboratory UK NEQAS EQA schemes). The annual record sheet will record participation and scores achieved for all distributions. It is envisaged that this report will be used by participants during their annual appraisal.
PR enables participants to quantify how much better or worse their performance is compared to other participants. This is a method of ranking. PR is the number of standard errors your score lies above or below the mean performance for participants undertaking the same distributions and is calculated using the formula:
(Cumulative score for participant (X) – Mean cumulative score for participants of the same distributions (Z))
Standard error of mean cumulative score of all laboratories for the same specimens (K)
Therefore you could have a record of very good performance and a high PR if the average score for participants is low, then in the following distribution if the average performance of participants is high and similar to your performance your PR may drop. See here for further detail on calculations. By the very nature of a mean, it is inevitable that some participants’ cumulative scores will be below it on occasions. However, consistent performance below the mean, or downward trends, are potential evidence of problems that need to be addressed. Examine the plots of your performance rating over time to identify the trends. A performance rating of more than 1.96 standards deviations below the mean indicates possible poor performance. Every year, the results obtained between the periods April to March are reviewed. Participants also receive record sheets showing their results in all specimens examined in this period. This is a good opportunity to review performance over the year and to see if there are any problem areas with particular pathogens or samples that have been overlooked.
It is intended that participants use the data provided on their report to monitor their own performance and to take any action required as problems are revealed. A ‘back-up’ procedure is used to bring errors to the attention of participants where for any reasons the above measures have been unsuccessful. This procedure is only applicable to UK and Irish clinical laboratories. The procedure used for the scheme is as follows: every 12 months, each participant’s performance during the previous 12 months is evaluated at UK NEQAS Microbiology. If analysis shows a participant’s total score to be more than 1.96 standard errors below the mean and if analysis following the next 12 month’s participation shows a participant’s total score again to be more than 1.96 standard errors below the mean then the results are presented to the National Quality Assessment Advisory Panel (NQAAP) for Microbiology. NQAAP is responsible for promoting, coordinating and protecting high professional standards in EQA and encouraging the development of clinically relevant and appropriate EQA schemes.
At this stage the NQAAP members are not aware of the names and addresses of the participants concerned, which are identified only by their individual code number. The Panel members then review the performance of these participants and if they agree that performance can be considered poor, the Scheme Organiser is requested to send a letter and a copy of results to the participants concerned. Although the Panel members write the letter, they are unaware of the identity of the laboratory and the letter is therefore addressed and posted at UK NEQAS Microbiology. The letter points out the poor performance in the period concerned and offers help and advice, which can be obtained by contacting any Panel member. If a participant is a poor performers in 3 or more consecutive 12-month periods, and if the participant concerned has not already contacted the Panel, the name and address of the participant is revealed to the Panel Chairman or, if the laboratory is from the Republic of Ireland, a representative from the Royal College of Physicians, Ireland. The Chair or representative writes personally to the consultant in charge, again offering help and advice.