Entrustability – what is it?
Most professions initially developed based on trust. Examinations were rare. Doctors (and other professions) learn through experience. For healthcare, as science advanced and increased in importance, examinations in basic sciences were introduced. Once qualified, further advance depended upon the trust of patients and other doctors. Even when I trained, the only examination after qualification was the MRCP. All other junior posts were obtained because other doctors were prepared to say that one was competent and trustworthy. The next phase (1997 and 2020) involved a more formal assessment of competence in a defined range of activities. However, as early as 2005, a move started towards establishing entrustability, trusting a doctor to undertake complex and prolonged activities. Having left trust for the apparently more sound idea of competence, we have returned to trust, but now trust is based on evidence. This post focuses on three articles in the July 2021 special issue of Academic Medicine, summarised in this overview. (here) A later post on assessing entrustment adds more helpful information.
Table of Contents
Context - education and its outcome
Education and rehabilitation are similar in many ways – (here) – and another similarity is that the essential outcomes are challenging to define and measure.
In rehabilitation, the critical outcomes are, ultimately, determined by the patient. They usually relate to social roles, interactions, emotional state, and satisfaction. In healthcare education, the critical outcomes are defined by two groups: those who employ healthcare professionals; and those who depend upon the person’s professional expertise, usually but not only patients. The quality of the outcome relies on a mixture of interpersonal skills, specific knowledge coupled with the ability to use it well, and specific technical skills.
I trained under an apprenticeship model. The consultant or consultants I worked with taught, and I learned, but there was no assessment. When applying for a post, they would, one hoped, say that one was competent and safe, and ‘good at the job‘. Other less relevant factors also influenced decisions. In educational terms, this is equivalent to saying that a student ‘attended the course and seemed to learn’.
Ten years later, when I started training and educating trainees, the emphasis was on teaching and assessing competencies. The assessments could have been more specific. The trainee’s competence was usually rated by reference to expectation:- being below, at, or above the level expected ‘for the stage of training‘, but the expected levels were never defined or explained.
There was a gradual move to measuring competence in terms of being competent to undertake the activity independently, where rating this was a realistic option. This, in turn, led to more attention being paid to activities that could be measured at the expense of other essential activities.
Two problems arise. The first is the increasing number of competencies identified and specified. The recent rehabilitation medical syllabus (here) identified 39 competencies, many of which could include many lower level competencies, which could also be defined and measured.
The second problem is more serious. As professional expertise becomes increasingly atomised, the learner and the educator need to remember the real purpose of being competent at a discrete activity, which is to undertake some complex and unpredictable high-level activity. In rehabilitation, for example, a person might be adept at selecting and using many individual assessment tools. Still, they might need help to assess a patient appropriately to allow the formulation of the situation because the person needs to appreciate the reason for the assessment.
Another example comes from 25 years ago when I was an external assessor evaluating a speech and language training course.
The external assessors met with recently qualified graduates of the course who all said that their first few months working in the NHS had been terrible because, for example, they had no idea how to run an out-patient clinic, nor could they diagnose and manage a patient who did not have a classical and isolated language impairment. The presence of cognitive losses, additional dysarthria, tearfulness, and incontinence completely overwhelmed them. Training has moved on since then!
High level outcomes
One response to this excessive focus on discrete activities was to reframe accreditation as being qualified as achieving a few ‘high-level outcomes’, activities that encompassed many individual activities within a complex whole.
A high-level outcome in rehabilitation would be for a patient to undertake a shopping trip. This requires a person to have many skills – planning and organising, remembering, communication and interpersonal skills, mobility and dexterity etc.
In an educational setting, the expert rehabilitationist should be able to draw up a plan to guide rehabilitation over a few weeks or months. This depends upon knowing about the disease and its prognosis, knowing or discovering the available rehabilitation options, establishing the patient’s priorities and wishes, using information from all other team members, and prioritising the team’s actions.
In medical rehabilitation training, the training outcomes have now been restricted to 14 high-level outcomes in each speciality. Six outcomes are generic, the same in every medical speciality. The remaining eight outcomes define the speciality. These speciality-specific outcomes are discussed elsewhere on this site. (here and here)
The essential characteristics of these high-level outcomes, known in medical specialities as Capabilities in Practice, are that they:
- are complex, integrative activities that typically take a long-time (hours);
- involve a large number of separate competencies;
- usually include working with other people; and
- are not easily measured using any single measure.
This last characteristic – the difficulty in measurement – poses a challenge because patients, employers and regulators all wish to know that the patients will receive a safe, effective, and efficient service from the doctor.
Entrustability
Concerns about using assessed competencies as the best way to evaluate doctors in training emerged about 15 years ago. Gradually the idea of entrustment emerged. An overview of entrustment published in 2016 gives information about its evolution and basic features. (here) The essence of the arguments is that:
- patients necessarily are vulnerable and cannot control or supervise the doctor’s actions; so
- patients must trust doctors always to maintain their (the patient’s) interests and safety.
At the same time, senior doctors undertaking training have to trust that their trainees (and, indeed, other medical and non-medical colleagues):
- will always act to maintain the patient’s interests and safety; and
- will ask for help or support when they need support.
The trainer will gradually entrust work to the trainee, allowing them to work independently. When they have sufficient evidence, through experience, that indeed the trainee can be fully trusted at the activity, then the trainer is said to have grounded trust, and the trainer can say that the doctor is entrusted with undertaking that activity,
This system of assessing a doctor’s expertise does not imply that competence is unimportant. To undertake the complex, high-level activity safely and successfully, the doctor will need to draw on a whole range of knowledge and skills. A doctor training in respiratory medicine would never become entrustable if they were not competent to place a chest drain safely. However, just because they can put a chest drain, they would not be entrustable. They also need, for example, to judge when it is needed or not needed, what the risks and benefits are for the patient, what to assess over time, when and how to remove it etc.
Entrustability requires a whole range of knowledge and skills that cannot easily to assessed and measured using a framework of individual competencies.
Entrustment in practice - purpose
One of the papers has an interesting discussion on how the assessment of entrustability is perceived by trainees and how these perceptions can be very different from that of the trainer. (here) The authors suggest that assessments should be considered on three axes, and for each to think: to what extent does the trainer and to what extent does the trainee consider the assessment lies on the
- Formative – summative spectrum; to be educational or to be evaluative?
- low stakes – high stakes spectrum; to matter in terms of progression in or completion of training
- ad hoc – structured range; undertaken relatively informally or within a clear set of rules.
They stress that problems are most likely to arise from different perceptions. Additionally, they point out that any high-stakes assessment must be, and must be seen and agreed to be, carried out within a transparent, fair set of rules. They suggest the need to:
- Align the process with the purpose. The less structured an assessment, the more it should be formative, perhaps as part of ongoing, longitudinal coaching;
- Align stakes with purpose. Only summative (evaluative) assessments should be high stakes, and all formative assessments should be low stakes;
- Align process with stakes. The obvious corollary is that the process needs to be appropriate for the importance; any high-stakes assessment should be planned and undertaken within a clear set of rules.
Because a trainee is vulnerable and often uncertain, it is crucial always for a trainer (assessor) to discuss explicitly with the trainee the purpose and stakes associated with an individual assessment. The more a trainer focuses on reducing hierarchy and creating a learning environment, the less likely problems will arise.
This does not mean that a low-stakes assessment is wholly discounted. Even low-stakes reviews will be considered when reaching an overall judgement, but there will be (should be) a large number, so the stakes associated with each evaluation are low.
Finally, the authors also review the words summative and formative and suggest a subtle but significant change in assessments: an assessment of learning (summative) and an assessment for learning (formative). The word will remind both the trainee and the trainer of the purpose of the appraisal.
Entrustment in practice - words or numbers
A second paper considers the same matter (summative-formative) from a different perspective; how to report the outcome of an entrustment assessment. They point out that there is a natural tendency to quantify results using numbers, even if comments are ‘allowed’.
Their conclusion is set out so well in the summary that I have reproduced it here:
“Finally, the authors address the tyranny of documentation created by programmatic assessment and urge caution in yielding to the temptation to reduce words to numbers to make them manageable. Instead, they encourage educators to preserve some educational encounters purely for feedback, and to consider that not all words need to become data.” (here)
The typical work-based assessment form two components: a scale, words or numbers (i.e. summative), and a space for comments (i.e. potentially formative). The authors report that 98% of the work-based assessment in psychiatry had no words, only numbers, despite the space for comments. I have not surveyed UK assessments, but many I have seen either have no remarks or bland comments such as ‘doing well’.
Numbers (or verbal quantification such as ‘good’, ‘satisfactory’, and ‘below expectation’) have two major weak points. They are often managed mathematically, which is usually invalid. More importantly, they give no information on the assessment’s context or circumstances. The number cannot be interpreted reasonably.
Comments can offer much more information:
- the context of the assessment;
- additional qualitative information about performance;
- justification for or explanation of any quantitative score;
- feedback to the trainee on performance
- what went well
- what might have been done better or differently
- suggestions to the trainee on changes or further learning
The authors provide evidence that comments, far from being unhelpful and subject to bias, are often more informative about a person’s standard than the quantified outcome. They are also consistent and reliable.
Several changes and improvements are suggested.
Simply placing the comment box above or before the quantification box increases the use of comments. Second, each comment box should be labelled with its purpose, such as contextual information, performance information, and educational feedback information.
Third, the authors suggest that, in the US, but I also suspect in the UK, we are focusing too much on documenting training assessments as evidence for judging entrustability, and that if used correctly as part of the process of learning, then too much information is available for review when making an overall judgement on entrustability. They refer to “the tyranny of documentation”. The solution suggested is to identify a proportion of work-based assessment as purely educational (formative).
In other words, they are suggesting:
- all assessments should use comments, but comments should be focused on the purpose:
- if summative, then on context, quality, and explanation
- if formative, which will be the more frequent purpose, then the comment focus on learning
- There should be enough higher stakes, summative assessments to ensure a proper holistic evaluation, but only a few to be reviewed.
- Numbers, or single words summarising the standard, should be avoided or interpreted by reference to comments that should be made first.
Implementation
A third article discusses the implementation of entrustment as a means of judging progress and achieving sufficient quality to practice independently. (here)
It draws on experience in the Netherlands, where a plan to reduce expenditure on training was necessary. One approach was to reduce the length of training. The medical establishment suggested that a blanket reduction was unwise, as it did not allow for variation between trainees. Instead, a goal of reducing the average length of training by introducing (a) entrustment decisions so that trainees could progress once entrusted and (b) flexibility in the training programme so that training duration could be shortened when appropriate but allowed to continue if necessary. A third strand was to make decisions as dependable as possible.
One fact to note is that, in the Netherlands, rehabilitation has 12 core entrustable professional activities, but the number varies from six in pathology to 30 in medical genetics. Emergency medicine has seven, with 17 other sub-specialist activities. In the UK, all specialities have the same six generic capabilities, and rehabilitation has eight speciality capabilities.
The article details the work undertaken by three project groups between 2015-2018; it was a substantial undertaking. The main conclusions appear to be that:
- an electronic portfolio was vital; with it, the project would succeed. They now have one.
- flexible, time-variable posts could be arranged, but it required a significant involvement of all parties, including health providers;
- the change will only succeed if the culture of all parties changes to support more trainee-centred training;
- one significant cultural change is to allow trainees more autonomy in their practice, while trainees. Using entrustment may assist this change.
The system in the Netherlands for certifying the entrustment of activity is summarised in the article (here):
“During the rotation, the resident collects documentation for achievements in the e-portfolio. When the resident feels ready for a summative entrustment decision for an EPA, he or she submits annotated proofs of achievement to the Clinical Competency Committee (CCC). The committee meets and considers five elements of the request:
- meeting the required knowledge, skills, and behaviour;
- clinical experience and relevant exposure related to the EPA;
- any tests relating to the EPA;
- educational and scientific activities undertaken related to the EPA;
- committee members’ personal experience and judgment concerning the trainee.
Most information about 1–4 is derived from the e-portfolio. In addition, the committees are advised to attend to the general criteria that have been acknowledged as important for entrustment:
- capability (specific knowledge, skills, experience, situational awareness),
- integrity (truthful, benevolent, patient-centred),
- reliability (conscientious, predictable, accountable, responsible),
- humility (recognises limits, asks for help, receptive to feedback), and
- agency (proactive toward work, team, safety, personal development), acknowledging that a well-grounded decision does not only weigh “objective” criteria but also aims to include intersubjective judgment.“
Conclusions
Entrustment can be used to evaluate all assessments of clinical activities, from those currently termed ‘competencies’ to those presently termed ‘high-level outcomes’ (i.e. Capabilities in Practice).
Entrustment is a judgement made by another senior, experienced doctor (in medical training) based on relatively formal structured work-based assessments (of discreet activities) or, for high-level outcomes, on a holistic review of the totality of the evidence. This holistic evaluation should be run according to general rules and must consider comments; comments may be more valuable than actual categorical classification. More information on evaluating entrustability is given in another blog post.
All work-based assessments should be recorded on forms that place comments before any performance classification. Further, both trainer and trainee should agree beforehand whether a work-based assessment is primarily developmental, educational, or evaluative. Any evaluative assessment should have a clear structure. To reduce information overload when judging the entrustment of high-level outcomes, it would help to identify evaluative work-based assessments as a separate category.
In the UK system, where entrustment is only used with high-level outcomes and not to work-based assessments, the forms evaluating or reporting on work-based reviews should be restructured to encourage comments. Comments should cover the context, justification for and/or explanation of the evaluative categorisation if given, and words should provide helpful feedback on a trainee’s performance.
This post has explored three articles on using entrustment to evaluate the performance of doctors (and other healthcare professionals) during their training. It has emphasised the importance of agreeing on the purpose of any work-based assessment and structuring any evaluative, high-stakes assessment. Most reviews should be considered developmental, an integral part of a programme of learning; as part of this, the importance of using written comments has been highlighted, and the risks associated with scores have been emphasised. The question of a variable training programme that is adapted to the speed of the trainee’s learning is still open. Still, suppose a variation is to be considered. In that case, the healthcare system needs a cultural change to allow trainees to be more autonomous at the later stages and to allow more flexibility in starting and ending jobs.
