Assessment competency
At 02.00 hrs on November 29th, I had an epiphany, “a moment of sudden and great revelation or realisation”. [OED] For many years, I have emphasised a distinction between assessment as a process and assessment as a measure (e.g. “the outcome assessment was the Rivermead Mobility Index”), and I have suggested that both should be called data collection tools. Recently I have started writing about competencies in the Rehabilitation Medicine syllabus, and I was considering number three, assessment competency. The published syllabus guidance (page 19) focuses on measurement. I have also posted a page on this site about person-centred assessment.
As I thought about what to write, I realised I was continuing to confuse the assessment and measurement processes. This led me to compare the medical process of diagnosis, which leads to a diagnosis of disease, with the rehabilitation process of assessment which leads to a holistic formulation of the situation. I realised that they were opposites. Medical diagnosis aims to identify a single cause for a problem, whereas rehabilitation assessment seeks to identify the many factors contributing to a patient’s health problem (illness). One is a closed process with a single answer, the other an open process with no definitive answer. Consequently, I will write two competencies, one on assessment and the other on detecting and evaluating change. This post will expand on my epiphany.
Table of Contents
Background
The published medical Rehabilitation Medicine syllabus was developed by a group of UK medical rehabilitation experts to accompany the curriculum for trainees. It was intended to provide trainers and trainees with more specific guidance on what knowledge was needed, what skills were required, and what behaviours were expected to be considered competent. The syllabus covered all aspects of rehabilitation, and four competencies were concerned with the process. One was an assessment competency.
When developed, the syllabus was not subject to any trials to check how practical or helpful it was. The syllabus is an additional advisory document, and there is no regulatory requirement to use it. It can be revised and adapted.
I intend to publish guidance on each competency on this site, a process I have just started. This means reviewing each competency critically. So far, I have done two out of the 39 research and scholarship and the use of the biopsychosocial model. I started on the third on November 27th and soon realised it needed to be better constructed, which made me think about it more critically, as described in the opening paragraphs.
I was surprised at my difficulty. For many years I had emphasised that an assessment and a measure were used almost interchangeably in rehabilitation despite having apparent differences in meaning; indeed, in 2004, I suggested that we should refer to data collection tools to cover both assessment and measurement tools. I had thought the difference was simple; both assessment and measurement collected data, but assessment interpreted it and quantified it.
When I started to write my piece for this site, I realised that the data collection forms within the assessment process fell into two distinct types. The first was standardised, fixed sets of items that measured something, such as using the Barthel ADL index. The second category comprised aide memoires or checklists, such as identifying whether a house has stairs, someone had access to a car, or their hobbies. Furthermore, I realised that most of the data collected during the assessment was given by the patient in an unstructured way and would not easily be fitted into any form.
Consequences for competencies
I realised that the published assessment competency, which focused on standardised measures, was not an assessment competency and that two separate areas of competency would be needed. The first was to be competent at carrying out an initial or later collection of data sufficient to allow formulation and rehabilitation planning. The second competency was selecting and using appropriate measures, whether as part of the assessment process or as part of evaluating change and the effect of treatments.
This split makes much more sense. The existing syllabus has nothing on being able to evaluate rehabilitation interventions. Indeed, in medical training, I suspect that little time is devoted to discussing how to judge whether a treatment is effective despite symptomatic treatments being used commonly. Evaluation of biomedical therapies is often unnecessary because the approach engenders a categorical judgement – whether the person is cured of their disease. In an era of multi-morbidity and long-term conditions needing symptomatic treatments, unequivocal, binary evaluation is rarely possible.
Consequently, I propose to adapt the published “assessment competency” into an “intervention evaluation competency” and to write a new assessment competency. This will be influenced by the other insight I had.
Medical diagnosis v rehabilitation assessment.
I then realised that my suggestion that medical diagnosis and rehabilitation assessment were analogous processes was flawed or being kinder, only partially true.
It is true that both processes occur early in an episode as a start of the problem-solving process and that both are based on a targeted collection of data working toward a goal. However, they differ markedly in their goals, not simply because one is focused on pathology, the other on disability.
There is a much more crucial difference between the two diagnostic processes. Medical diagnosis looks for a single cause (the diagnosis) within the body, usually a structural or functional abnormality within an organ or organ system. In marked contrast, the rehabilitation assessment looks for the influences that may affect the patient’s situation, with a particular interest in those that can be acted on to improve the situation and those that may give prognostic information. This difference is illustrated in a figure.
Put another way, a medical diagnosis considers a closed problem, with only one answer and a limited range of options. In contrast, the rehabilitation assessment considers an open problem, with no limit on what information might be helpful and no way to say that the assessment is complete; it is always possible to know a bit more.
The person undertaking a rehabilitation assessment is not testing and ruling out hypotheses, progressively narrowing the range of possible answers. Instead, they are exploring an uncharted country, looking for helpful information that adds to the whole. Thus, one vital skill rehabilitation professionals must develop is to know when they have sufficient information and that they are unlikely to have overlooked something that would radically alter the formulation and plan. The profession must learn to accept uncertainty.
Evaluation of treatment
In the biomedical approach to healthcare, there is only one target of specific treatments, the disease, so there is little ambiguity about success or failure.
The evaluation of curative medical (including surgical) interventions is relatively straightforward. If the disease process is stopped, hopefully before too much damage or with repair, it will be successful. Many disease processes can be measured in one way or another, and usually, the outcome can be evaluated within a short timeframe.
Increasingly, people have long-term medical conditions that cannot be cured but can be controlled. Many diseases have reasonably good control markers, and treatment can be evaluated over time.
In rehabilitation healthcare, however, evaluating treatments (interventions targeted at improving outcomes) is much more complex. Treatments are aimed at many different targets. The patient effect of interest varies from patient to patient. Factors other than the treatment will influence whatever is being measured. There are complex interrelationships between different treatments and changes in various components of the biopsychosocial model. Change is slow and may take months or years to be seen.
Despite these difficulties, it is often possible to evaluate the effectiveness of single interventions on single outcomes; the problem arises when considering the more important long-term effects on social integration, well-being, and family relationships.
The critical skill needed in biomedical and rehabilitation practice is using pragmatic n-of-one or single case study designs.
Conclusions
Previously, I have considered the difference between biomedical and rehabilitation healthcare to be one of relative emphasis. Biomedical healthcare is more interested in disease, takes a short-term approach, and pays less attention to disability and social matters.
I now understand there is a categorical difference. Biomedical medicine works within a closed system, whereas rehabilitation works in an open system. Although both biomedical and rehabilitation services are solving problems, there is only one target in medical practice the disease. Still, in rehabilitation, the initial goal is to gain a holistic understanding of the patient’s problems so that several actions aimed at different targets can be started. I will explore this difference in more depth in another blog post.