Predicting benefit?

As editor of Clinical Rehabilitation, I see a small stream of studies that attempt to predict who will benefit from a rehabilitation intervention. Terms used include ‘responder analysis’, which assumes that a person who ‘responds’ can be identified and then their characteristics used to select patients for treatment. Separately managers and commissioners want selection criteria, to choose patients who will benefit or to exclude people who will not benefit. Last, teams are always looking for people ‘ready for’ or ‘suitable for’ rehabilitation. These ideas all assume that a patient’s chances of benefiting can be estimated, and indeed categorised (good prospect or poor prospect). Is this true? Is it even likely that it can ever be true?

Involvement in rehabilitation leads to better outcomes for patients with a disability. This statement is supported by much evidence, and applies widely (see here). The question this post considers is, “Are there any criteria that can be used to predict who will benefit from rehabilitation?” An alternative question might be, “Are there any criteria that predict who will not benefit from rehabilitation?”

Within rehabilitation, there are also specific interventions that lead to benefits, in some more restricted area such as degree of spasticity (e.g. botulinum toxin) or arm function after stroke (e.g. constraint induced movement therapy). A similar, if more restricted question arises, “Are there any criteria that predict who will, or who will not benefit from the specific intervention?”

These questions are not unique to rehabilitation. Identical questions apply to many services – intensive care units, stroke units, hospitals in general, and many specialist services – and also to many specific interventions – chemotherapeutic anti-cancer drugs, anti-hypertensive drugs, carotid endarterectomy etc.

For specific interventions, it is sometimes possible, on the basis of evidence, to identify patients who are unlikely to benefit. For example, patients who cannot actively extend their fingers are unlikely to benefit from constraint-induced motor therapy.

However, for the great majority of situations where evidence shows that something is beneficial to a group of patients, it is not possible to say with any certainty who will benefit, and indeed everyone may benefit, some more than others.

The nature of the evidence

To understand the difficulty, one has to understand the evidence. Almost all evidence comes from randomised trials where two interventions are compared (see here), and one intervention is found to be better that the alternative. If the intervention that is associated with the less good outcome is assumed not to be doing harm, then one can assume that the other is associated with benefit.

The evidence arises when the data show a difference between the two groups: a better mean score, or a greater number achieving some category, or a faster achievement of some event. The data tell you that the patients in one group have benefited. The data do not tell you who has benefited.

In most studies, one can assume that the change seen in each group has a statistically normal distribution. In other words, the difference seen is a shift in the distribution of outcome towards a better outcome. It is unlikely that the patients in the experimental groups will fall into two populations, with an obvious and discernible population much better that the remainder of the group who would have the same distribution of change scores seen in the alternative group.

Therefore all that one can say is that, if a person has the experimental treatment, there is a probability that their outcome will be a bit better. For some people that ‘bit’ might be important, but one cannot know whether the important incremental change occurred because of the treatment, or would have happened anyway.

The second important fact is that, in many studies, there is natural change in the patient population. Sometimes the study is in a recovering condition, such as stroke; sometimes it is in a deteriorating condition, such as motor neurone disease; sometimes it is less predictable, and people may fluctuate with some getting better, some getting worse, and others changing little.

Identifying ‘responders’

In that context, it is worth looking at the techniques used to try and predict ‘response’; these methods are supposed to identify people who benefit, so that their characteristics can be studied and compared with non-responders, allowing one to identify criteria to distinguish the two groups – responders and non-responders.

People who respond are usually identified by looking at the difference between baseline measurement and final follow-up measurement and selecting people who changed by more than a certain amount, or people who were in the top 20% (or some other arbitrary centile) of change. In other words, a ‘responder’ is someone with a greater change over time. (In deteriorating conditions, the reverse logic would apply – it would be patients with the least change.)

However these are not ‘those who responded most’, they are simply ‘those who changed most’. And, in as far as there are any factors that are associated with change, all one is achieving is the identification of prognostic factors for change.

Logically, the reality is that, if a group study has shown benefit, then a group has to be treated with the intervention. After giving the treatment, one may decide that the intervention should be stopped or changed on the basis of no detectable change after a suitable duration of treatment. This approach is used for disease-modifying treatments in multiple sclerosis and many other diseases, such as leukaemia.

It is simply not possible, on the basis of evidence, to select patients who are more or less likely to benefit from rehabilitation in general, nor is it possible on the basis of evidence to select people for specific interventions. As with other medical treatments, often one has to start an treatment that has been proven effective, and then modify and adjust management in the light of success or failure.

Predictive value

There is a further logical and statistical difficulty that must never be forgotten, one that has gained prominence in this pandemic (lateral flow tests). In the unlikely event that some factor were identified to predict that someone would (or would not) respond to rehabilitation or to a particular rehabilitation intervention, one must realise the following.

First, even to be considered, a factor would need to be established in one large population (several hundred) and then verified on a second completely separate population. This is unlikely every to occur.

Second, one needs to know the diagnostic characteristics: how may testing positive will truly benefit; how many testing negative will truly not benefit; how many testing positive will actually not benefit; and how many testing negative would have benefited.

Third, these figures will only be valid if applied to an exactly similar population; if the population has a different overall proportion of people who will or will not benefit, then the sensitivity and specificity will be different.

In the unlikely event that there were valid selection criteria, nevertheless, as I pointed out in 2003 (here), one is likely to select for rehabilitation many patients who would not benefit and, at the same time, to refuse rehabilitation to many patients who would benefit from rehabilitation. The article shows the calculations supporting this assertion, in Table one.

The solution

The solution has several parts. The first part is that there must be a sufficiently resourced and flexible expert rehabilitation service to ensure that everyone needing or wanting it can be assessed for their needs, and given advice.

The second part is to allow the rehabilitation service flexibility to make professional judgement about access to parts of the service that are under-resourced. The third part is to ensure that there is a minimal service available to everyone needing it, if only to monitor change and change a patient’s priority for treatment if needed.

At present these solutions are not available in the UK, but they could be (see here).

In this solution, professional judgement is the selection mechanism. Professional judgement is obviously not perfect, and it is subject to many biases and other problems. But, until a better method is developed, it is probably the least bad option and least likely to make terrible errors. Moreover, the decisions should be taken by, or overseen by, a multi-disciplinary team which will reduce the risk of any one individual making biased or irrational decisions.

The last part of this solution is to accept that a ‘trial of rehabilitation’ (or a ‘trail of an intervention’) is a sensible, clinically appropriate policy. At the same time, the team must accept that a patient who is judged not to be benefiting sufficiently from their involvement must be transferred out of the service so that other patients may benefit. Unfortunately this is often difficult, given the unsatisfactory nature of residential care placements and of the support services for people with long-term care and support needs.

In summary, it is not possible on any evidential or rational basis to determine who will or will not benefit from rehabilitation, or from a specific intervention. Every patient should have an expert assessment which should identify their rehabilitation needs. These needs should be prioritised in the context of the available services, and as many of the needs as can be should be met. Whether or not any needs can be met, every person should be given advice. If the rehabilitation service is organised on a District-wide basis (see here), then this approach may well ensure that most patients have many of their needs met. Using selection criteria will lead to a lower quality service and a waste of resources through selecting patients inappropriately.

Make your comment or suggestion here

Scroll to Top
%d bloggers like this: