Our ability to forecast accurately is heavily dependent on the quality of the data we can use. In this box we explained how statistical and expenditure data could be distorted by problems in delivery of the benefit, and the difficulties this creates in identifying emerging trends in the data.
Although administrative data from benefit systems are usually of higher quality than survey-based sources, they are susceptible to changes in how benefits are administered. If such changes create real-world effects (such as changes in award rates) then this should not be problematic. But if those changes simply affect how the real world is measured, this causes difficulties in understanding trends, and can result in incorrect conclusions being drawn. Most benefits have a degree of error in their measurement – as shown by comparison of spending data and statistical data – but this is only a problem for forecasting if this error is inconsistent over time.
A particular issue arises when backlogs build up in the administrative system, whether caused by lack of capacity within DWP, its contractors, or claimants taking longer to respond with information than they should. If the claim is successful, then benefit will be paid from the date of claim onwards (unless the qualifying period has not been met), but it will not be recorded in the statistics as being a claim in payment for the period before the claim decision is made. Similarly, if a claimant appeals successfully – which can take even longer than the initial claim process – the benefit will be awarded back to the date of initial claim, but will not appear in the statistics for the backdated period.
If the undercount is inconsistent over time, this could give a misleading impression of underlying trends. This was a particular issue in the early days of PIP. Clearance times – between the point of registration for a claim and the decision on the claim being made – peaked in July 2014 at 42 weeks, compared to an average of 12 weeks for the period since mid-2015.a Consequently, the number of claims in payment in mid-2014 would have been substantially understated in the statistics relative to the actual number of claims that eventually received a payment for that period. Based on registrations and clearances data, we estimate that the data for May 2014 undercounted the caseload by around 110,000 (on a measured caseload of 68,000).
Administrative backlogs also affect spending, potentially resulting in it being recorded in a later period than that to which the spending relates. Benefit spending data record payments only when legal entitlement has been established, which is when a decision-maker has awarded the benefit. Any backdated benefit, for however long it covers, is accrued at that decision point. Spending figures can consequently include substantial arrears and, if these vary between years, may also give a misleading impression of trends in spending. For PIP, backlogs built up during 2013-14, pushing spending into 2014-15, but far less would have been moved from 2014-15 into 2015-16 as the backlog of outstanding claims was more than halved between March 2014 and March 2015. Spending in 2014-15 would therefore be higher due to arrears relating to 2013-14, with 2013-14 appearing lower as a result. Arrears can also result from mandatory reconsiderations and appeals, and vary according to how long these parts of the process take.
Data on arrears payments are not routinely available from DWP systems, other than for ESA, so they have to be inferred from statistical sources. Arrears are typically more volatile than overall spending, being related to a flow of decisions and awards, together with a variable number of weeks’ backdating, rather than a stock of claimants receiving regular payments.
These potential distortions are greater for carer’s allowance, which is dependent on a successful claim for a disability benefit. A claim might not even be lodged until the disability benefit has been awarded, but entitlement can still be backdated to the start of the qualifying claim. The caseload will thus be understated by more, and even measures of outstanding claims will be inaccurate as many will not be made until well after effective entitlement commences. During the early days of PIP, claims for carer’s allowance fell (having been on a continuous upward trend previously), but bounced back once the PIP claims backlog had been reduced.
This box was originally published in Welfare trends report – January 2019