In our 2018 Welfare trends report, Chapter 4 described how we model the effects of universal credit on spending. This draws heavily on two models owned and operated by DWP - the Policy simulation model (PSM) and the Integrated forecasting model (INFORM). PSM is a static micro-simulation model that uses Family Resources Survey (FRS) data to analyse policy changes. This box detailed how the FRS is used in PSM and some of the issues that raises for our UC forecast.
This box is based on DWP data from 2015 .
The PSM is the main micro-simulation model used by DWP to analyse policy changes. It is based on the annual Family Resources Survey (FRS), which details benefit income streams alongside information about the circumstances of each ‘benefit unit’.
The FRS is the best available source for modelling benefit entitlement, but has some limitations. As a self-reported survey, it relies on claimants (and interviewers) providing accurate responses. Several shortcuts are taken to reduce the burden on respondents, which might otherwise affect the sample size. For example, comparing FRS results with administrative data on welfare spending suggests that ESA, tax credits and housing benefit income tends to be under-reported. For broader earnings, the PSM uses an FRS variable that reflects claimants’ interpretation of their ‘usual pay’, which is unlikely to be a perfect match for the earnings relevant to calculating UC awards. Net income is also required for the UC calculation and the reporting of tax and National Insurance payments are known to be less robust in the FRS.a
FRS data are published with a lag of two to three years after collection. Incorporating new results into the PSM also takes time. The PSM underpinning our November 2017 forecast was based on data from the 2015-16 FRS that was published by DWP in June 2017. The survey is not therefore able to capture recent developments in the economy (e.g. further falls in unemployment since 2015-16) or policy (e.g. the progressive replacement of disability living allowance with the new personal independence payment). These effects have to be captured via other means – for example by aligning to alternative estimates or off-model adjustments. This adds further uncertainty to the PSM modelling.
The sample size relevant to the overall UC modelling is relatively large (around 4,000), but for some lower-level breakdowns it can be very small (e.g. around 25 benefit units for the ‘WTC-only, non-self-employed, no housing benefit, single’ calibration group, which is expected to account for just over 170,000 cases in the steady-state UC population). Conclusions drawn at these lower levels will be far less robust. But in the absence of administrative data, the FRS remains the best available source on which to base the PSM analysis.
Given some of the known issues with the FRS, and surveys in general, DWP’s PSM team clean the raw FRS data (mainly earnings) and carry out some imputation (capital, income from tax credits, childcare costs) to generate the best possible input to the model. The FRS data are projected forward on a static basis – so that employment states in the base year are held constant, for example – but with sample weights adjusted through calibration to legacy benefit caseloads in each year (using either outturns or our forecasts, as appropriate). Entitlement rates are uprated with announced policy while earnings and other incomes and outgoings (e.g. rents or mortgage interest) are grown in line with the relevant determinants from our latest forecast.
The PSM uses this information to model benefit entitlements under a base system and a proposed change (or ‘scenario’, in this case moving to UC). The difference in individual award between these two model runs is treated as the effect of the policy change.
This box was originally published in Welfare trends report – January 2018