Changes to Main Data Sources

1 We made a recommendation in last year’s report reflecting our concern that changes the Office for National Statistics (ONS) had made to its data could affect the quality of our analysis. We are pleased that ONS has responded positively. In this appendix we outline the main data sources used in our analyses of employment and earnings and review the principal changes made since our 2008 Report.

2 There are three main sources of data that we use in this report to measure earnings: the Annual Survey of Hours and Earnings (ASHE), the Labour Force Survey (LFS) and the Average Earnings Index (AEI). These are all published by ONS.

3 There are two main sources of employment information: the LFS and the ONS employee jobs series. The LFS captures the number of people in employment, whereas the employee jobs series measures the number of jobs in the economy. This is an important distinction as a person can have more than one job.[1]

Annual Survey of Hours and Earnings

4 ASHE is the main source of structural earnings data in the UK and is regarded by ONS as the best source of earnings information. It provides information on the levels, distribution and make up of earnings, as well as on hours, gender, age, geography, occupation, and industry. It is a survey of employees completed by employers and conducted in April each year. Results are based on a sample of employees in Pay-As-You-Earn income tax schemes obtained from HM Revenue and Customs. The self-employed are excluded.

5 Due to methodological and definitional changes mentioned in our 2008 Report, there are four annual earnings datasets:

• 1970–2003 New Earnings Survey;
• 1997–2004 ASHE excluding supplementary information;
• 2004–2006 ASHE including supplementary information; and
• 2006–2008 ASHE using 2007 methodology.

6 In 2007 the ASHE sample was cut by 20 per cent, which has remained the same for the 2008 ASHE sample.[2] The cut has had a limited impact on the precision of estimates at an aggregate level but it has had an effect at more disaggregated levels, such as at the four-digit industry level. Following discussions with other government departments concerning funding, ONS recently announced that the pre-2007 sample size will be reinstated for ASHE in 2009 and guaranteed this funding to 2011.

Low-pay Weights

7 In our Report, estimates of the number of jobs paid below certain thresholds using ASHE are based on low-pay weights that have been developed by ONS. These weights remove those employees whose pay in the reference period of the survey has been affected by absence and weight the remaining employees up to UK population estimates. Estimates of the level of earnings use the standard ASHE weights. These weight all employee responses up to the UK population and then remove (after weighting) those whose pay has been affected by absence. Our analyses of earnings differ from those available on the ONS website as ONS remove (after weighting) those employees not on what the employer considers to be an adult rate of pay. We include those on trainee rates in our analyses.

Low-pay Statistics

8 According to the provisional 2007 ASHE data published in November 2007, there were 292,000 employee jobs paid below the minimum wage in April 2007. On 14 November 2008, the final 2007 ASHE data revised the estimate upwards to 296,000, which was equivalent to 1.1 per cent of all jobs. The provisional 2008 estimates show there were 288,000 jobs paid below the minimum wage in April 2008, which was also equivalent to 1.1 per cent of all jobs. The provisional 2008 data are therefore broadly in line with the 2007 data.

Average Earnings Index

9 The AEI is based on data from the Monthly Wages and Salary Survey (MWSS) and is regarded by ONS as the best short-term measure of growth in average earnings. It captures changes in the annual (as opposed to monthly) composition of employment between industries because of fixed industry weights. The AEI is available monthly, both including and excluding bonuses. We regard the AEI excluding bonuses series as a proxy to a basic pay series.

Labour Force Survey

10 The LFS is the official data source used to measure employment and unemployment. It is a quarterly survey of around 52,000 UK households conducted on a rolling monthly basis and provides information on employment, earnings, and personal and socio-economic characteristics including gender, ethnicity and disability.

11 In our Report, analyses of aggregate employment, unemployment and hours worked use monthly and quarterly LFS data published on the ONS website. These are seasonally adjusted estimates and are re-weighted to the latest population estimates. For detailed analyses of the labour market by age, ethnic status and disability, we have used the LFS Microdata. These data are weighted to previous population estimates and are not seasonally adjusted. As the latest population estimates are higher than the previous estimates, the analyses based on LFS Microdata produce estimates of levels that are lower than the aggregates published by ONS. Estimates of proportions are not affected as much, as both the numerator and the denominator are lower.

12 ASHE contains no information on disability, ethnic background, country of birth, nationality or education level. The LFS is therefore the source of data on earnings for people with disabilities, ethnic minorities, migrants and people with no qualifications. Data on pay and hours in the LFS may be less reliable than in ASHE because the sample is smaller, people often answer the earnings questions without reference to pay documentation (although they are prompted to consult available documents), and some information is provided by proxy respondents. ASHE collects information from employers about employees’ paid hours, whereas the LFS collects information from individuals about their actual and usual hours of work, which might include unpaid hours. This generally leads the derived hourly earnings variable in LFS to be below the derived hourly pay rate recorded in ASHE. Where a stated hourly rate of pay is unavailable from the LFS, ONS has developed an imputation method using a nearest neighbour regression model, which also takes account of information on second jobs. This new methodology reduces the differences between hourly earnings estimates from the LFS and ASHE.

13 In January 2006, in order to comply with EU requirements, ONS moved to produce LFS Microdata on a calendar quarter rather than seasonal quarter basis. ONS had released only a limited back-series on a calendar quarter basis before publication of our 2008 Report, resulting in a break in the series. ONS has since released an entire back-series of LFS data and there is no longer a break.

Employee Jobs

14 The employee jobs series provides a timely industrial breakdown of jobs in the UK. A number of Short Term Employer Surveys are used to collect data to compile the employee jobs series. The surveys collect information from businesses across the economy. Figures at a more detailed level, however, are available only for Great Britain and are not seasonally adjusted. This makes quarter-to-quarter comparisons problematic, particularly as much of the employment in the low-paying sectors is of a seasonal nature, for example Christmas trading in the retail sector. But comparisons between one quarter and the same quarter a year ago overcome this particular problem.

15 The employee jobs series is published quarterly and is benchmarked annually to the latest results from the Annual Business Inquiry Part One (ABI/1). The annual benchmark moved from December to September in 2006. ONS also introduced some methodological changes designed to improve the estimates of both the levels and changes. This has resulted, however, in a break in the employee jobs series between December 2005 and September 2006.[3]

16 The changes have resulted in a discontinuity in our time series analysis of annual changes in employee jobs between December 2005 (which can be compared with December 2004 on the old basis) and September 2007 (which can be compared with September 2006). This has seriously affected our ability to track the impact of the National Minimum Wage on jobs in the low-paying sectors over time. The ONS is currently undertaking work to provide a back series to fill this gap and we look forward to analysing the data to remove the discontinuity.[4]