Executive summary

Statistics New Zealand has estimated that since 1996 increases in outputs of the public sector have largely been associated with increasing labour inputs. In the education sector, for example, the average annual increase in output of 1.0% between 1996 and 2015 was composed of average annual labour input growth of 2.5% while labour productivity fell on average by 1.5% per annum. These data use standard aggregate productivity methods and are not part of the National Accounts. They involve no explicit quality adjustment. This is important as it is difficult to fully understand productivity data (especially trends over time) without considering the impact of changes in quality.

Yet while important in principle adjusting public sector productivity data for quality changes is complex in practice. As an example, the United Kingdom Office for National Statistics (ONS) has had to revise its approach to quality adjusting education quantity when practices regarding students sitting exams changed. This paper thus estimates a range of quality adjusted productivity measures and discusses the benefits and risks of different approaches (e.g., regarding teacher salaries, students’ performance in tests, or impact on earnings). The measures are illustrated with data on schools.

Why education?

A focus on the productivity of the education sector is consistent with a desire to help upskill the economy (Atkinson, 2005) and the Better Public Services programme. This is also a topic of interest to researchers concerned with productivity measurement more generally given the variety of messages that emerge for this sector from different approaches to productivity analysis. While national accounts data show declining labour productivity in the education sector as a whole a number of cross country studies (largely focussing on schools) have suggested that the New Zealand education system performs relatively well internationally (e.g., Afonso and Aubyn, 2005; Sutherland et al., 2007; and Schreyer, 2010). More recent work (Dutu and Sicari, 2016), has however, suggested that New Zealand may have fallen back to the middle of the pack.

This investigation on quality adjustment and school productivity could thus help with interpretation of any evidence of a lagging productivity performance in the public sector, including whether this tells us something about the public services themselves or more about the measures being used.

So what difference could quality adjustment make?

Estimates of labour and multifactor productivity including quality adjustments for New Zealand schools were developed using data at the sector-level (the measures and key results are summarised in Table 1). To illustrate the potential of existing data – and also to allow these measures to be replicated – emphasis was given to using data from publicly available sources.

These data illustrate both the importance and the difficulty of quality adjusting sector-level productivity data. Policy decisions (e.g., regarding smaller class sizes) are reflected in the basic labour productivity measures. Further, when the measure of labour input is adjusted in an effort to capture quality changes (e.g., through using data on teachers’ salaries) this labour productivity performance also worsens. But there are caveats to this. These caveats include questions over the use of salaries as a proxy for quality of inputs – particularly given the nature of public service labour markets (e.g., whether a change in salaries reflects quality or compositional changes) and the importance of missing inputs such as the previous performance of students (needed for measures of value added).

Nonetheless, a similar story emerges from measures that adjust outputs based on attainment in international assessments (such as New Zealand students’ PISA scores), where performance has Working paper 2017/02 9 worsened. This reflects a decline in aggregate PISA points (an average annual decline of 0.1%), which itself reflects a larger fall in the average PISA score (an average annual decline of 0.3%). However, there are differences in measured attainment according to international and domestic assessments. Indeed, (labour) productivity based on a measure that adjusted for domestic attainment (e.g., the proportion of students leaving school with at least NCEA level 2 (or equivalent)) increased between 2002 and 2014. A related measure (the series using school revenue as a measure of inputs) was used to compare the results in this paper to those of the Office for National Statistics (ONS) in the United Kingdom.

Finally, measures were adjusted for final outcomes (in this case the performance of school leavers in the labour market).