The trial and error of policy experimentation has long been recognized as a useful tool for governments in modern states. A government's motivation for launching regional experimentation rather than a unified national policy has been debated among scholars. Conventional theories on policy experimentation either highlight the autonomy of local authorities and the importance of local knowledge in decentralized systems that contribute to economic leapfrogging, political innovations and policy learning and diffusion,Footnote 1 or else accentuate the roles of central government designers who regard policy experimentation as a way to demonstrate policy correctness,Footnote 2 identify uncertainties or errors,Footnote 3 or to recognize successful local practices.Footnote 4 In other words, research to date has mainly focused on the relationship between the central and local levels but has paid little attention to the role of the public in policy experimentation, especially with regards to potentially controversial policies. In this study, we argue that policy experimentation can be designed as a crucial step in communicating with the public and building a social consensus for future reforms.
We consider a quasi-experimental trial policy in China, the “pension insurance pilot scheme in urban areas,” which offers a counterfactual case of the effect of policy intervention on the attitude of the general public towards welfare responsibility allocation and regime support. This experimental pilot project was conducted against a background of welfare retrenchment in socialist and post-socialist countries starting from the 1980s and the 1990s, where the old enterprise-based, state-financed social security systems were gradually reformed and adapted to fit with market economies. However, the public has not reached a consensus regarding the relative position of the state and individuals in terms of social security provision. The public may oppose or not easily adjust to the rapid change, and a different scenario of welfare provision may cause anxiety and deep mistrust of the regime. Government authorities are therefore under pressure to take adaptive steps by issuing reforms to mitigate public concerns. The Chinese central government launched three waves of urban pension insurance experimental programmes across different provinces in the 2000s. We argue that by using policy experimentation as an instrument with which to communicate with the public, the government can reshape public opinion on the shared responsibility between the state and the individuals for social welfare provision.
We collected randomly pooled data on over 5,000 residents in 8 treatment and 12 control provinces from two nationwide household surveys (“Chinese attitudes towards inequality and distributive injustice,” CAIDI hereafter), conducted in 2004 and 2009. We can evaluate the effects of the pension policy pilot using the survey rounds conducted before and after the policy implementation. The empirical results show that the pilot launched by the Chinese central government has significantly affected citizens’ understanding of shared responsibility and privatized social risks in general – that is, the public appears to accept the promoted emphasis on individual responsibility for pension contributions. The exposure duration of the policy amplifies the attitudinal change of personal perception of individual roles in shared responsibility. Moreover, we observe that official news coverage at the local level, which emphasizes the image of an omnipotent and generous government owing to its dependency on socialist rhetoric and the varied hedged efforts of local governments, has acted as a moderator of the treatment effect of the pilot scheme on the public's understanding of shared responsibility. However, we also find that the intensity of local official news referring to the policy during the experiment immediately consolidates political trust in the short run while posing a challenge to government credibility in the long run.
Policy Experimentation as a Communication Instrument
Policy experimentation is the process of conducting moderate and manageable policy changes within enough space for the government to learn from experiences.Footnote 5 In comparison with a unified, simultaneous national reform, policy experimentation is less costly because it is more exploratory and reversible and has a higher degree of fault tolerance.Footnote 6 It has been used in wide-ranging policy fields and in diverse institutional structures.Footnote 7 For instance, policy experimentation is used in the EU to solve common problems, such as environmental issues under the cooperation of territorial authorities and central units.Footnote 8 In some other cases, private sectors are encouraged to implement innovations when responding to the oversight of the public sector on issues such as food safety and local public goods provision.Footnote 9 In the context of China, some studies have argued that the central government's policymakers have a large capacity to determine the themes and details of experimentation, whereas other evidence shows that local units’ active participation drives policy innovation.Footnote 10
Existing studies, however, have taken little account of the public when considering most experimentation with social and public policies. Support and objections from the public can be critical, particularly for policies related to the redistribution of social and economic resources, regardless of the type of political institution. A lack of social consensus on the policy reform may diminish the effectiveness of the new policies and lead to the obstruction of further reforms. Practitioners use several methods to increase the public's acceptance and support for new policies, including improving information delivery, such as transparency and social construction via media,Footnote 11 and strengthening communications and interaction with the public through expert consultation, citizen participation and deliberative, collaborative or co-productive governance techniques.Footnote 12 Some discussions on policy experimentation cover changes in public opinion;Footnote 13 however, the literature has yet to theoretically consider that the pilot policy itself can act as an instrument of communication between the government and the general public.
Our study links the policy experimentation and political support literature in a way that enables us to accentuate the possibility of using policy experimentation as a tactic for communicating with the general public and building a social consensus for further reforms. First, the idea of experimentation as a communication tool differs from the conventional understanding of policy experimentation in the theoretical spectrum, which essentially focuses on central–local relations, by highlighting the general population and bringing in the importance of the state–individual relationship in the policy process. Second, innovation in the form of policies being launched is a strong signal from the central government to the communicated audience – that is, the general population – on its reform direction and preferred solutions for certain social problems. Policy experimentation can convey specific information,Footnote 14 reshape the appreciated virtue, institutional context and social environment, and thus change the expectation and preference of the general public.Footnote 15 Third, for the purpose of communication, the randomization of site selection is not particularly important, although randomized control trials are still the best way to conduct policy experimentation. Policymakers sometimes intentionally select pilot points that have specific features. For example, if a country plans to promote a new shale gas development scheme, where the main concern of the new facilitators is risk allocation along with potential negative externalities,Footnote 16 then the policy might start from the pilot points with fewer obstructions and less disapproval of the policy from residents rather than be launched in places that are randomly selected within all technically feasible sites. Fourth, promoting a policy by experimentation also demonstrates a new form of evidence-based decision making. Evidence gained through actual policy experimentation in the domestic or internal environment can be more convincing to the public than proof of a policy's effectiveness derived from the theoretical rationale of experts or from experience in other localities.
Controversial Reforms: Privatized Social Risks and Shared Responsibility in China
In the late 20th century, socialist and post-socialist countries were faced with the demands of relieving the government of fiscal responsibility while boosting economic efficiency. Under the market-oriented reform, people's social rights to income and social security were gradually re-framed as individual rights rather than as collective rights as in the previous system. Similar to the “retrenchment” and “risk privatization” process in Western welfare states, individuals are given flexibility but with increased responsibility for managing various social risks related to their personal lives.Footnote 17 China, along with other socialist countries, launched large-scale market reforms, where the privatization of “the public” became a state policy.Footnote 18 Market-induced competition has led the state and urban collective enterprises to reduce or renege on pensions, health payments and housing for employees,Footnote 19 and the government has promoted joint responsibility as the practical tenet behind funding, welfare provision and social security regulation to fit with the socialist market economy.Footnote 20 The urban pension insurance reform acted as an important segment of welfare responsibility reconstruction in the process of moving away from the state socialist welfare model and rebuilding the social security system.Footnote 21
Many studies have carefully examined the details of the urban pension policy during this period, including the return rate, coverage and return on investment of pension trusts.Footnote 22 However, the effects of the aforementioned transitional process on changing social beliefs, especially the public perceptions of the state–individual relationship, have not been adequately investigated. The rhetoric of “socialism” dictates that the state or state-managed public sectors are the source of social security.Footnote 23 As a result, there remained a strong sense of state dependency, attachment to welfare states and organized stakeholders favouring the welfare setup among the population of socialist and post-socialist states.Footnote 24 China's pension reforms of the 1990s and 2000s therefore may have resulted in a gap between the state's actions and individual perceptions regarding the role of government in social security provision.
Since the 1990s, the original pay-as-you-go system for urban pension insurance has gradually changed into a mixed two-tier system comprising social and individual accounts. The transformation officially started in 1997 when the State Council issued its “Decision on establishing a unified system of basic pension insurance for enterprise employees” (Document No. 26),Footnote 25 which announced that the responsibility for raising funds for the new pension system should be shared among enterprises, employees and the government.Footnote 26 However, individual accounts were often “empty” owing to the insufficient allocation of funds and the diversion of funds to social accounts. This situation also caused a “common pool” problem in that the current pension contributors expected the social account, and ultimately the state budget, to cover their pension payment responsibility at all times.
Policy experimentation has been considered as an important working method for “mass line” and “democratic decision making” in China.Footnote 27 As a key working method, experimentation could be a functional choice in reducing the hastiness and cost of the implementation of new policies, winning mass support and expanding the social foundation of the new policies. In the case of pension reform, to further clarify the division between the pooling of individual and social accounts as well as to cover the deficit in individual accounts, the central government issued Document No. 42 in December 2000 to promote a new reform of “Fully funding the individual accounts.”Footnote 28 The pilot policy reform was first implemented in Liaoning province in 2001. It specified that contributions to individual accounts had to be provided solely by employees and at a rate of 8 per cent of their salary. In 2003, the pilot policy was extended to Heilongjiang and Jilin, which adopted similar policy schemes that differed only slightly in terms of regulations regarding the contribution rate. In 2005, the central government issued its “Decision on improving the basic pension system for enterprise employees,” which added eight more provinces to the second wave of the pilot scheme – Tianjin, Shanxi, Shanghai, Shandong, Henan, Hubei, Hunan and Xinjiang – from the beginning of January 2006.Footnote 29 Jiangsu and Zhejiang joined the third wave in 2008. Overall, 13 provinces participated in the pilot reform.
Despite the highlights in central initiatives, the risks of policy flaws and resistance from the public during these pilots are unevenly distributed within the hierarchical structure: central units act as the main initiators of policy experimentation but local governments act as the frontline practitioners who implement the pilot policies. The public is therefore more likely to blame local governments for the flaws and deficiencies in policies. Local governments may take precautions to hedge the potential risks of local residents being dissatisfied and critical of new pilot policies. For instance, local units may magnify the necessity and urgency of policy changes, particularly by connecting them to short-term social problems.Footnote 30 Local news reporting on a controversial policy may emphasize where the government's duty and credibility are increased but overlook the areas where the government has withdrawn from its old role, or reporting may glorify and exaggerate the benefits of new policies to the public, especially among the target groups of those policies.Footnote 31 The different dynamics of policy experimentation between the central and local governments may coexist in the long run. Such desynchronized motives across levels of governments in China may lead to prompts from the central government and even criticism of the local implementation of pilot schemes.Footnote 32
In the case of the policy experimentation of “fully funding the individual accounts,” local governments implement the policy, adjust the details of policy content and promote the reform through the local official media. Local newspaper articles make an association with the pilot policy and the omnipotent and generous role of the state as well as the notion of “shared responsibility.” The messages of the government's generosity, efficiency and conscientiousness in ensuring social justice, the framing of a “good government” and the government taking “people's livelihood into account” – all of which are concordant with socialist rhetoric – fill the news. The emphasis on “national finance” and “subsidy” suggests the government's generosity and ultimate responsibility in managing and solving relevant problems. These narratives seem to be in tension with the reform directions of the new pension scheme but are in line with the tradition of the official socialist news rhetoric and the rationale of local governments under the hierarchical political trust in China. We provide detailed qualitative evidence of the pilot policy and local news coverage in Online Appendix B.
Testable Hypotheses
To examine the causal relationship between the trajectory of the social security reform of shared responsibility and the change in individuals’ perceptions, we propose a set of testable hypotheses on the basis of the policy effect of this quasi-natural experiment brought by the pension pilot scheme for enterprise employees in the 2000s. The main research interest is the change in individuals’ attitude towards the locus of responsibility (LoR) of certain social security benefits and the trust given to political institutions across regions and periods. Individual perceptions on social security responsibility allocations are sensitive to the changes in related social policies and are of great importance in understanding political support in general.Footnote 33 Sharing the responsibility for pension insurance contributions among individuals, the market and society is one of the goals of the pension insurance reform. For the central government, the ideal micro-level outcome of the reform is the general population's recognition of individual responsibility for pension contribution and elderly care, thereby achieving a sustainable system of pension contributions for future retirees. Thus, we argue that the experimentation discussed in this article implies an increase of individual responsibility that can lead to changes in public opinion. We propose our first hypothesis as follows:
Hypothesis 1: The implemented policy experimentation of the basic pension insurance reform increases the popular acceptance of individual responsibility on elderly care in general. The longer the public experiences the experimentation, the more intensely they are affected by the experimentation.
The emphasis on incorporating individual responsibility into elderly care, including pension contribution and other types of investment, differs from public expectations during the egalitarian-socialist period, which accentuated and glorified the duty of the state. Qualitative evidence shows that local official newspaper coverage of the pilot policy normally associated it with the generous role of the state and its untiring efforts in managing and solving relevant problems. Reporting in local official newspapers that describes the pilot reform with socialist rhetoric can act to offset the potential negative effects of the rapid change in welfare policies. Drawing on these arguments, we test the following hypothesis:
Hypothesis 2: A higher intensity of policy coverage in local newspapers that emphasizes the generous role of the government offsets the policy experimentation's effect on public perceptions and shifts the people's perception of governmental responsibility.
The socialist rhetoric emphasizing the state's generosity may keep the public's faith in the short term; however, in the long run, individuals will likely recognize what the reform actually delivers and may even resist its implementation.Footnote 34 In the case of the pension insurance policy reform, although coverage in the local official media highlights the government's efforts to improve the public pension system, individuals reportedly have been faced with paying increasing individual contributions to their pension, which was still not fully funded, and have encountered difficulties in claiming benefits upon reaching retirement age. Thus, the reality may change public perceptions of age-related risks, social security and the state–individual relationship.Footnote 35 This mismatch between official media and policy experimentation may undermine the public's confidence and trust in government institutions over time. Thus, we propose the following hypothesis regarding the changes in the public's political trust brought by the intensity of local news coverage of the pilot policy:
Hypothesis 3: In the short term, a higher intensity of coverage in local official news that emphasizes governmental responsibility regarding the pilot policy increases the public's support for the regime. However, in the long run, a higher intensity of socialist rhetoric in local news can decrease the public's support owing to its mismatch with the policy experimentation.
Data and Variables
Our measurement of dependent variables (DVs) has benefited from two rounds of household surveys, entitled “Chinese attitudes toward inequality and distributive injustice,” which were conducted in 2004 and 2009.Footnote 36 The two surveys used the global positioning system for randomized sampling.Footnote 37 The sample pool of the national adult population included respondents aged 18–65. The total observation from the two surveys is 6,119. Table A1 in Appendix A shows the description of each province and survey round of individual observations. Given that the three provinces in north-east China launched the pilot policy before 2004, we drop the samples of these three provinces from the dataset, thus constraining the analytical samples within the window of the two surveys (2004 and 2009). This modification produces 5,280 observations from 20 provinces (8 treated and 12 control). All variable descriptions, data sources and summary statistics can be seen in Table 1.
Table 1. Variable Descriptions, Sources and Summary Statistics
The two core questions in the survey that have been used to construct the DVs for hypothesis testing are as follows:
• (DV in hypothesis set 1): Between the government and the individuals, who should take greater responsibility for old-age support (including pension contribution, elderly care and other elderly related investments)? (The larger value indicates greater individual responsibility.)
• (DV in hypothesis set 2): Do you trust the central, provincial or local governments? (Asked as separate questions, the larger value indicates higher political trust.)
In addition to the DVs, the two-round survey also provides demographic information about the respondents, which is then used for control variables in our models. Variables include age, gender, educational attainment, marital status, Party membership, household income level and residential registration (hukou 户口) status, among others.
The main independent variable in our study is the different waves of the pilot policy. The full pilot policy started in 2001–2003 in three provinces and expanded across a further eight provinces in 2006 and two more in 2008. The construction of treatment variables will be explained in the model identification section.
To measure the independent variable of local official news intensity of the pilot policy, we gathered newspaper articles from the “China knowledge resource integrated database,” which contains data starting from 2000.Footnote 38 Official newspapers published by the local provincial Party committee were selected since the rhetoric of provincial Party newspapers can assist in constructing public opinion in provinces in the 2000s. Official attitudes regarding the current welfare policy that are revealed in provincial newspapers are disseminated and appear in other media platforms across provinces. People who do not read or subscribe to official provincial newspapers are also informed about such attitudes. Data were collected using keyword searches and manual selection. To capture the intensity of news coverage of the pension insurance reform, we collected articles containing the exact name of the pilot policy (i.e. “fully funding the individual accounts”) to construct the variable “news intensity” and use the ratio of the variables rather than absolute numbers.Footnote 39 Moreover, we calculated the accumulated ratio in 3 or 5 years (
$Ratio = {{\sum {( {Article\;of\;Pilot\;Policy} ) }_t} \over {\sum {( Total\;News\;Articles) }_t}}$, where t equals 3 or 5 years before the two survey years of 2004 and 2009, respectively) to capture the long-term effects of the news intensity.
To compare the provincial-level covariates between treatment provinces and the rest of the country, we collected provincial-level data on social and economic variables, which can influence the possibility of certain provinces being selected as pilot provinces and the public's perception of pension insurance, from the National Bureau of Statistics for the period covering 2000 to 2010.Footnote 40 The selected social and economic variables include regional economic performance, demographic characteristics, fiscal revenue and expenditure distribution, implementation and participation rate of pension insurance, and so on.
Identification Strategy of Policy Effect
The nature of the pilot policy and the two rounds of survey data permitted us to adopt the difference-in-differences (DID) model for estimating the average policy effect on individuals through counterfactual inference. The DID model is commonly used in policy evaluation in the political and public administration literature.Footnote 41 We provide detailed justification of the counterfactual DID design in Appendix B, including parallel trends, individual and regional balances. In the DID model, we define the treatment group as all the samples in the provinces that have participated in the pilot, whereas others are used for the control group. Our baseline model is to estimate the difference between the treatment area and the control area before and after the policy experimentation.
$$LR_{it} = \alpha + \beta _1Pilot_i + \beta _2Post_t + \beta _3Pilot_iPost_t + \beta _4X_{it} + P_i + \varepsilon _{it}, \;\;\;$$where LRit denotes the individual's attitude towards the LoR of pension insurance; Postt is a dummy variable that equals 1 for year 2009 and 0 for year 2004; Piloti is the treatment variable that equals 1 for samples in pilot provinces and 0 otherwise; the β 3 of the interaction term between Piloti and Postt is the average treatment effect on individuals; and Xit is the vector of control variables that is employed to capture minor imbalances in demographic factors that can interfere with the interested outcome. Given that the pilot sites selected are at provincial level, we include a dummy variable P i for provinces to ensure that the selections do not lead to an overestimation of treatment effects. We further analyse the occupational and residential differences by using different subsamples.
In addition to the dualistic treatment-or-control variable, we also code a continuous variable denoted by Duration to capture the gradual feature of the policy implementation by substituting the variable Pilot. Duration corresponds to the length of time each treatment province had implemented the pilot by the time the post-treatment survey was conducted in 2009, and the value is set as 0, 1 or 3. Duration also captures the slight policy differences between two different waves: the full scale of funding the individual accounts by local governments and the financial subsidies from the central government vary slightly across the three waves. The model with “duration” as the explanatory variable is similar to Model (1).
$$LR_{it} = \alpha + \beta _1Duration_i + \beta _2Post_t + \beta _3Duration_iPost_t + \beta _4X_{it} + P_i + \varepsilon _{it}, \;\;$$where Durationi is the time length each sample experienced the pilot policy.
In investigating the mixed effect of the pilot policy and the local official news intensity of the pilot policy, we further constructed the difference-in-difference-in-differences (DDD, or triple difference) model as follows:
$$\eqalign{LR_{ijt} = &{\rm }\alpha + {\beta _1}{Pilot_i} + {\beta _2}{Post_t} + {\beta _3} {NewsIntensity_j} + {\beta _4} {Pilot_i} {Post_t} + {\beta _5}{Pilot_i} {NewsIntensity_j} \\ & + {\beta _6} {Post_t} {NewsIntensity_j} + {\beta _7} {Pilot_i} {Post_t} {NewsIntensity_j} + {\beta _8} {X_{ijt}} + P_i + \varepsilon _{ijt}{\rm ,}}$$where NewsIntensity j represents the local official news intensity of the pilot policy. The coefficient β 7 of the interaction of pilot effect (Pilot*Post) and news intensity thus catches the concurrent effect of pilot policy on the outcome variable accounts for the different intensities of local official news. We contain the same control variables in vector Xit and province dummy P i.
To discuss the change in the public's political trust, we use the question asked in the 2009 survey to measure the level of Chinese citizens’ trust in the central government, provincial government and local government (county or district). This is because no questions related to political trust were asked in the 2004 survey. Short-term and long-term variations in local news intensity are identified by the accumulated ratio of articles that contain the exact name of this pilot policy in the previous year, three years and five years. We constructed the following model by using the interaction between pilot policy and local official news intensity to catch the marginal effect on the public's political trust in the treatment provinces as follows:
$$Trust_{ij} = \alpha + \beta _1Pilot_i + \beta _2NewsIntensity_j + \beta _3Pilot_iNewsIntensity_j + \beta _4X_{ij} + P_i + \varepsilon _{ij}, \;\;$$where the marginal effect of local official news intensity on political trust is calculated as:
$$ \displaystyle{{\partial ( {Trust_{ij}} ) } \over {\partial ( {NewsIntensity_j} ) }} = \beta _2 + \beta _3Pilot_i.\;$$Thus, the coefficient β 2 + β 3 × 1 indicates the estimated marginal effect on the public's political trust in the pilot provinces. Here, the dichotomous Piloti can be replaced as the continuous Durationi (to be discussed in the empirical section), which then turns the measurement into the marginal effect of local official news intensity on political trust for an additional one year. Using Durationi helps to identify the long-term and short-term effects of local official news intensity in spite of the stepwise pilot policy.
Pilot Effects on the Locus of Government Responsibility
We present the DID regression results of the public's attitude regarding the LoR on pension policy by using Pilot, Post and the interaction between Post and Pilot along with other control variables in Table 2. Clustered standard errors at the provincial level are reported in parentheses. Our key object of interest, the coefficient of the interaction (Pilot*Post), shows a significant positive effect (0.126), which indicates that the policy in the treatment provinces after the pilot has increased public acceptance of individual responsibility for general old-age support. After controlling for province, year and individual demographic variables, the effect remains positive (0.103) but insignificant. This mixed effect can be partly addressed in the following section when discussing the DDD effect of the pilot policy and local official news intensity.
Table 2. Policy Effects on Locus of Government Responsibility
Notes: * p < 0.1, ** p < 0.05, *** p < 0.01. The table presents ordinary least square (OLS) results. Clustered standard errors at the provincial level are reported in parentheses. The estimates of pilot, duration, demographic controls, which include age, age square, gender, education attainment, marital status, Party membership, household income level and residential registration (hukou) status, are not reported. The estimates of constants, provincial dummies and year dummies are also not reported. We provide the full table in the Online Appendix (Table OA1).
As shown in Table 2, the interaction between Duration and Post has a significant positive effect (0.077) after controlling for demographic factors, provinces and year dummies. This result indicates that the people in the provinces who experienced longer pilot policy experimentation are more accepting of their individual responsibility for general old-age support. Thus, Hypothesis 1 is supported.
Mixed effects of policy experimentation and local policy news intensity
As shown in Table 3, after controlling for the demographic features and province dummies, local news intensity shows a contrary effect on the public's LoR conditioning in pilot situations. In other words, people affected by the pilot policy will likely hold a stronger belief in governmental responsibility when exposed to stronger local official news intensity. The result from the decomposed subsample indicates that the “offsetting” effect of local official news intensity is significant for enterprise employees (–0.159) and public sector employees (–0.551) in urban areas. Hypothesis 2 is thus supported.
Table 3. Effect of Pilot Policy and Local Official News Intensity on Locus of Government Responsibility in Pilot Policy
Notes: * p < 0.1, ** p < 0.05, *** p < 0.01. The table presents ordinary least square (OLS) results. Clustered standard errors at the provincial level are reported in parentheses. The estimates of pilot, local news intensity, demographic controls, which include age, age square, gender, education attainment, marital status, Party membership, household income level and residential registration (hukou) status, are not reported. The estimates of constants, provincial dummies and year dummies are also not reported. We provide the full table in the Online Appendix (Table OA2)
Concurrent Effect on Political Trust
Despite the building of an image of a “caring and accountable” government that may have effectively swayed public opinion in the short term, our presented case shows that the divergence between the images in official newspapers and the signals received by the public from the pilot are likely to result in political distrust in the long run. The results of the marginal effect of local news intensity on the public's political trust is shown in Table 4, using the cross-sectional 2009 survey data. The coefficient of the interaction between treatment and news intensity indicates that local official news coverage intensity in pilot areas significantly increases the public's trust in local governments by 0.286 (=0.494–0.208) and in provincial governments by 0.131 (=0.296–0.165) in the short term (one year), according to Equation (5). The change in trust in the central government (0.013) is not as significant as the change in trust in both local and provincial governments. In the long term (three years), local official news intensity significantly but negatively affects trust in local and provincial governments (−0.278 and −0.127, respectively). The change in the public's trust in the central government (−0.012) is insignificant. The coefficients of accumulated news intensity for five years indicate a similar pattern – that is, it affects the public's trust in local and provincial governments (−0.131 and −0.060, respectively) significantly and negatively, whereas the effect on trust in the central government is extremely weak (−0.006). The effect on trust in the central government is clearly not statistically evident, which is reasonable considering that the statistically measured news intensity and pilot policy only take place at the provincial level. Therefore, Hypothesis 3 is supported.
Table 4. Short-term and Long-term Effect of Pilot Policy and Local Official News Intensity on Political Trust
Note: * p < 0.1, ** p < 0.05, *** p < 0.01. The unreported demographic control variables and clustering are the same settings as in Table 2.
In this model, we also distinguish a confounding variable to capture the self-interest factor under the policy effect: number of elderly family members who need care. The coefficients of this variable in the various models suggest that the direct self-interest factor has limited influence on the public's political trust.
Robustness Test
Considering that all the models in the previous discussions are OLS regression estimations, we re-ran the empirical models with ordered logit model for robustness testing. Table OA3 in the Online Appendix A shows that the abovementioned results on the pilot policy effect are robust. In addition, we conducted tests using multilevel models that allow cross-province random intercepts, and the results are consistent with our main models. The multilevel models and the results are shown in Table OA4.
We also conducted a test to identify intergenerational difference by adding retirement as a third dimension along with the cross-time and cross-province difference using data from the urban samples. As shown by the robustness test results in Table OA5, individuals who were retired at the time of the survey favoured the government having more responsibility, but they did not express any significantly different attitudinal changes in terms of pilot policy experience. We also tested the baseline models by adding other confounding variables in Table OA6, such as the expectation of upward mobility and whether the respondents received their pension insurance at the time of survey. It could be argued that individuals with higher expectations of upward mobility are more aware of individual responsibility. However, after controlling for these variables in our models, the effects of the pilot and local official news intensity we proposed and examined in the empirical section did not change.
A logical concern on the difference between the short- and long-term effects of the intensity of local news coverage of the pilot policy on political trust (as shown in Table 4) may arise because of the different waves of policy implementation in the treatment provinces. Therefore, we added another robustness test to show how the changes brought by the news intensity vary across different waves (OA7). Thus, we replaced Pilot with Duration in Equation (4). The marginal effect of local official news intensity on individuals exposed to the pilot policy for different years is calculated by β 2 + β 3Duration i, which indicates the change in political trust for an additional one year in pilot provinces with different intensities of local official news coverage. The results, which are consistent with those in Table 3, further prove our hypothesis on the incremental policy. Although it presents a more complex scenario for short-term news intensity with the different waves of the pilot, if we accumulate the news intensity effect for longer terms (for example, three and five years), then the negative change in trust in local and provincial governments greatly increases with prolonged exposure to the pilot policy. We provide the full data for Table 2 and Table 3 in Table OA1 and Table OA2, with validations generated from data with multiple imputation in Tables OA8 and OA9, which further support the robustness of our main results.
Conclusion
Policy experimentation has been widely used when delivering reforms, testing undecided policies, encouraging policy innovations and facilitating policy diffusion and regional coordination within the framework of central–local relations. The function of policy experimentation in modern states can be further extended to a state–individual relationship. Our study argues that when policies concern a redistribution of social and economic resources and risks, conflicts of interest and ideologies among the population may prevent these policies from being implemented as unified, national ones. Experimentations that are used in these cases might permit policymakers to promote policy changes in a manageable way to demonstrate the determination of the government and send signals regarding the direction of the reform. Therefore, policy experimentation can be used as an effective communication tool to convince the public to change its policy preference.
China's pension reform presents an effective case of consensus building through policy experimentation. The central government has designed the policy experimentation to serve as an instrument through which the state can communicate with the public to achieve social consensus on a social security system with hybrid contributors. Alongside the policy experimentation designed by the central government, the coverage of the pilot in the media, with variations in local intensity, presents an omnipotent and generous image of the government so as to maintain the public's faith in the regime's capacity and governance. Although some scholars have argued that the pension pilot scheme has been a failure in the sense that the issue of empty individual accounts has yet to be completely resolvedFootnote 42 and the reform has stagnated owing to the financial concerns of local governments,Footnote 43 our study shows its effect on individual attitudes. The experimentation has managed to change people's perception regarding welfare responsibility.
We have also found a hidden hazard within the reform process: the disjunction between the launch and implementation of policy experimentation and official media rhetoric can lead to public distrust in local governments in the long run. In other words, to serve as a tool to communicate with the public, policy experimentation needs to be accompanied by an appropriate and constantly adjusted narrative and social construction. Moreover, our study highlights issues that require further discussion around the effectiveness of using social policies to garner political support, such as ensuring consistency between what the policy promises, what the policy delivers and what the policy has been described as in the media in the long run.
The experimentation on urban pension reform, despite the potential hazard, was crucial in building a social consensus on who should contribute to social security and in preparing the target population for further reforms in China. China's successive pension reforms, such as the New Rural Social Pension Insurance (launched in 2009) and the Reform of Pension Systems for Public Sector Employees (launched in 2015), were all implemented quite smoothly and without any strong resistance from rural residents or public sector employees. These pension reforms all follow the rationale and design of shared responsibility in pension contributions. The changes in public opinion on privatized social risks, expectations of individual responsibility and the retrenchment of the role of the state that were forged through the process of policy experimentation with the urban pension insurance reform has effectively prepared the way for the building of a hybrid social security system.
Our theoretical argument can be applied to address issues in other policy areas and different time periods in contemporary China. It is common to see obstacles and resistance to a policy from different social groups, sectors and even individuals.Footnote 44 As China's “reform and opening up” enters its fourth decade, large-scale reforms can be challenging, especially those that rely on a reshuffle/redistribution of benefits/interests. In the past 20 years, policy experimentation has been frequently used for reforms and policy innovations that have required the cooperation and support of the general population – for instance, the policy experiment for affordable housing, which was launched in 2013 and expanded in 2022,Footnote 45 and the low-carbon city pilot scheme, which was initiated in 2010 with the second and third waves conducted in 2012 and 2017, respectively.Footnote 46 Another example is the property tax reform. When faced with considerable resistance from local governments and the public, the central government decided to initially launch the pilots in two metropolises in 2011.Footnote 47 In 2013 and most recently in 2021, the initiative concerning a larger scale of pilots was promoted by the national congress and has gradually changed the expectations of the public regarding the housing market.Footnote 48
Moreover, the communication mechanism identified in the case of China's policy experimentation also has the potential to address reforms and policy innovations in other contexts. For instance, the responsibility-reallocation process in China is similar to the hybrid welfare reforms rolled out in Western Europe in which the traditional welfare state model has been replaced with a residual welfare state model. In transitional socialist countries other than China, social policies are also following the privatization trend as the previous socialist welfare model has been gradually replaced with a mixed welfare model in which individuals share more responsibilities. The common root of social dissatisfaction and unrest during all these transitions is that the social consensus may not accord with the politicians’ reform designs. Thus, the exploration of the policy process as a potential communication mechanism – policy experimentation, in our case – will ultimately help to explain the politics of policy from a broader perspective.
Acknowledgements
We thank Martin Whyte, Shen Mingming and Yan Jie for sharing the survey data. Earlier versions of this article have been presented at seminars and workshops organized by Tsinghua University, University of Oxford, Fudan University, Taiwan University, Academia Sinica, Renmin University of China, Huazhong University of Science and Technology, among others. We also especially thank Wu Yu-Shan, Leng Tse-Kang, Huang Minhua, Chen Shuo, Meng Tianguang and Lü Xiaobo for their helpful comments. This research was sponsored by the High Level Project in Arts, Humanities and Social Sciences of Tsinghua University (2021TSG08101) and the Innovative Research Group Project of the National Natural Science Foundation of China (71721002).
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0305741023001273
Conflicts of interest
None.
Appendix A. Data Explanation
Table A1. Individual Data
Appendix B. Counterfactual DID Design: Selection of Pilots
Individual Balance
The legitimation of counterfactual DID design relies on the parallel trend assumption, which assumes that the counterfactual “natural” change in the outcome for the units in the treatment group between time 0 and 1 would have been the same as the change in the outcome for the units in the control group between periods 0 and 1. In this study, it specifies that people's attitudes towards welfare responsibility allocation or political attitude in treatment provinces would have been the same as those of people in the control provinces, if not the policy experiment, or as shown in formula E[Y 0(1) − Y 0(0)|D = 1] = E[Y 0(1) − Y 0(0)|D = 0], which is drawn from the derivation of the average treatment effect on the treated estimation under a DID design. Given that the parallel trend assumption is not directly testable, especially for two periods of data, we address this assumption with several approaches. First, we run a simple t-test of our main outcome variable (LoR, a large value indicates considerable agreements on individual responsibility) of the 2004 survey samples (i.e. at period 0) by the treatment and control groups. E[Y(0)|D = 1] and E[Y(0)|D = 0] indicate no significant difference (p = 0.51).
Second, treatment during the policy pilot programme is arguably exogenous for individual preferences and political attitudes. That is, ΔY 0 should be independent from the assignment of D. Most sites of policy experiment in China are not randomly selected, and potential bias with certain confounding factors may affect each province's probability of being selected as a pilot. Nevertheless, an in-depth case study on the development of the Chinese pension system argues that the urban pension insurance pilot scheme is designed by the central government, which carefully considered the issue of representativeness in site selection (Zhu and Zhao Reference Zhu and Zhao2021). We conduct the following data description and balance check of key determinants in the selection of pilot regions to empirically address the identification challenge of DID. Third, we run the baseline models depicted above with various control variables to minimize the bias brought by potential confounders. Finally, we leverage the time effect conditional upon different groups (such as different intensities of news) with triple difference models.
Time Trend
Although we cannot display the long-range development of individual perceptions owing to data constraints, we can present the variance of provincial-level covariates that may affect the outcome variables. We use the panel provincial data from 2000 to 2010 and compare the aggregated long-term trends of the treatment and control provinces on economic development, demographic features and social conditions. As shown in Figure B1, the trends of the two groups are nearly parallel to one another on most of the indexes.
Figure B1. Time Trend of Provincial Indexes
Regional Balance
Scholars typically find it difficult to identify the selection process of the pilot provinces. Several latent factors can cause potential self-selection bias in the process. For instance, provinces affected by the aging problem in terms of demographic structure are more likely to be chosen as a pilot, but provinces with good fiscal and economic performance will likely be better in deploying the reform. Selection based on unobservable information, such as the motivations of provincial leaders, which varies across provinces, is also possible, and the motivations may have changed as the tenure stages of the leaders changed. Although we are constrained by the availability of insider stories, we can still rule out the endogenous problems through statistical analysis by using observable data (Gentzkow Reference Gentzkow2006).
Considering the nature of the selection process for pilot provinces, we first construct a provincial sample pool through sampling without replacement to address the potential selection bias. For each wave, denoted by year t, we use the social and economic data in year t–1 and then code the province selected for the pilot pool as 1 and other provinces as 0. In the selection of the next wave of pilot provinces, the previously selected provinces are dropped from the selection pool. In other words, a province that has started to implement the pilot policy is not compared in the next wave of pilot selection.
We then conduct an event history analysis (EHA) of the significant variables while satisfying the requirement of the variance inflation factor test to measure the imbalance between selected and unselected provinces. The results are shown in Table B1. Both the time discrete result and the time series result indicate that the difference between selected and unselected provinces is insignificant in terms of economic performance, fiscal condition, demographic situation and existing pension insurance system (column “All sample”). The three provinces in north-east China possess some specific features compared with the other provinces: a larger proportion of SOEs, longer history of industrialization, more severe problems with outflow emigration, and so on. Fortunately, these provinces are part of the first pilot wave and thus are not covered by the individual-level data in our study. Thus, we conduct EHA test dropping on three provinces (columns “2004–2009 sample”). As shown by the result, the difference between the selected and unselected provinces in the second and third waves has been largely reduced. Thus, even if the provinces that participated in the pilot were not the de facto ones in the random selection, they are still statistically representative.
Table B1. Event History Analysis for Balance Test
Xufeng ZHU is professor and dean at the School of Public Policy and Management, Tsinghua University, China. His major research interests in China studies focus on the policy process, think tanks and public governance. He is the author of seven books. Recent publications include articles in Journal of Public Administration Research and Theory, Public Administration Review, Public Administration, Governance, The China Quarterly, Policy Studies Journal, Public Management Review, among others.
Yan WANG is a lecturer in social statistics at the department of social statistics, Manchester University (from December 2023). Previously, she was a lecturer in digital sociology at Lancaster University and research fellow at the School of Public Policy at the London School of Economics and Political Science. Her research seeks to understand the issues of state legitimacy, public opinion and the redistribution of public goods. Her recent publications include her book, Pension Policy and Governmentality in China: Manufacturing Public Compliance, and articles in journals including Information, Communications and Society and the Journal of Chinese Political Science.
