Introduction
The concept of ‘workfare’ dominates social welfare policy in many countries. In welfare systems based on workfare, recipients of social assistance (SA) receive their allowance on the condition of working, mandatory volunteering, or job training (Ravallion, Reference Ravallion1998; Jordan, Reference Jordan2018; Kampen & Tonkens, Reference Kampen and Tonkens2019).Footnote 1 In such systems, social assistance becomes more conditional, requiring participants to make an effort to find paid employment (Lødemel & Trickey, Reference Lødemel and Trickey2001, p. xii; Jordan, Reference Jordan2018). In the current SA system in the Netherlands, the country of study, recipients receive their SA allowance conditionally on making an effort to seek work. They are expected to accept any available job, leave welfare, and transition to a (potentially) better job from there (Delsen, Reference Delsen2016; Groot et al., Reference Groot, Muffels and Verlaat2019). This line of thought is grounded in various perspectives, including the ‘rational actor’ perspective: when given the chance to receive ‘money for nothing’, recipients will feel disincentivised to leave SA (Edzes et al., Reference Edzes, Muffels, Zulkarnain, Jongen, Sanders, Verlaat and Venhorst2021). This idea of ‘the calculating unemployed person’ entered the Dutch mainstream debates in the early 90s (although estimations indicated that only 20% of SA recipients corresponded to this classification, Spies & Van Berkel, Reference Spies and van Berkel2001). Workfare is furthermore grounded in the notion that work stimulates social integration, making prompt labour market reintegration a matter of self-interest (Lødemel & Trickey, Reference Lødemel and Trickey2001). Consistent with this workfare paradigm and in line with international developments in social policy (Knotz, Reference Knotz2018), a welfare system was built containing many obligations, restrictions, and fines for non-complying Dutch SA recipients. These were further substantiated in the Participation Act, the current Dutch SA law that came into effect in January 2015.
This workfare dominance drew criticism from policy makers, experts, and academics. Scholars raised questions about the effectiveness of this increasing reliance on the workfare paradigm: while studies showed positive impacts of workfare on finding employment, these tended to be of short duration, with job quality and stability suffering from strict conditionality in the long run (Wright et al., Reference Wright, Dwyer, Jones, McNeill, Scullion and Stewart2018; Pattaro et al., Reference Pattaro, Bailey, Williams, Gibson, Wells, Tranmer and Dibben2022). Furthermore and importantly, while tight benefit sanctions did not lead to persistent labour market re-entry, they did seem to cause financial hardship and health problems (Wright et al., Reference Wright, Dwyer, Jones, McNeill, Scullion and Stewart2018; Dwyer et al., Reference Dwyer, Scullion, Jones, McNeill and Stewart2020; Wickham, Reference Wickham, Bentley, Rose, Whitehead, Taylor-Robinson and Barr2020; Williams, Reference Williams2021; Pattaro et al., Reference Pattaro, Bailey, Williams, Gibson, Wells, Tranmer and Dibben2022). Policy makers and professional experts also expressed concerns about the lack of autonomy and choice, the strict, conditional, and distrusting regime of control and sanctions, and the bureaucracy required to enforce this (Westerveld, Reference Westerveld2015; Vliegenthart, Reference Vliegenthart2016). Moreover, this ‘carrot and stick’ approach, with the stick growing bigger and bigger, was considered particularly ill-fitting for vulnerable groups among SA recipients, such as those with a low level of education, mental health problems, and/or a prolonged distance to the labour market (Delsen, Reference Delsen2016). There was also a political desire to look beyond the scope of labour market outcomes, and to consider the regime’s impact on other relevant and important life domains such as health, well-being, and participation (Westerveld, Reference Westerveld2015).
Participation is the main object of interest in this study.Footnote 2 We focus on four participatory activities: volunteering, (intensive) informal care, schooling, and setting up for self-employment – the merit of studying these specific activities is expanded upon in the theory section. Under the Participation Act, such activities are not encouraged since they might interfere with job seeking, and recipients are required to obtain explicit permission from the municipality before engaging in them. However, empirical knowledge on how the strictness or conditionality of the SA regime influences the behaviour of recipients is lacking: does this lead to fewer hours spent on unpaid socio-economic activities USEA)? This study provides causal empirical evidence from a randomised social experiment that includes a pre- and post-measurement. Here, the regular workfare-based SA regime serves as a control group for two alternative, less conditional treatment groups. Such real-world experiments are rare, and as far as they have been conducted in the past, the focus has usually been on economic outcomes, particularly ‘reintegration into the labour market’ (Greenberg & Schroder, Reference Greenberg and Shroder2004). The experiment, from which the data is used for this research paper, did not appear to stimulate entry into the labour market based on measurements taken 1 and 2 years into the experiment (Betkó et al., Reference Betkó, Spierings, Gesthuizen and Scheepers2020; Edzes et al., Reference Edzes, Muffels, Zulkarnain, Jongen, Sanders, Verlaat and Venhorst2021). Our social experiment allows us to answer the following research question (Q1): Is there an effect of a less conditional SA regime on time spent on USEA over time? Additionally, little is known about for whom different types of SA work, and why. Using qualitative data, gathered from participants and officials involved in this experiment, we first generate theoretically informed hypotheses on this issue, after which we empirically test them to answer our second research question (Q2): Do the treatments have different effects on time spent on USEA for different social groups over time, and if so, for which groups are the effects stronger?
Our study contributes to existing knowledge in at least three ways. We provide robust evidence from an experiment that allows for causal inference, demonstrating the effects of less conditional welfare on the hours that welfare recipients spend on USEA. Second, we generate theoretical insights based on qualitative data about who benefits most from a less conditional welfare approach. Third, we empirically test these newly generated theoretical insights, thereby providing quantitative evidence on which social groups experience a stronger impact of the treatments – and are thus better off in a less conditional regime. This is also relevant for policy makers, as it does not only address the question ‘what works?’ but also ‘what works for whom?’.
Theoretical background
General proposition
As addressed above, the regular Dutch SA regime, embedded in the Participation Act, is based on a workfare approach (Groot et al., Reference Groot, Muffels and Verlaat2019). Obligations enclosed in the Participation Act can be broadly divided into two categories. The first is the requirement for recipients to prove their continuous eligibility for SA, obliging them to report earned income, received gifts of any kind (resulting in a reduction of their benefit equal to the value of the gift), and changes in living situation. The second regards reintegration into the labour market. SA recipients are expected, among other things, to actively search for employment (which could be specified as a minimum number of job applications per week), to accept any job offer, to relocate for a job, and to dress appropriately for job seeking. Recipients usually need to request permission for any activities that could interfere with their job search efforts.
Criticism on the Participation Act, and more broadly on the workfare paradigm as described above, has a theoretical foundation grounded in behavioural science, which goes beyond welfare-focused studies. Insights from behavioural perspectives are highly relevant for questions on the functioning of welfare regimes. For instance, studies on mental bandwidth and stress have shown that financial and psychological stress resulting from obligations and insecurity can impair people’s executive functions and long-term perspective, thereby diminishing their decision-making abilities (Mullainathan & Shafir, Reference Mullainathan and Shafir2013; Haushofer & Fehr, Reference Haushofer and Fehr2014). Furthermore, the experienced burden due to administrative hurdles and obligations can lead to stress and reduced mental bandwidth – affecting those with lower financial resources more strongly (Moynihan et al., Reference Moynihan, Herd and Harvey2014; Mullainathan & Shafir, Reference Mullainathan and Shafir2013).
These general findings from psychological and economic behavioural research have strong implications for the functioning of workfare-based SA systems: they imply suboptimal results regarding persistent integration, due to these negative by-effects of administrative burdens and obligations. Many Dutch SA recipients live below the poverty line; they have limited financial resources and experience financial stress and mental health problems (Delsen, Reference Delsen2016; Scholten et al., Reference Scholten, Betkó, Gesthuizen, Fransen-Kuppens, de Vet and Wolf2023). Moreover, empirical findings indicate that the ‘carrot and stick’ regime places a considerable experienced administrative and psychological burden on recipients due to its obligations and restrictions regarding reintegration (Eleveld, Reference Eleveld2020). Experienced stress and reduced mental bandwidth negatively influence willpower and self-control, causing risk aversion and a tendency to be less open to persistent reintegration routes (Haushofer & Fehr, Reference Haushofer and Fehr2014). This presumably creates a downward spiral that makes it increasingly harder for SA recipients to take effective steps. These steps might require openness towards alternative routes, a longer-term effort, and stepping away from the work first paradigm – to eventually land a job and permanently escape unemployment and poverty (Mullainathan & Shafir, Reference Mullainathan and Shafir2013; Haushofer & Fehr, Reference Haushofer and Fehr2014; Moynihan et al., Reference Moynihan, Herd and Harvey2014).
These notions correspond with findings from empirical studies on conditionality and sanctions in welfare, as briefly mentioned in the first section of this study. Pattaro and colleagues (Reference Pattaro, Bailey, Williams, Gibson, Wells, Tranmer and Dibben2022) reviewed a large number of studies on the effects of benefit sanctions, and overall, tight benefit sanctions seemed to cause material hardship and health problems instead of promoting structural labour market reintegration (see also Wright et al., Reference Wright, Dwyer, Jones, McNeill, Scullion and Stewart2018; Dwyer et al., Reference Dwyer, Scullion, Jones, McNeill and Stewart2020; Wickham, Reference Wickham, Bentley, Rose, Whitehead, Taylor-Robinson and Barr2020; Williams, Reference Williams2021). While many studies demonstrate a positive short-term impact on finding employment, job quality and stability actually seem to be negatively affected in the long run (Pattaro et al., Reference Pattaro, Bailey, Williams, Gibson, Wells, Tranmer and Dibben2022; Wright et al., Reference Wright, Dwyer, Jones, McNeill, Scullion and Stewart2018). This corresponds with earlier insights on workfare, which state that ‘…too rigorous application of a workfare regime can have a counterproductive effect, in terms of reducing social exclusion’ (Spies & Van Berkel, Reference Spies and van Berkel2001). All of these considerations lead to the general proposition that an alternative SA regime – one that reduces administrative burdens and psychological stress due to poverty, and thus enhances mental bandwidth and executive functions such as willpower and self-control – would possibly enable SA recipients to make more effective choices. While the general proposition is that alternative SA regimes could improve a wide array of outcomes for SA recipients, this study specifically focuses on the time spent on USEA, which is likely influenced by the treatments. This will be elaborated upon in the following sections.
Participatory activities
In our research, hours spent on USEA are a measure of (changes in) hours spent on schooling, setting up for self-employment, volunteering, and informal care together. Each of these activities is considered ‘valuable’, as they could either serve as stepping stones towards permanent entry into the labour market, or as contribution to the collective good. Schooling is a well-known contributor to human capital that increases chances of finding a job, and becoming self-employed to make a living is, if successful, a direct way to end unemployment and SA dependency, providing the income is sufficient. Volunteering is sometimes also considered to improve employability, and although this is contested (Paine et al., Reference Paine, McKay and Moro2013), it does serve a wider purpose of contributing to the collective good and it promotes social integration (Wilson, Reference Wilson2000). As such, it is stimulated by both local and national governments (Movisie, 2014). Finally, under the flag of ‘self-reliance’ and contributing to the ‘participation society’, the government started to encourage citizens to engage in informal care (e.g. long-term care for a sick family member or friend outside one’s own home), while downscaling formal care: people who need care should, in the first place, rely on their own network. As such, providing informal care is an officially encouraged activity that benefits communities. All activities are included in the ‘participation ladder’ of the Association of Dutch municipalities. This tool for municipalities classifies six steps for citizens between ‘isolation’ and ‘employment’ (the highest step), exemplifying the relevance and importance attached to these activities, regardless of paid employment status (Terpstra, Reference Terpstra2011).
Hypothesis average treatment effects
The criticism on the Participation Act and the general proposition that a less conditional SA system, based on trust and autonomy, could be a general improvement compared to a workfare approach, led to a number of similar randomised social experiments in Dutch municipalities – sometimes seen as part of the global wave of experiments with welfare, based on or inspired by the universal basic income (Groot et al., Reference Groot, Muffels and Verlaat2019; Gielens et al., Reference Gielens, Roosma and Achterberg2023). In these Dutch real-life experiments, the Participation Act regime serves as a control group and is compared with alternative, less conditional treatments that give recipients more autonomy (Groot et al., Reference Groot, Muffels and Verlaat2019; Betkó et al., Reference Betkó, Spierings, Gesthuizen and Scheepers2019; Edzes et al., Reference Edzes, Muffels, Zulkarnain, Jongen, Sanders, Verlaat and Venhorst2021). In the experiment in Nijmegen, which took place from December 2017 to January 2020, two alternative treatments were compared with the control group under the Participation Act. Participation was voluntary and open to most people receiving SA (see Appendix 1 for more information on recruitment and eligibility). All participants gave their consent to participate in the experiment and could withdraw at any time. Appendix 2 provides more information about the ethical considerations around studying human subjects in an experiment, as well as the informed consent signed by the participants. After eligibility screening, applicants were randomly assigned to one of three groups, one of which was the control group.
The first alternative treatment group (the exempted group) was granted an exemption from all reintegration obligations and associated fines for non-compliance. Consequently, their administrative burden was reduced, potentially increasing mental bandwidth. They were given trust and autonomy by the municipal government. They could contact the municipal social service or the regional reintegration office for assistance if they wished to, but were not obliged to do so.
For the second treatment group (the coached group), all reintegration obligations as mentioned in the previous paragraph were replaced with intensive (monthly) group coaching. Participation in this coaching was the only obligation participants had, thereby reducing their administrative burden and potentially increasing mental bandwidth. They had full autonomy in deciding whether they wanted to look for a job, a part-time job, become self-employed, or engage in volunteer work, as well as where and how to pursue these activities.
Additionally, if they found temporary or part-time work, participants in both groups were allowed to keep some extra income compared to the control group, giving them an opportunity to reduce financial stress. Appendix 3 outlines more details on both the treatment as usual, received by the control group, and the two experimental groups.
After outlining the overarching theoretical framework and the treatments, we can now connect the treatments to the USEA of our focus: schooling, setting up for self-employment, volunteering, and informal care. In one way, this connection is very direct: since participants in the experimental treatments are no longer required to dedicate a specific amount of time to job seeking, they can spend this time on other activities. Furthermore, the regular rules require that permission is sought for all these activities, which is not granted if the reintegration office believes that the activity may hinder job seeking efforts. Additionally, both treatments potentially alleviate financial stress by permitting a small job in addition to the allowance. They are also less conditional and demanding than the regular regime, as they have no or fewer obligations regarding reintegration and no threatening fines in case of non-compliance – something which obviously has a profound impact on people living on a minimum income. This should decrease the psychological burden (Moynihan et al., Reference Moynihan, Herd and Harvey2014) and increase mental bandwidth (Mullainathan & Shafir, Reference Mullainathan and Shafir2013), enabling people to spend more time and energy on long-term planning and benefitting themselves and/or society, whether through volunteering, informal care, schooling, or setting up as self-employed. Altogether, this leads to the hypothesis H1: participants in both treatment groups will spend more hours on USEA compared to the control group.
Qualitative exploration: who might benefit?
Sampling and analytical strategy
Besides presenting a unique test of how SA regimes influence the time recipients spend on USEA, we aimed to generate and test new theoretical insights regarding the specific subgroups for which these alternative regimes might have relatively strong effects on time spent on USEA. Since it seems unlikely that the treatments affect all SA recipients in the same way, we explored if there are subgroups, based on personal characteristics, which benefit more from the treatments. People receiving SA form a very specific (and vulnerable) group, often not envisioned when theories on social and economic participation are formulated. The response of different subgroups within this population to welfare policies has certainly not been theorised. To fill this gap, we collected qualitative data before and during the experiment, allowing us to generate theoretical insights ‘bottom-up’, in order to explore why people did or did not spend time on volunteering, schooling, informal care, or setting up as self-employed. Our qualitative sources are broadly divided into two categories. The first comprises interviews with professionals who had contact with a large number of (potential) participants. The focus of these interviews was to collect general information on the implementation, successes and failures of the experiment. These sources, including the number of people interviewed and the timing of these interviews during the experiment, are described in Appendix 4, which also contains an example of an interview guide. The second category of qualitative data is derived directly from the participants: each survey contained an ‘open question’ at the end, allowing participants to mention anything of interest to them. About two-thirds of the participants used that option in each of the surveys, with responses varying from a single-line remark up to hundreds of words. To analyse these data, we used hybrid iterative coding (Vennix, Reference Vennix2011). More details about the analytical strategy can be found in Appendix 5.
Generating new hypotheses
Among the characteristics that potentially impact the strength of the treatment effects, age and education emerged as recurring factors. Recruiter 3 mentioned in her interview that reasons for people to participate differed by age and education: ‘… the somewhat higher educated people, they were not mainly in it for the [opportunity to earn extra] money. More like “…, we have the opportunity for some self-development, and more opportunities in general”. I also frequently noticed this among the older people…’.Footnote 3 The contact person from the municipal social service also stated that generally, most people participated because they wanted more freedom and fewer obligations. However, specifically younger participants were more interested in working part-time to earn extra income in addition to the SA allowance.
Similarly, in the first focus group with interviewers who conducted the face-to-face interviews with participants, it was said that ‘being above a certain age … was mentioned a lot’ by participants as a reason for not being able to find work. This was repeated in answers to the open questions in wave 1 and 2, and one participant explicitly argued this was a reason to look for volunteer work. These indications are relevant as they suggest that higher educated and older people were more interested in the opportunities for self-development, indicating a stronger intrinsic drive to perform USEA (regardless of a monetary gain). There would be more room for this in the less conditional treatments. Furthermore, older people, who often struggle finding a job due to their age, might seek out alternatives for work for that reason. This leads to the following exploratory hypotheses:
H2a: When subjected to a less conditional welfare treatment, older participants will show stronger increases in time spent on USEA than younger participants, compared to participants in the control group.
H2b: When subjected to a less conditional welfare treatment, higher educated participants will show stronger increases in time spent on USEA than lower educated participants, compared to participants in the control group.
In the focus groups with the interviewers, two other subgroups of participants came to the forefront. Firstly, people with a migration background were mentioned multiple times. For instance, it was mentioned that ‘… often, for people with a migration background, it was indeed due to this migration background’ [being unable to find a job] – ‘I had a respondent who had been in the Netherlands for 14 years, who spoke Dutch fluently… but still. And a girl with a headscarf. She spoke Dutch very well. And that was not the problem.’ ‘[interviewer] Did they experience discrimination when looking for a job?’ ‘Yes, yes.’.
Another problem mentioned in a focus group also concerned people with a migration background: foreign degrees are not always recognised, making it more difficult to obtain a job. If people with a migration background struggle more to find a job due to discrimination on the labour market and unrecognised certificates, it is reasonable to expect that they put effort into other socio-economic activities, such as volunteering or self-employment, or schooling to obtain a recognised degree. The open questions confirmed this line of thought, and provided examples of individuals with a migration background who were specifically interested these activities. We thus hypothesise:
H2c: When subjected to a less conditional welfare treatment, participants who were not born in the Netherlands will show stronger increases in time spent on USEA than those born in the Netherlands, compared to participants in the control group.
People with mental health issues were also addressed in a focus group. They got the opportunity to focus on other activities in the less conditional and restrictive regimes: ‘…I had a conversation with a woman who ended up on SA because she had mild burn-out symptoms. She experienced a lot of pressure, and was now part of the exempted treatment. She experienced much more calmness without all the obligations, started doing things she enjoyed, spent more time designing, and she noticed that due to this freedom she received orders with which she could make money…’.
Although the example above concerns mental health, a number of participants confirm that people with poor health in general (including physical health) spend less time looking for regular work (full-time or part-time). Sometimes they explicitly mention spending their time on other activities that are studied: ‘I have chronic pain and that impairs me enormously when looking for a job, and this led to social isolation. I am thinking about becoming self-employed.’ This leads to the following hypothesis:
H2d: When subjected to a less conditional welfare treatment, participants with poorer health will show stronger increases in time spent on USEA than those with better health, compared to participants in the control group.
Finally, the open questions revealed one other subgroup that was not mentioned in the interviews with professionals: parents. In both surveys, a number of people mentioned that taking care of their (young) child(ren) was a reason why they could not be (fully) employed. Several people mentioned that as the reason they had other activities, such as the ones studied in this paper: ‘I study and I take care of my children, so I have no time to apply for jobs’. This leads to the following hypothesis:
H2e: When subjected to a less conditional welfare treatment, participants with children will show stronger increases in time spent on USEA than those without children, compared to participants in the control group.
Quantitative design: measurements and methods
After supplementing our hypothesis on average treatment effects with hypotheses about which socio-demographic groups could benefit most from the experimental treatments, we proceed to the quantitative analysis. Data on the outcomes during the experiment were collected via a panel-design survey, with a baseline survey before the experiment started and one after approximately 2 years – a few months before the end of the experiment. The surveys were conducted as personal, computer-assisted interviews. The survey data were supplemented with data on personal characteristics, sourced from the municipal administration in Nijmegen. While the initial number of participants was higher, the analyses were run on 163 participants due to attrition (see Appendix 6). These samples are compared in Appendix 7 to assess the potential impact of attrition bias. As is elaborated upon in this appendix, some minor differences were found which point in different directions (i.e. those with better mental health were slightly more likely to drop out, while the opposite was true for self-rated health). Consequently, it seems highly unlikely that our results are completely driven by attrition. We will revisit this in the limitations section. We report significant (p < 0.05) and marginally significant (p < 0.10) results, which is an increasingly common practice (Pritschet et al., Reference Pritschet, Powell and Horne2016). Given the small sample size, an overly strict binary interpretation of p-values would increase the chance of false negatives considerably. However, we do exercise caution in the interpretation of marginally significant results.
The dependent variable in our models is the difference score in ‘hours per week spent on USEA’ between the baseline and the final survey, based on the hours per week participants spent on volunteering, informal care, schooling, or setting up a company (see Appendix 8 for the specific questions in the questionnaire). We summated the recorded hours for each activity into a single combined variable, based on our primary interest to investigate whether a less conditional approach stimulates people to spend more time on USEA in general, rather than on specific activities: a general increase would indicate a tendency towards activity that is considered valuable for the individual and/or for communities. This makes sense, since these activities are a zero sum game up to a certain point: one cannot (or is highly unlikely to) spend 25 hours per week on volunteering, while providing 20 hours of informal care and also spending 20 hours on schooling. Nevertheless, we also analysed hours per week spent on each separate activity as a robustness check. Where relevant, these analyses are provided as additional information about factors that drive the result of the combination variable.Footnote 4 We also analysed models where the dependent variable was transformed into five categories as a robustness check, which did not lead to substantially different conclusions (see Appendix 9).
We capped the number of hours spent at 40: higher numbers are both less realistic and would carry disproportional weight in the statistical analysis. Following common practice in lagged dependent variable panel models using difference scores as dependent variables (Kessler & Greenberg, Reference Kessler and Greenberg1981; Allison, Reference Allison and Clogg1990), we added the baseline level score to capture ceiling effects. After all, participants who already spend many hours per week on societal and economic activities are less likely to further increase this.
The core predictor variable is the group in which a participant was placed: ‘exempted’, ‘coached’ or ‘control’.
Following the qualitative outcomes, we tested for moderating effects of age, education, country of birth, health, and parenthood. Age is divided into two categories: younger than 50, and 50 and older (50+ being a common category used in studies on older age and work, e.g. CBS, 2006). Education is used as a continuous variable divided into four categories: basic, lower secondary, higher secondary, and tertiary, ranging from 0 to 3. Having a migration background is measured as ‘No: born in the Netherlands’ or ‘yes: not born in the Netherlands’. Parenthood is indicated with a binary response (yes/no). Health is measured in two complementing ways. Self-rated health, known to be a valid composite measure for both mental and physical health (Jylhä, Reference Jylhä2009), ranges from 0 ‘bad’ to 4 ‘excellent’. Mental health is measured using the Mental Health Inventory-5 measurement instrument, which assesses how the respondent felt over the last 4 weeks (Ware & Sherbourne, Reference Ware and Sherbourne1992): very nervous?; so down that nothing could cheer you up?; calm and relaxed?; sad and dejected?; happy?” Answers could vary from 1 ‘never’ to 5 ‘always’. In correspondence with the method that Statistics Netherlands uses to work with this indicator (Driessen, Reference Driessen2011), we transformed item scores into a 0–100 index, lower score meaning poorer mental health. The descriptive statistics can be found below in Table 1.
Table 1. Descriptive statistics
Source: Surveys experiment Participation Act Nijmegen (October/November 2017, February/March 2018, November/December 2018, September/August 2019); register data Nijmegen municipal administration (December 2017, April 2018).
N = 163.
For reasons of parsimony, which is obviously important in small N studies, we do not add additional control variables. Since the logic behind using control variables is to account for common causes (Spector & Brannick, Reference Spector and Brannick2011) of both the independent variable (treatment groups) and the outcome variable (changes in USEA), both randomisation and adding the baseline number of hours spent on USEA ensures to a large degree that the (moderated) impacts of the treatments are free of undetected spuriousness. Furthermore, the addition of control variables as a robustness check did not alter our conclusions.
Average and moderated treatment effects
The results of the average effects of the treatments on changes in USEA are presented in Table 2.
Table 2. Changes in unpaid socio-economic activities in the Nijmegen experiment, regressions with baseline score as control variable
N = 163; R = 0.47; No other variables are included in this model.
Effects in the expected directions were observed in both the coached and exempted treatment groups. However, these effects cannot be statistically distinguished from 0, refuting hypothesis H1. Given the modest sample size, we conclude that no large and highly consistent effect is present. Nevertheless, the positive effects may indicate that the amount of time recipients spent on USEA might have increased modestly, or only among specific groups, which has not been detected in the analyses presented in Table 2. In a robustness check, we examined the treatment effects for each of the four aspects of the dependent variable separately. We found that all aspects (volunteering, informal care, setting up as self-employed, schooling) contributed positively to the effects in both groups, except for volunteering in the exempted group: this effect was negative. None of the effects were statistically significant. In another robustness check, we investigated whether spending more time on USEA resulted in a reduced effort in job seeking or came at the cost of employment. This does not seem to be the case (see Appendix 10). We will revisit this in the limitations section.
When zooming in on the specific subgroups derived from the qualitative analysis, we find that the time spent on USEA increases for some, as shown in Table 3. In all cases where effects of (marginal) statistical significance were found, the experiment’s treatment effect is positive in terms of the development in hours spent on USEA over time.
Table 3. Unpaid socio-economic activities in the Nijmegen experiment (moderations)
N = 163; Values shown are the B-coefficient measured in hours, and the p-value is given in brackets. The moderator variables are only included in the model in which the specific differential in effect of the treatment by that moderator is tested, of which the direct effect is reported per model above. In other words, of all variables included in each model the coefficients are reported.
The first subgroup for which a positive development in time spent on USEA was found, is the subgroup of people aged 50 years and older in the coached group. There is a positive and significant effect of 7.2 hours (p = 0.03), meaning that SA recipients aged 50 and older in the coached group increase their time spent on USEA with 7.2 hours relative to the control group. The age interaction effect of −6.5 hours for the coached group is not significant itself, but does indicate that the relative increase of the coached treatment was only 0.7 (7.2–6.5) for the younger SA recipients in the coached group (additional analyses switching the reference category confirm this treatment effect is not statistically significant at all). This corresponds with our hypothesis 2a; however, we only partly accept this hypothesis, because only one treatment group is affected. The robustness check where each activity was studied separately, shows that the result is mainly driven by volunteering and informal care.
We also find positive treatment effects for participants with a migration background in both the exempted (8.1 hours) and coached (7.4 hours) treatment group, although the effect is only marginally significant in the exempted group (p = 0.09). These are the treatment effects for the reference group, which in our analyses are SA recipients with a migration background. The interaction terms of −7.8 (p = 0.07) for the exempted and −6.0 for the coached group (p = 0.17), indicate that people with a migration background account for almost the entire relative increase in time spent on USEA; the increases for people in the treatment groups with no migration background are respectively 0.3 (8.1–7.8 hours) and 1.4 (7.4–6.0 hours) and far from significant by themselves. These findings suggest that people with a migration background spend more time on USEA when exempted from reintegration obligations or when given supportive coaching instead. Consequently, we cautiously consider H2c to be supported.
We also found a positive treatment effect for people in the exempted group with relatively poor mental health. The main effect of the exempted treatment (13.5) shown in Table 3 is significant (p = 0.04), and the interaction term with mental health is marginally significant (p = 0.08). The interaction term of −0.2 means that the main effect decreases by 0.2 for every point of increase in mental health (on a scale from 0 to 100). People with relatively poor mental health spent extra time on USEA when exempted from reintegration obligations, and the strength of this positive effect diminishes when mental health increases. However, the main effect persists for people with a mental health score of 0, which is the conceptual minimum. The actual minimum encountered in our data is 15 (Table 1). An additional model showed that for people with a mental health score of 15, the exemption effect is 10.8 hours (p = 0.04) and the main effect remains statistically significant (p < 0.05) up to a mental health score of 50. For the 33% of our sample with the lowest scores on mental health, we thus find a positive impact of the exemption treatment. Therefore, we consider hypothesis 2d partly supported: a significant effect is only found upon testing for mental health and not for self-rated health, and only one of the two treatments yields this effect.
Furthermore, we found that educational level and parenthood do not seem to affect hours spent on USEA, refuting hypotheses H2b and H2e.
Conclusions
Core findings
Our analyses showed that the amount of time spent on USEA increased in both treatment groups, but that these average treatment effects were not statistically significant. Our qualitative analysis indicated that a less conditional and less burdensome SA regime might work better for participants who are older, higher educated, have a migration background, have relatively poor (mental) health, or have children. There were three subgroups for which the quantitative results indicated that the alternative treatments led to a stronger increase in hours spent on USEA: people aged 50 and older (coached treatment), people with a migration background (both treatments), and people with relatively poor mental health (exempted treatment).
In addition to the theoretical expectations and the simple fact that participants in the coached and exempted group no longer needed explicit permission for these activities, the qualitative analysis helped us gain further in-depth understanding of these findings. People aged 50 and older mention their inability to successfully find employment, and explore other opportunities for self-development. People with a migration background refer to being discriminated on the labour market (empirical research shows that discrimination indeed occurs, e.g. Van den Berg et al., Reference Van den Berg, Bijleveld, Blommaert and Ruiter2017), and the issue of their foreign certificates not being accepted. This pushes non-native people in the direction of other USEA instead of trying to find a job. In the default Dutch SA regime, recipients are discouraged to take up such activities, as this is believed to interfere with finding a job. Both alternative treatments, however, do allow and support this focus. Given their labour market hardship, participants not born in the Netherlands are more likely to spend more hours on USEA. This reasoning aligns with and expands on administrative burden theory, stating that such burdens affect those with little human capital most strongly (Moynihan et al., Reference Moynihan, Herd and Harvey2014), like people with a migration background. The qualitative data indicated that people with relatively poor physical and mental health can be stimulated to spend more time on USEA through an alternative, less conditional regime. For these people, a regular job is nearly unobtainable. However, other USEA, which were permitted in the alternative treatment, offer an escape from social isolation, whereas the regular regime only increases stress. The statistical analyses only showed this difference in effects for mental health, and only among those in the exempted group (versus the control group). The latter particularly aligns with the stress mechanism, as only the exempted group’s treatment is really unconditional regarding reintegration: all obligations are abolished. The coached treatment still has a conditional aspect, albeit with more autonomy. It seems that people with poorer mental health benefit most from a truly unconditional regime: even light obligations cause additional mental pressure and reduce their available mental bandwidth. This impairs them in becoming active, as argued by theories on mental bandwidth (Mullainathan & Shafir, Reference Mullainathan and Shafir2013). Furthermore, the finding that people with mental health impairments are disadvantaged in a conditional welfare regime aligns with the existing literature (Dwyer et al., Reference Dwyer, Scullion, Jones, McNeill and Stewart2020).
We found no evidence that education or parenthood influences the treatment effects. Given the indications from the qualitative study and our modest sample size, we do recommend further research on these aspects.
Limitations
The results of this study should be considered in its own context. As mentioned before, the most obvious limitation is the relatively small sample size of 163 participants, which reduces the chance that smaller effects reach statistical significance. Given the average treatment effects, we consider it plausible that a modest average treatment effect does exist. Moreover, the results strongly indicate that the alternative treatments are not harmful, compared to the control group.
More generally, selective participation can be an issue in social experiments, especially when participation is voluntary. This affects the generalisation towards the wider population (Greenberg & Schroder, Reference Greenberg and Shroder2004). Earlier analysis of the data indeed showed some differences between those who applied for the experiment and the general population receiving SA. Recipients in a more advantageous position, including those who already had a part-time job, who were higher educated or did not have a migration background, were more likely to participate. Meanwhile, qualitative evidence indicated that recipients who experienced considerable stress were less likely to participate. For instance while 45% of the general population receiving SA has a migration background, this is the case for only 27% of participants in the experiment (see Betkó et al., Reference Betkó, Spierings, Gesthuizen and Scheepers2019 for details). Although we cannot determine the exact impact hereof, this probably leads to an underestimation of the impact of the treatments on USEA. The treatments particularly benefit those who are underrepresented in the experiment compared to the general population.
Similarly, drop-out bias (Heckman et al., Reference Heckman, Hohmann, Smith and Khoo2000) might occur when a selective group withdraws from the experiment. However, our robustness checks (see Appendix 7) suggest that these effects are limited and unlikely to lead to false positives. There are some indications that those more oriented towards economic activities and success on the labour market, compared to social activities, are more likely to drop out. This might suggest a potential crowding out effect, leading to a (slight) overestimation of effects. However, as discussed, a more direct assessment did not reveal this effect, indicating a low risk of a strong overestimated effect due to crowding out. Regarding the impact of crowding out: reduced obligations might have crowded out the transition to work, by reducing the time spent on job seeking or being in employment. Given that there is no negative correlation, but an insignificant positive correlation between USEA and hours worked/employment (see the robustness check in Appendix 10), this does not seem to be the case at the treatment level. This is in line with an earlier study that showed increasing job search activities in the treatments (Betkó et al, Reference Betkó, Spierings, Gesthuizen and Scheepers2020). For some subgroups, the extra USEA might replace job-search activities: the qualitative data suggest that people with impaired mental health do become more active (engaging in extra activities without reducing time working or job searching), while older people and people with a migration background pursue these activities at the cost of job searching. Since the qualitative evidence suggests that these job searching efforts are mostly in vain, it is questionable whether their replacement with other USEA is problematic.
Altogether, the limitations of this study suggest that underestimation of effects is more likely than finding false positives.
Implications
In terms of future research and policy, we have shown that a less conditional welfare regime, based on trust and autonomy, could stimulate some groups of SA recipients to spend more time on volunteering, informal care, setting up their own business, and schooling/training. Accordingly, governments could consider giving SA recipients with mental health problems ‘a break’ (Scholten et al., Reference Scholten, Betkó, Gesthuizen, Fransen-Kuppens, de Vet and Wolf2023). Obligations and fines might hinder their mental health recovery, as other studies suggest (Williams, Reference Williams2021), and thus be counterproductive for social and economic reintegration. Additionally, governments could look into why the current SA regime does not appear to work as well for people with a migration background. Labour market discrimination drives people with a migration background to other activities than paid work, which was clearly found in our qualitative material. In the current SA regime, this group suffers from a double burden: discrimination complicates their job search, while maintaining eligibility for their allowance forces them to conform to the strict regime that prevents alternative activities.
Several of the activities we studied – particularly schooling and setting up a business, and arguably also volunteering (Paine et al., Reference Paine, McKay and Moro2013) – might function as steps towards work (thus reduce governments’ welfare costs in the long run) by creating social and human capital. All the activities we studied are seen as steps towards work in the ‘participation ladder’ framework used by Dutch municipalities (Terpstra, Reference Terpstra2011). Moreover, volunteering and informal care have value in themselves, for instance through increasing health (Piliavin & Siegl, Reference Piliavin and Siegl2007), social integration (Wilson, Reference Wilson2000), and by stimulating flourishing communities. It is therefore in the (financial) interest of governments to promote these activities, and more research should be directed to the impact of SA regimes thereon. In that light, the generalisability of the patterns found in the Dutch context remains to be established, as well as the level of context dependence of these patterns. This study could serve as a springboard for further research on this.
Supplementary material
For supplementary material accompanying this paper visit https://doi.org/10.1017/S0047279423000545
Acknowledgements
The experiment was partly funded by an ESF-SITS grant, project number 2017EUSF201587. With the exception of the SITS grant, the experiment was fully facilitated by the municipality of Nijmegen, which took the initiative to conduct the experiment, and was responsible for the practical execution. The Ministry of Social Affairs provided assistance and gave permission to deviate from applicable law (the Participation Act). We thank these parties for this unique opportunity to conduct a randomised social experiment. All analyses and results are produced without any interference from the municipal management and administration and are the responsibility of the researchers.
Competing interests
JB is employed by the municipality of Nijmegen, and conducts this study as part of an external PhD program. In no way did the municipality influence his research activities.
NS, MG and PS declare none competing interests.
Open access
