Introduction
Increasing employment rates is a central target of the European Union, as illustrated by the European Commission’s 2030 goal of 78 per cent employment in the 20–64 age group. This aim is related to welfare state challenges, such as labour shortages and budget deficits resulting from high healthcare and pension spending. However, employment targets often remain unmet, and employment rates are strongly differentiated by population subgroups (European Commission, 2021; OECD, 2021). Available literature discusses numerous potential supply-side barriers to employment among jobseekers, including motivational problems (e.g. Liu et al., Reference Liu, Huang and Wang2014), poor institutional knowledge and job search behaviour (e.g. Kanfer et al., Reference Kanfer, Wanberg and Kantrowitz2001; Moynihan et al., Reference Moynihan, Roehling and LePine2003), lacking experience and skills (e.g. Becker, Reference Becker1964), and limited social networks (Caliendo et al., Reference Caliendo, Schmidl and Uhlendorff2011; Franzen & Hangartner, Reference Franzen and Hangartner2006; Granovetter, Reference Granovetter1995). Demand-side barriers include hiring costs, limited information on jobseekers’ workplace performance, and discriminatory practices (Baert, Reference Baert and Gaddis2018; Larsen & Vesan, Reference Larsen and Vesan2012; Marx, Reference Marx2001; Van Belle et al., Reference Van Belle, Di Stasio and Caers2018; Van Borm et al., Reference Van Borm, Burn and Baert2021).
Consequently, many welfare states have introduced active labour market policies (ALMP), including a wide range of training programmes. These programmes by design typically target specific subgroups of unemployed jobseekers experiencing different supply- and/or demand-side barriers to employment. Given differences between programmes, available literature remains fragmented as most contributions evaluate a specific training programme in a given country or region (Card et al., Reference Card, Kluve and Weber2010; Fitzenberger et al., Reference Fitzenberger, Osikominu and Völter2008; Greenberg et al., Reference Greenberg, Michalopoulos and Robins2003; Hujer et al., Reference Hujer, Thomsen and Zeiss2006; Kluve, Reference Kluve2010; Lalive et al., Reference Lalive, Van Ours and Zweimüller2008; Richardson & Van Den Berg, Reference Richardson and Van Den Berg2001; van Den Berg & Vikström, Reference van Den Berg and Vikström2014). In contrast, the amount of studies evaluating two (Biewen et al., Reference Biewen, Fitzenberger and Osikominu2014; Dengler, Reference Dengler2019; Hotz et al., Reference Hotz, Imbens Guido and Klerman Jacob2006; Larsson, Reference Larsson2003; Strandh and Norlund, Reference Strandh and Nordlund2008) or more (Gerfin & Lechner, Reference Gerfin and Lechner2002; Kasrin & Tübbicke, Reference Kasrin and Tübbicke2022; Lechner et al., Reference Lechner, Miquel and Wunsch2011) subtypes of training programmes in the same context remains limited.
The large body of training programme evaluations contrasts with limited understanding of why particular programmes seem (in)effective (Card et al., Reference Card, Kluve and Weber2017; Filges et al. Reference Filges, Smedslund and Jørgensen2018). Consequently, our first contribution is that we take innovative steps to explain why ALMP training programmes are (in)effective. We exploit variation in ALMP training design features to (I) unpack Belgian (Flemish) ALMP training programmes for unemployed jobseekers by different types of these programmes, and (II) interpret (in)effectiveness connected to variation in the specific theoretical barriers to employment which are targeted by particular design components and witnessed by particular subgroups of unemployed jobseekers. Hence, in contrast to literature reviews and meta-analyses that tend to generalise across countries, this study evaluates the effectiveness of different types and subtypes of ALMP training in Belgium. Effectiveness implies that a training programme succeeds at stimulating employment entry for its participants.Footnote 1 Based on general ALMP classifications (Bonoli, Reference Bonoli2013; Card et al., Reference Card, Kluve and Weber2010, Reference Card, Kluve and Weber2017; Dinan, Reference Dinan2019), and adult education programmes (Boeren & Whittaker, Reference Boeren and Whittaker2018), we adopt a three-level hierarchical typology of training (see ‘Flemish labour market and typology of ALMPs’). Subsequently, as each of these training programme components connect to underlying theoretical mechanisms that are assumed to stimulate employment entry, we relate our estimates of (in)effectiveness to the theoretical barriers to employment targeted by the training programme. Although direct empirical tests for such underlying explanations lie beyond the scope of this research, exploiting variation in programme components to link available theories of (un)employment to our evaluation results constitutes a meaningful step towards explaining the (in)effectiveness of specific programmes.
Our second contribution adds to literature assessing employment stability (Bergemann et al., Reference Bergemann, Fitzenberger and Speckesser2009; Crépon et al., Reference Crépon, Ferracci and Fougère2012; Kopf, Reference Kopf2013) by studying whether training programmes stimulate employment entry, but also whether participants experience employment spells lasting at least one year.Footnote 2 Whereas the assessment of employment stability is often hampered by the inability to follow up unemployed jobseekers once they have entered employment, this study benefits from a data linkage between the employment office and social security data on labour market positions. Understanding which training programmes effectively assist unemployed jobseekers to gain a stable foothold in the labour force is essential in light of individual- and household-level financial security, well-being, and employment-related eligibility criteria in social policy (e.g. parental leave) (Kil et al., Reference Kil, Wood and Neels2018b; Marynissen et al., Reference Marynissen, Wood and Neels2021), and aggregate-level costs of unemployment benefits and activation in case of recurrent unemployment spells.
Flemish labour market and typology of ALMPs
The Flemish labour market is, similarly to neighbouring countries the Netherlands, Germany, and France, characterised by high employment protection, minimum wages, and unemployment benefit generosity (Andersen, Reference Andersen2012; Eurostat, 2020). As a result, unlike countries with more flexible labour markets (e.g. Denmark) or countries with lower minimum wages (e.g. USA or UK), labour market segmentation in Flanders implies relatively high levels of (long-term) unemployment amongst outsiders such as low-skilled subpopulations or groups with a non-European migration background (Kil et al., Reference Kil, Neels, Wood and de Valk2018a; Noppe et al., Reference Noppe, Vanweddingen and Doyen2018; OECD, 2016).
Spending on ALMPs as a percentage of GDP is relatively high in Belgium (0.92 as a GDP percentage in 2020) (Andersen, Reference Andersen2012; OECD, 2024), particularly in Flanders (Federaal Planbureau, 2020). The employment office relies on job search assistance and training, open to all jobseekers,Footnote 3 to facilitate a swift (re-)entry into (stable) regular employment. Job search assistance is provided by caseworkers. The type of job search assistance depends on jobseeker-caseworker interactions and can consist of the following components. Vacancy notifications imply that an algorithm matches vacancies to jobseekers’ profiles automatically, or that caseworkers manually notify jobseekers. Mandatory interviews oblige unemployed jobseekers to participate in interviews selected by caseworkers, followed by evaluation. In case a jobseeker is unresponsive, file transmission means that the unemployment benefits agency considers a withdrawal or limitation of unemployment benefits.
Training programmes are categorised in a three-level hierarchical typology. First, at the highest level, all training programmes are considered together. Second, programmes are subdivided into two main types: classroom and workplace training. Third, each type can be further subdivided into two subtypes depending on the employment barriers targeted. Classroom training facilitates knowledge and skill acquisition in a classroom and can be further subdivided into general classroom training which targets unemployed jobseekers’ motivation and job search skills,Footnote 4 and occupation-specific classroom training geared towards knowledge and skills for specific occupations. Workplace training combines knowledge and skill acquisition with workplace-specific social capital accumulation, and allows employers to assess unemployed jobseekers’ workplace performance. Unlike non-contractual workplace training,Footnote 5 contractual workplace training additionally provides wage subsidies during temporary employment contracts (1–6 months) after the training programme,Footnote 6 to compensate employers for low productivity.
Information on ALMP assignment is essential in the context of effect evaluations as (non-)participation in a given programme is not random. Research for Flanders (Elloukmani & Raeymaeckers, Reference Elloukmani, Raeymaeckers, Wood and Neels2020) shows that, whereas jobseekers are mostly unaware of different training options, caseworkers are trained to identify barriers to employment, assess whether training is necessary, and select the training type geared towards those barriers. Objective criteria such as age, experience, level and field of education, language proficiency, and job search behaviour are combined with jobseeker-caseworker conversations on preferences (e.g. preferred working hours or employment sector), and other conditions for employment (e.g. work-family balance). Enrolment in workplace training is also affected by employers. Whereas the employment office and employers install non-contractual workplace training in collaboration, employers directly recruit jobseekers for open vacancies under contractual workplace training. These selection mechanisms imply that, regardless of the training considered, the comparison of participants’ and non-participants’ employment outcomes requires controls for (self-)selection bias.
Theory
Available literature on unemployment indicates various determinants, depending on the population subgroup considered. Supply-side barriers to employment include motivational problems (e.g. Liu et al., Reference Liu, Huang and Wang2014), little institutional knowledge and ineffective job search behaviour (e.g. Kanfer et al., Reference Kanfer, Wanberg and Kantrowitz2001; Moynihan et al., Reference Moynihan, Roehling and LePine2003), limited experience or human capital (e.g. Becker, Reference Becker1964), and social networks that entail few job opportunities (e.g. Caliendo et al., Reference Caliendo, Schmidl and Uhlendorff2011; Franzen & Hangartner, Reference Franzen and Hangartner2006; Granovetter, Reference Granovetter1995). In addition, commonly suggested demand-side barriers are hiring costs, limited information on jobseekers’ productivity, and discrimination (e.g. Baert, Reference Baert and Gaddis2018; Larsen & Vesan, Reference Larsen and Vesan2012; Marx, Reference Marx2001; Van Belle et al., Reference Van Belle, Di Stasio and Caers2018, Reference Van Borm, Burn and Baert2021). Different training programmes target different (sets of) barriers to employment entry for a subset of unemployed jobseekers. Consequently, effects of programme participation are also likely to vary depending on the training type considered. An overview of ALMP training programmes considered and targeted barriers to the entry in (stable) regular employment amongst unemployed jobseekers is provided in Table 1.
Table 1. Overview of ALMP training programmes considered and underlying mechanisms potentially affecting entry in (stable) regular employment amongst unemployed jobseekers
Source: Authors’ synthesis based on Employment Office documentation.
General classroom training
Following self-regulation framework asserting that successful task completion is dependent on motivation and ability (Liu et al., Reference Liu, Huang and Wang2014), general classroom training - to enhance unemployed jobseekers' motivation and job search skills - potentially stimulates employment entry through motivational pathways and skill accumulation. Regarding the former, the theory of planned behaviour (e.g. Liu et al., Reference Liu, Huang and Wang2014) states that the intention to perform an action is influenced by individual attitudes, prescribed social norms, and perceived personal control. General classroom training might persuade unemployed jobseekers with initially negative attitudes towards job search to apply for a job, by focusing on the development of realistic job goals. With respect to social norms, literature suggests that involving jobseekers’ family and friends positively affects job search behaviour (Liu et al., Reference Liu, Huang and Wang2014; Song et al., Reference Song, Wanberg and Niu2006), yet Flemish general classroom training includes no such features. Regarding personal control, self-efficacy is enhanced through interaction, role playing, feedback sessions, and ‘verbal (self-)persuasion’ through positive framing (e.g. ‘experienced’ versus ‘too old’) (Liu et al., Reference Liu, Huang and Wang2014). Available literature indicates that jobseekers’ job search attitudes and perceived self-efficacy positively affect job search behaviour and outcomes (Kanfer et al., Reference Kanfer, Wanberg and Kantrowitz2001; Moynihan et al., Reference Moynihan, Roehling and LePine2003). With respect to skill accumulation, general classroom training develops four types of job search skills. First, modules on resumé-writing provide employers the information necessary to make hiring decisions (Liu et al., Reference Liu, Huang and Wang2014). Second, application letter-writing skills presumably stimulate job interview invites. Third, since employers rely heavily on job interviews, courses on interview performance are organised (Hall et al., Reference Hall, Jackson Gradt and Goetz2011; Liu et al., Reference Liu, Huang and Wang2014). Fourth, general classroom training programmes also include language training, as language proficiency is required to develop the aforementioned competences and needed in many occupations (Liu et al., Reference Liu, Huang and Wang2014).
Available literature also provides potential reasons for non-effects or even unintended negative effects of participation in general classroom training. For example, Card et al. (2017) show that general classroom training effects are more limited than those of occupation-specific training or internships, which is often related to the limited accumulation of human capital that is directly relevant to perform in a job and the absence of contact with potential employers in general classroom training programmes . Finally, despite potential positive signalling effects (particularly for candidates with a larger distance to the labour market (Liechti et al., Reference Liechti, Fossati and Bonoli2017), negative signalling towards employers and stigmatisation of jobseekers might also occur, particularly if employers estimate individual agency in programme participation to be limited (e.g. in case a caseworker decides which programme is most relevant) (Fossati et al., Reference Fossati, Liechti and Wilson2020).
Occupation-specific classroom training
Closely related to human capital theory’s expectation that employers select workers with high human capital, occupation-specific classroom training focusses on skills and knowledge to perform specific occupations. We distinguish two pathways through which occupation-specific classroom training can positively affect employment entry. First, occupation-specific classroom training increases the marginal productivity of participants’ labour (Becker, Reference Becker1964) which is expected to stimulate employment entry. Second, occupation-specific training can entail positive signalling regarding relevant competences as it is organised in collaboration with employers and mostly geared towards bottleneck occupations (i.e. occupations exhibiting labour shortages), or signalling of other individual traits (e.g. motivation).
However, neutral or negative effects of participation in occupation-specific classroom training are also theoretically possible. Based on available literature on social networks and employment (Caliendo et al., Reference Caliendo, Schmidl and Uhlendorff2011; Franzen & Hangartner, Reference Franzen and Hangartner2006), we could also expect non-effects of occupation-specific classroom training due to limited opportunities to meet and convince potential employers. Furthermore, negative signalling is also plausible, depending on employers’ perceptions regarding whether jobseekers themselves chose to enrol in the programme, and the relevance of the programme (Fossati et al., Reference Fossati, Liechti and Wilson2020; Liechti et al., Reference Liechti, Fossati and Bonoli2017). Finally, programme participation might also raise unemployed jobseekers’ reservation wages as a side-effect, which might prolong the unemployment spell (Thomsen et al., Reference Thomsen, Walter and Aldashev2013).
Non-contractual workplace training
We distinguish two pathways through which non-contractual workplace training potentially affects employment entry. First, similar to occupation-specific classroom training, non-contractual workplace training allows knowledge and skill accumulation, and positive signalling of ability and motivation (Liechti et al., Reference Liechti, Fossati and Bonoli2017). In contrast to classroom training, skills and knowledge are geared towards a specific workplace setting, which might be more relevant for employers. Second, work experience in non-contractual workplace training allows unemployed jobseekers to acquire sector-specific social capital, potentially benefitting job search. Available research indicates that the characteristics of individuals in one’s network determine opportunities in particular (Caliendo et al., Reference Caliendo, Schmidl and Uhlendorff2011). Amongst employers, limited information on job candidates’ performance is likely to entail hesitation to hire applicants or even discriminate the job seeker (Baert, Reference Baert and Gaddis2018; Larsen & Vesan, Reference Larsen and Vesan2012; Van Belle et al., Reference Van Belle, Di Stasio and Caers2018, Reference Van Borm, Burn and Baert2021). In this context, non-contractual workplace training participants are given the opportunity to showcase their abilities.
However, negative signalling is also possible (e.g. participation signalling ALMP dependence) (Ingold & Stuart, Reference Ingold and Stuart2015; Liechti et al., Reference Liechti, Fossati and Bonoli2017), and increased reservation wages might also entail a negative effect on employment entry. Furthermore, in contrast to occupation-specific classroom training, the transferability of skills acquired in a given workplace to other workplaces might be limited.
Contractual workplace training
Participation in contractual workplace training is likely to affect employment in ways similar to non-contractual workplace training. However, contractual workplace training interacts differently with the demand side of the labour market. First, to overcome the barrier of low marginal productivity relative to hiring costs, contractual workplace training relies not only on human capital accumulation to increase productivity, but also wage subsidies to lower hiring costs (Bell et al., Reference Bell, Blundell and Van Reenen1999). Second, the fact that contractual workplace training also includes a temporary employment contract might grant access to other types of human capital (e.g. more advanced knowledge and skills) and social capital (e.g. social relations beyond the internship coach), might facilitate employment entry.
However, in addition to potential negative signalling of wage subsidies and ALMP-dependence, increased reservation wages, and transferability similar to non-contractual workplace training, available literature provides several additional concerns. Regarding the wage subsidy, previous research highlights the possibility that employers initiate programmes including wage subsidies as a result rather than a cause of subsequent hiring (Bell et al., Reference Bell, Blundell and Van Reenen1999; Marx, Reference Marx2001). A potential underlying reason is that efficient recruitment strategies select the most productive unemployed jobseekers, whilst using wage subsidies whenever possible. With respect to the potential stepping-stone effect of a temporary employment contract, available literature might again fuel doubt (Korpi & Levin, Reference Korpi and Levin2001), as temporary workers typically experience less opportunities for upskilling to establish a stable position (Forrier & Sels, Reference Forrier and Sels2003).
Data and method
Data and sample selection
We use the 2005–2016 ‘Migration, Integration and Activation’ (MIA) panel (Wood and Neels, Reference Wood and Neels2020) on the working age population aged 18–65 in Flanders, constructed from two sources: (I) monthly employment office monitoring of job search and ALMP participation amongst unemployed jobseekers, and (II) quarterly data on labour market positions from the Crossroads Bank for Social Security. We select all unemployment spells that start within the observation window and are registered at the employment office.Footnote 7 This selection entails a sample of 43,369 unemployment spells (425,553 person-quarters) experienced by 17,281 randomly sampled individuals.
Evaluation question
We focus on the first training programme that an individual participates in within an unemployment spell, assuming that later (non-)participation in training programmes, as well as enrolment duration and successful completion is endogenous (e.g. a participant may quit a programme because of a job offer). Hence, treatment is defined as starting training at a given duration of unemployment, and the group who starts is compared to a group of jobseekers that did not start (yet). The outcome of interest is the hazard of entering (stable: i.e. lasting at least one year) regular employment at any time since entry into a training programme. We start measuring the impact from the start of participation, considering lock-in effects as meaningful parts of the effect (Sianesi, Reference Sianesi2004, Reference Sianesi2008).
We perform one-by-one programme-specific estimations of the average treatment effect on the treatedFootnote 8 by contrasting observed hazards of entering (stable) regular employment amongst the group who started a training programme, to the unobserved counterfactual situation in which the group of participants would not have started participating. As the latter cannot be observed directly, a propensity score matching approach is used to match participants when joining a program with non-participants at given times since the start of the unemployment spell, conditional on a set of observed matching variables.
Dynamic propensity score matching
As treatment status varies with duration of unemployment, we follow Sianesi (Reference Sianesi2004) and apply dynamic propensity score matching which converts the dynamic problem of ALMP participation into a static one by conditioning on duration of unemployment in discrete-time quarters. For every training programme, we compare the group who started in quarter 1, 2, 3 or 4 with a matched control group who did not (yet) start the programme at that specific duration of unemployment. Low cell frequencies prevent the evaluation of programme participation starting after the first year of unemployment, but we cover 86.1 per cent of all programme startsFootnote 9 in our data. To identify the unobserved counterfactual, the dynamic propensity score matching model estimates the probability of starting programme participation as a function of a vector of observed characteristics using a probit equation (Rosenbaum & Rubin, Reference Rosenbaum and Rubin1983). Based on the resulting propensity scores, every treated individual joining a training programme at a particular duration of unemployment is matched to a corresponding ‘statistical twin’ who did not start participation (yet), using the nearest-neighbour algorithm with replacement.Footnote 10 All treated individuals are on common support, resulting in a match for all treated jobseekers. A first major advantage of dynamic propensity score matching is that it mimics a true experimental context to approximate causal effects of training programme participation using observational data. A second advantage is that the dynamic stratification of the matching procedure allows to cope with dynamic evaluation contexts in which treatment can be started at any point in time (i.e. unlike many other experiments in which (non-)treatment is administered at a fixed time point).
However, the degree to which dynamic propensity score matching – like all selection-on-observables approaches – can produce causal estimates of ALMP participation, depends on whether the conditional independence assumption holds. This assumption implies that, conditional on unemployment duration and observed characteristics used to match participants to statistical twins, the fact that the one unemployed jobseeker starts programme participation while the other does not, is not associated with the employment outcomes for the participating jobseeker regardless of participation. Whether this assumption holds, depends on processes of (self-)selection into training programmes and the richness of the data to control for selection. In case groups starting ALMP participation and a corresponding matched groups differ in a way that also affects their subsequent employment entry, the estimates might be biased (i.e. selection bias).
This study benefits from the identification of (self-)selection determinants using qualitative research (Elloukmani & Raeymaeckers, Reference Elloukmani, Raeymaeckers, Wood and Neels2020). Non-participant observations of jobseeker-caseworker interactions and in-depth interviews highlighted five domains which determine whether an unemployed jobseeker enrols into a training programme: skills and credentials, previous labour force participation, employment preferences, past ALMP participation, demographic and family contexts. All identified dimensions (see Appendix A) could be operationalised or proxied using the register data at hand and the propensity score matching results (see Appendix B) indicate that all significant distributional differences between the group of participants and non-participants disappear after matching. This suggests that – due to the usage of preparatory qualitative analyses and the richness of the quantitative data – the condition of independence assumption holds. However, although quasi-experimental identification strategies generally yield similar results to true experimental approaches (Card et al., Reference Card, Kluve and Weber2017), we cannot rule out the possibility that our estimates are biased due to unobservable variation, which is not accounted for.
Follow-up: discrete-time hazard models
The treated group is followed until a transition is made into regular employment or censoring occurs due to inactivity, death, emigration, reaching age 65 or the end of the observation window on December 31, 2016. Individuals in the matched control group are also observed until a transition to regular employment or censoring for the same reasons as the treated group, as long as they have not started the training programme considered. Individuals in the matched control group are not censored in case of enrolment in other ALMP programmes. A discrete-time hazard model (see Equation [1]) estimates the quarterly hazard of entering regular employment
$H{\left( t \right)_i}$
as a function of the time since the start of programme participation or selection into the matched control group
$( T)$
(baseline, cubic effect), participation in the training program
$\left( {TP} \right)$
, and interactions between program participation and the baseline hazard function. In this equation,
$q\left( t \right)$
is the conditional probability of entering regular employment for individual
$i$
in quarter
$t$
since entry into the programme (or matching) among individuals who had not entered regular employment prior to quarter
$t$
. Due to the complementary log-log link function, exponentiated parameter estimates represent hazard ratios (Allison, Reference Allison and Hardy2004). As the sampling design of the MIA Panel is disproportionately stratified by age and migration background, inverse probability weights are used.
$$H{\left( t \right)_i} = - ln\left[ {1 - q{{\left( t \right)}_i}} \right] = \matrix{ {{e^{\widehat \alpha }}.{e^{\widehat \beta {T_i}}}.{e^{\widehat \beta T_i^2}}.{e^{\widehat \beta T_i^3}}.{e^{\widehat \beta T{P_i}}}.{e^{\widehat \beta T{P_i}{T_i}}}.{e^{\widehat \beta T{P_i}T_i^2}}.{e^{\widehat \beta T{P_i}T_i^3}}} \cr } $$
In addition, we also estimate hazards of entering a stable regular employment spell, which lasts for at least one year. Although using a different outcome variable, all other model specifications are identical.
Cumulative incidence and cumulative return on training participation
As the quarterly hazard of entering regular employment reflects the hazard of entering regular (stable) employment that is accumulated in a given quarter, we use life table functions to provide an indication of the (differential) cumulative incidence of entry into regular (stable) employment since entry into the program (or selection into the matched control group). This allows us to conceptualise the average treatment effect on the treated as the difference in the cumulative proportion of ever entering regular (stable) employment between participants and matched non-participants, which is labelled as ‘cumulative return’ over time (see Maes et al. (Reference Maes, Wood and Neels2019) for a detailed discussion).
Empirical results
The effect of training participation on regular employment entry
Figure 1 presents the effects of participation in training, considered as a homogenous treatment, on the hazard of entering a regular employment spell. Training initially locks jobseekers in (Fig. 1.1) as participation decreases the hazard of employment entry significantly by 5.22 percentage points compared to those who did not (yet) participate. This finding for instance suggests that unemployed jobseekers enrolled in ALMP training programmes temporarily spend less time on job search. Subsequently, between five and seven quarters since the programme start, participants exhibit significantly higher hazards of employment entry. In the longer run (i.e. after 2 years), treatment effects weaken.
Figure 1. The effect of participation in training programmes on transition to employment amongst unemployed jobseekers, Flanders 2005–2016.
Notes: *Matched results using nearest-neighbour dynamic propensity score matching with replacement ( Reference SianesiSianesi, 2004)
Source: MIA panel dataset 2005–2016 (weighted), calculations by authors.
Figure 1.2 indicates that the cumulative return on training turns positive from the fifth quarter onwards. In addition, we find that when estimated without the use of dynamic propensity score models, the initial lock-in effect and positive effects of training programme participation afterwards are severely overestimated. The latter implies that ALMP training programme participants are typically positively selected (by caseworkers or themselves), for instance in terms of higher level of education (see Table 3 in Supplementary material). Consequently, the return on programme participation would have been clearly overestimated in the absence of the applied propensity score matching model.
Figure 2. The effect of participation in classroom training programmes on transition to employment amongst unemployed jobseekers, Flanders 2005–2016.
Source: MIA panel dataset 2005–2016 (weighted), calculations by authors.
Figure 2 illustrates participation effects of classroom training, general classroom training, and occupation-specific classroom training, and exhibits strong variation depending on the training programme studied. Similarly to the previous results for all programmes combined, classroom training (Fig. 2.1) initially exhibits a significant negative lock-in effect. Significant positive effects occur from the fourth to the seventh quarter since the start of participation, followed by insignificant effects. Cumulative return turns positive from the fourth quarter onwards and stabilises thereafter, and comparing participants to non-participants without controlling for selective participation entails an overestimation of the positive cumulative return on training in the longer run.
However, contrasting findings between general classroom training and occupation-specific classroom training clearly illustrates how lumping together all classroom training masks notable variation in effectiveness. Results for general classroom training deviate considerably from the general pattern for classroom training, with mostly insignificant treatment effects (Fig. 2.2), and up to 5.94 percentage point lower cumulative incidence of employment entry among participant jobseekers, compared to matched non-participants. This pattern of non-effects or even negative effects of participation in general classroom training contrasts strongly with the results for occupation-specific classroom training. The latter (Fig. 2.3) displays a significant and strong negative lock-in effect in the first quarter since programme start, which might be related to the fact that this training is geared towards a new occupation, which presumably puts job search on hold. However, this initial negative effect is more than compensated for later on, with positive effects of participation, entailing a positive cumulative return of 9 percentage points in cumulative incidence of employment entry among participants.
Figure 3. The effect of participation in workplace training programmes on transition to employment amongst unemployed jobseekers, Flanders 2005–2016.
Source: MIA panel dataset 2005–2016 (weighted), calculations by authors.
Figure 3 shows participation effects of workplace training programmes combined, non-contractual workplace and contractual workplace training programmes. These findings further highlight the need to unpack ALMP training programmes. The treatment effects of workplace training programmes combined (Fig. 3.1) exhibit an initial significant negative lock-in effect. As a result of significant positive effects in the medium run, the negative initial return on training turns positive from the third quarter onwards. It reaches a maximum return of 15.06 percentage points higher cumulative incidence of entry into employment among participants, and cumulative return values of around 10 percentage points in the longer run. This high cumulative return indicates that workplace training is more effective in stimulating the entry into regular employment amongst its participants, in comparison to classroom training performance amongst its respective participants.
Furthermore, when distinguishing subtypes of workplace training programmes, further differentiation in the effect patterns emerges. Non-contractual workplace treatment effects (Fig. 3.2) exhibit no short-term negative effects, and significant positive effects from the second to the fifth quarter since the programme start, increasing the hazard of employment entry by 12.32 to 19.04 percentage points for participant jobseekers. Consequently, cumulative return is positive throughout the observation window, with up to 30.79 percentage points higher cumulative incidence for non-contractual workplace training participants compared to matched non-participants in the sixth quarter since programme start, and a cumulative return approximating 20 percentage points in the longer run. In contrast to all other programmes, when comparing participants to non-participants without controlling for selective participation, cumulative return is consistently downward biased, indicating that participants are negatively selected regarding observed characteristics associated with employment entry. Our matching results (available upon request) indicate that unemployed jobseekers with little employment experience are disproportionately more likely to participate in non-contractual workplace training. This difference in evaluation results depending on whether controls for selectivity are used, highlights again the added value of applying dynamic propensity score matching models. The effects of participation in contractual workplace training programmes differs strongly from non-contractual workplace training programmes (Fig. 3.3), with a sharp negative initial lock-in effect, as well as the relatively short-lived nature of positive effects and significant negative differential hazards for participants thereafter. Initially lower cumulative incidence amongst participants turns positive by the third quarter, yet returns to negative values from the fifth quarter onwards.
Stable employment entry
Question remains whether training programme participation also assists unemployed jobseekers to gain a stable foothold in the labour force. Figure 4 illustrates cumulative return on training when estimating the hazard of entering an employment spell which lasts for at least one year as a measure for stable employment. Findings indicate that cumulative return on training programmes most strongly stimulating employment entry (occupation-specific and non-contractual workplace training) increases further, indicating that these training programmes are particularly effective in assisting unemployed jobseekers to achieve stable regular employment. The maximum cumulative return on training in terms of entry into regular employment increases from 9.39 to 16.26 percentage points for occupation-specific classroom training participants, which implies that a lack of occupation-specific skills might yield higher employment instability for this subgroup. The maximum cumulative return on non-contractual workplace training programmes increases more moderately from 30.79 to 36.55 percentage points. In contrast, when considering the transition to stable regular employment, cumulative return on general classroom training participation decreases further, and cumulative return on contractual workplace training remains limited to the short term.
Figure 4. Cumulative return on training with respect to transition into employment versus stable employment (min. duration 1 year) amongst unemployed jobseekers, Flanders 2005–2016.
Notes: ATT: average treatment effect on the treated Source: MIA panel dataset 2005–2016 (weighted), calculations by authors.
Robustness checks
Finally, two robustness checks were performed. First, with respect to the different algorithms that can be used in propensity score matching, as the usage of Caliper or kernel matching entails similar substantive findings,Footnote 11 we chose to present the simpler nearest-neighbour approach. Second, in the main analyses matched control group units are censored whenever they start participating in the training programme of interest (e.g. general classroom training) in order to be able to interpret this group as ‘non-participants’. However, as later enrolment into the training programme amongst matched control units who did not start participation at the unemployment duration of matching can also be considered a meaningful effect of initial non-participation, we also ran all models in which control units are not censored in case they enrol into the training programme of interest. These models typically yield only marginally weaker treatment effects, due to the low share of programme starts occurring at longer unemployment durations (see section on dynamic propensity score matching).
Discussion and conclusion
Since the available literature on ALMP training for unemployed jobseekers remains fragmented, and provides little insight in the underlying reasons for (in)effectiveness (Card et al., Reference Card, Kluve and Weber2017; Filges et al. Reference Filges, Smedslund and Jørgensen2018), this study (I) unpacks training into different subtypes in Flanders, and (II) interprets evaluation results in terms of programme components and the employment barriers they address. Hence, we estimate the effects of different ALMP programmes, but also provide potential underlying explanations for (in)effectiveness of particular training programmes. As such, we take two important steps toward understanding whether and how ALMP training works. We evaluate four subtypes of ALMP training. Due to relatively high spending on ALMP training programmes in Flanders, our results are of interest to scholars and policymakers in other high-income countries increasing ALMP spending.
Regarding classroom training, our results for general classroom training programmes indicate that, aligning with previous findings (Gerfin & Lechner, Reference Gerfin and Lechner2002; Greenberg et al., Reference Greenberg, Michalopoulos and Robins2003), participation in such a programme alone does not seem to stimulate regular employment entry. Although such non-effects could signal that general classroom training fails to address the targeted barriers in terms of job search skills and motivation, the programmes considered include most, if not all, components put forward in previous literature as stimulating employment (Liu et al., Reference Liu, Huang and Wang2014). Consequently, these non-effects suggest that general classroom training participation, though essential for unemployed jobseekers with poor motivation and/or job search skills, does not suffice as other barriers to employment remain unaddressed. Two important unaddressed theoretical barriers to employment are low human capital and productivity (requiring more advance training), and limited opportunities to convince potential employers (requiring workplace training). This conclusion aligns with contemporary policy evolutions in Flanders to consider additional components to be included in general classroom training or combine such training with follow-up programmes (Horemans & Ghysels, Reference Horemans, Ghysels, Wood and Neels2020).
Participation in occupation-specific classroom training, providing occupation-specific knowledge and skill set, is found to facilitate the transition to regular employment, and particularly stable regular employment. This finding suggests that, amongst unemployed jobseekers targeted for this type of programme, participation in occupation-specific classroom training increases human capital, which in turn increases marginal productivity and the employability of programme participants, as perceived by employers (Becker, Reference Becker1964; Liechti et al., Reference Liechti, Fossati and Bonoli2017). It also indicates that the absence of direct contact with employers in such programmes does not prevent effectiveness.
This study also evaluates training programmes at workplaces and our findings contribute to social policy discussions regarding the role of employers in demand-side-oriented ALMP training programmes. Participation in non-contractual workplace training is found to strongly stimulate the transition to (stable) employment amongst its participants. Given the finding that participants in non-contractual workplace training are also negatively selected (e.g. lower educational attainment), the identified positive effect of participation suggests that the opportunity to accumulate human capital whilst also meeting employers through work experience, stimulates their employment entry (Caliendo et al., Reference Caliendo, Schmidl and Uhlendorff2011; Franzen & Hangartner, Reference Franzen and Hangartner2006). With respect to the demand-side barriers of imperfect information on skills and performance and potential discrimination, participation in non-contractual workplace training is likely to generate positive signals of workplace experience, and partly resolve employers’ information problem (Baert, Reference Baert and Gaddis2018; Larsen & Vesan, Reference Larsen and Vesan2012; Van Belle et al., Reference Van Belle, Di Stasio and Caers2018; Van Borm et al., Reference Van Borm, Burn and Baert2021). These identified positive treatment effects of training programmes focussing on human capital accumulation, particularly in combination with workplace experience, align with the widely accepted conclusion that ALMP focussing on human capital and workplace experience are most effective (Card et al., Reference Card, Kluve and Weber2017; Gerfin & Lechner, Reference Gerfin and Lechner2002; Gerfin et al., Reference Gerfin, Lechner and Steiger2005; Greenberg et al., Reference Greenberg, Michalopoulos and Robins2003; Kluve, Reference Kluve2010; Nekby, Reference Nekby2008; Sianesi, Reference Sianesi2008; Vikström, Reference Vikström2017). Furthermore, our findings suggesting that contact with employers increases effectiveness aligns with recent studies on employer engagement, often highlighting employers’ doubt about the use of ALMP when they have not directly participated (Ingold, Reference Ingold2018; Orton et al., Reference Orton, Green and Atfield2019).
Finally, our findings suggest that the fact that contractual workplace training addresses the widest range of barriers to unemployed jobseekers’ regular employment entry, is trumped by the way in which this training type interacts with the demand side of the labour market. In line with previous findings contrasting long-term effects of training with short-lived effects of subsidized employment (Strandh & Norlund, Reference Strandh and Nordlund2008), we only find short-term positive effects of contractual workplace training, and neutral or even negative effects when controlling for strong positive selection into such programmes. The former suggests that contractual workplace training to some extent successfully matches labour demand and supply, but also questions the transferability of the acquired human and social capital to other workplaces. The strong positive selection signals that employers recruit the strongest candidate for an open vacancy and initiate a contractual workplace programme to receive a wage subsidy. Consequently, policymakers might reconsider whether the employment office can act as a matchmaker between jobseekers and employers willing to participate in contractual workplace training, rather than leaving the initiative to employers who who are prone to cream-skimming. This recommendation aligns with recent evolutions in Flanders strengthening the employment office’s activities around contractual workplace training (Horemans & Ghysels, 2020; Vlaamse Regering, Reference Werk2014, Reference Werk2019). Contributing to an inconclusive body of evaluations of workplace experience programmes and temporary wage subsidies (Card et al., Reference Card, Kluve and Weber2017; Card et al., Reference Card, Kluve and Weber2010; Dahl & Lorentzen, Reference Dahl and Lorentzen2005; Gerfin & Lechner, Reference Gerfin and Lechner2002), our interpretation of the limited effectiveness of contractual workplace training in terms of selective recruitment resonates with long-standing findings of deadweight losses of wage subsidies in case participants would have also been hired in the absence of wage subsidies (Bell et al., Reference Bell, Blundell and Van Reenen1999; Marx, Reference Marx2001). Whilst public employment creation by government has routinely been proven ineffective (Card et al., Reference Card, Kluve and Weber2010, Reference Card, Kluve and Weber2017), governmental efforts to avoid cream-skimming by employers seem warranted.
To conclude, we present two pathways for future research. First, this study exploits variation in training programme components to interpret (in)effectiveness in line with theoretical mechanisms underlying these components. Building upon our theoretical interpretations of (in)effectiveness, a fruitful pathway for future research would be to measure the presumed mediators in the different programmes and model the relationship between programme participation (e.g. general classroom training), mediating variables (e.g. motivation), and job search outcomes (e.g. employment entry). The longitudinal linked register data used in this paper did not provide sufficient information to do so, yet it would strengthen our theoretical interpretations of (in)effectiveness. Second, this study indicates that unpacking ALMP training into (sub)types unveils substantial variation in effectiveness. Possible avenues to further increase the level of detail include further unpacking the groups of ALMP training programmes in even more specific categories, and studying effects of sequential or parallel participation in different programmes.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0047279423000648
Acknowledgements
This research was funded by the Research Foundation Flanders (FWO grant number G045722N), the Research Council of the University of Antwerp (BOF grant number 36985), and the VIONA chair MIA ‘Migration, Integration, Activation’ [‘Leerstoel Migratie, integratie en arbeidsmarkt’].
Competing interests
The author(s) declare none.
Open access
