Mating system and the critical migration rate for swamping selection

XIN-SHENG HU

doi:10.1017/S0016672311000127

Mating system and the critical migration rate for swamping selection

Published online by Cambridge University Press: 06 May 2011

XIN-SHENG HU

Show author details

XIN-SHENG HU*: Affiliation:
Department of Renewable Resources, 751 General Service Building, University of Alberta, Edmonton, Alberta T6G 2H1, Canada
*: *Correspondence address: 1400 College Plaza, University of Alberta, Edmonton, AB, Canada T6G 2C8. Tel: 780-248-1739. Fax: -1900. e-mail: [email protected]

Article contents

Summary
Introduction
General assumptions
Exact simulation model
Analytical model under weak selection
Discussion
References

Rights & Permissions

Summary

Crow et al. (1990) and Barton (1992) have examined the critical migration rate for swamping selection in the nuclear system. Here, I use the same methodology to examine the critical migration rate in the cytonuclear system for hermaphrodite plants with a mixed mating system. Two selection schemes for a nuclear gene (heterozygote disadvantage and directional selection) and the directional selection scheme for organelle genes are considered. Results show that under random mating, the previous results are applicable to plant species by appropriate re-parameterization of the migration rate for nuclear and paternal organelle genes. A simple complementary relationship exists between seed and pollen flow in contributing to the critical migration rate. Under the mixed mating system, the critical migration rate of seeds and pollen for nuclear and paternal organelle genes can be changed due to the effects of selection and the cytonuclear linkage disequilibrium generated by migration and inbreeding. A negative but not complementary relationship exists between seed and pollen flow in contributing to the critical migration rate, varying with the mating system. Partial selfing can also adjust the critical seed flow for the maternal organelle gene, with a small critical migration rate for species of a high selfing rate. Both concordance and discordance among cytonuclear genes can occur under certain conditions during the process of swamping selection. This theory predicts the presence of various contributions of seed versus pollen flow to genetic swamping for plants with diverse mating systems.

Type: Research Papers
Information: Genetics Research , Volume 93 , Issue 3 , June 2011 , pp. 233 - 254

DOI: https://doi.org/10.1017/S0016672311000127 [Opens in a new window]
Copyright: Copyright © Cambridge University Press 2011

1. Introduction

The joint effects of selection and migration have long been appreciated (Haldane, Reference Haldane1930; Wright, Reference Wright1931), and their importance has received considerable examinations in evolutionary theory and conservation biology. In a simple case where a balance can take place between effects of selection (soft selection) and immigration, the allele frequency in the recipient population may vary from zero (extinction) to unity (fixation), depending on the relative strengths of selection and immigration (Wright, Reference Wright1969, pp. 36–38). When the immigration rate is greater than a threshold value (the minimum migration rate required for fixing migrated alleles or genotypes), such a balance vanishes, and the immigrating allele can eventually swamp the locally dominant allele, irrespective of the adaptation or maladaptation of immigrating alleles to local habitats. This migration threshold is termed as the critical migration rate by Crow et al. (Reference Crow, Engels and Denniston1990) in addressing phase three of Wright's shifting balance theory (SBT; Wright, Reference Wright1977, pp. 443–473), and its role in impeding population fitness peak shift is extensively studied in theory (Barton, Reference Barton1992; Barton & Rouhani, Reference Barton and Rouhani1993; Rouhani & Barton, Reference Rouhani and Barton1993; Barton & Whiltlock, Reference Barton, Whiltlock, Hanski and Gilpin1997).

In flowering plants, this critical migration rate may include both haploid pollen and diploid seed flow. The existing theories are mainly developed for animal species except that Haldane (Reference Haldane1930, p. 229) indicated the necessity of generalizing the critical ratio of selection to immigration to inbreeding plants. The contribution of seed and pollen dispersal to gene flow is related to both the mode of gene inheritance and the reproductive system of species (Ennos, Reference Ennos1994). For maternally inherited organelle genomes, gene flow is mediated by seed dispersal only. This can occur for chloroplast genomes in angiosperms and mitochondrial genomes in both most angiosperms and gymnosperms (Mogenson, Reference Mogenson1996). For the nuclear and paternally inherited organelle genomes, gene flow can be mediated through both seed and pollen dispersal. Such differences in gene flow lead to unequal spreading rates in space, evidenced from differential introgression between cytonuclear genes in hybrid zones (Avise, Reference Avise1994, pp. 293–295). When combined with effects of natural selection, different critical migration rates are expected in the presence of unequal selection strengths among the three genomes, as implied from the discordance among cytonuclear clines (Hu & Li, Reference Hu and Li2002). In practical terms, the critical migration rate is of particular interest for insights into the contamination of natural forests resulting from the immigration of artificial plantations (genetically modified) with high frequencies of target genes (Ellstrand, Reference Ellstrand2003; Hu et al., Reference Hu, Zeng and Li2003). Contamination may come from genes on either nuclear or organelle genomes, or both, and the required migration rate for gene swamping in natural populations could be different through seed or pollen dispersal.

Ample evidence indicates that distinct population genetic structure, assessed using nuclear and/or organelle genome markers, exists in species of various mating systems (Ennos, Reference Ennos1994; Ennos et al., Reference Ennos, Sinclair, Hu, Langdon, Hollingsworth, Bateman and Gornall1999). A plant mating system directly affects pollen flow, and a plant population with a high outcrossing rate often receives a high immigration rate of alien pollen (Pakkad et al., Reference Pakkad, Ueno and Yoshimaru2008). For a predominantly outcrossing species, pollen dispersal often plays a dominant role in bringing about gene flow among mature forest populations (Chen et al., Reference Chen, Fan and Hu2008), enhancing the migration rate of both nuclear and paternal organelle genes. For the predominant selfing species, pollen dispersal only contributes a small portion of gene flow, enhancing comparable migration rates among nuclear, paternal and maternal organelle genes. Thus, the mating system as an important agent is involved in altering the critical migration rate through adjusting effective pollen dispersal (Haldane, Reference Haldane1930).

In addition, the joint effects of migration and the mating system facilitate interaction between nuclear and organelle genomes. Previous theories show that seed and pollen dispersal can generate cytonuclear linkage disequilibrium (LD) in the mainland–island model (Asmussen & Schnabel, Reference Asmussen and Schnabel1991; Schnabel & Asmussen, Reference Schnabel and Asmussen1992) and in hybrid zones (Arnold, Reference Arnold1993). Such an outcome can also be implied from the presence of stable LD between selective nuclear loci in structured populations (Li & Nei, Reference Li and Nei1974; Slatkin, Reference Slatkin1975). The cytonuclear LD as a biological barrier, generated by seed and pollen dispersal, can impede the spread of nuclear or organelle genes in space (Hu, Reference Hu2008, Reference Hu2010). Partial selfing reinforces the generation of cytonuclear LD in plant species (Asmussen et al., Reference Asmussen, Arnold and Avise1987, Reference Asmussen, Arnold and Avise1989; Maroof et al., Reference Maroof, Zhang, Neale and Allard1992; Asmussen & Orive, Reference Asmussen and Orive2000), similar to its function in generating LD between nuclear sites. Therefore, it is of both theoretical and practical significance to examine the similarity and differences in the critical migration rate among cytonuclear genes.

The purpose of this study is to bring together different combinations of cytonuclear systems to show how mating pattern changes the critical migration rate of seeds and pollen. This necessarily generalizes the theory to the cytonuclear system from the previous studies on the solely nuclear system where LD is negligible under the assumption of weak selection and random mating (Crow et al., Reference Crow, Engels and Denniston1990; Barton, Reference Barton1992). The cytonuclear system cannot be approximated simply by setting the recombination rate as ½ in the nuclear system, as indicated from the study of comparing two-locus LD in finite populations (Fu & Arnold, Reference Fu and Arnold1992) or from the studies on spreading neutral alleles in structured populations (Hu, Reference Hu2008, Reference Hu2010). The mixed mating system can exert distinct effects on diploid nuclear genes from on haploid organelle genes due to different modes of inheritance in addition to generating cytonuclear LD. Also, the functional epistasis between cytonuclear genes cannot be simply approximated in theory by the epistasis between nuclear sites due to the asymmetric mode of inheritance between cytonuclear genes. Cytonuclear epistasis may take place at the different levels of metabolic pathways for the products from organelle and nuclear genomes (Elo et al., Reference Elo, Lyznik, Gonzalez, Kachman and Mackenzie2003; Wolf, Reference Wolf2009), due to the long-term endosymbiotic co-evolution in a plant cell (Stoebe et al., Reference Stoebe, Hansmann, Goremykin, Kowallik, Martin, Hollingsworth, Bateman and Gornall1999; Rand et al., Reference Rand, Haney and Fry2004). Thus, it is important to consider the genetic swamping effects in shifting the fitness of the cytonuclear system.

In my analysis of the critical migration rate of seeds and pollen, three cytonuclear systems are separately addressed: the nuclear and maternally or paternally organelle genome system; the nuclear and maternal and paternal organelle genome system. In the following sections, I begin by describing the exact general model where the critical migration rates of seed and pollen can only be estimated through numerical simulations. I then separately examine each system under weak selection where the analytical approximations for critical migration rates are derived. These approximations are checked through the exact simulation results. Inferences on how the mating system shapes the critical migration rate of seeds and pollen are drawn from both analytical and simulation results under the cytonuclear system.

2. General assumptions

Consider one natural population of a hermaphrodite plant species, with constant immigration rates of pollen (m _P) and seeds (m _S) per generation from a source population. The source population may have either the same or different fitness from the recipient population. For simplicity, the reverse directional migration is not considered or is assumed to be not important (Fig. 1). Mutation rate is assumed to be very much smaller, and its effect is excluded. Each population size is assumed to be large so that genetic drift effect is negligible, similar to previous studies (Haldane, Reference Haldane1930; Crow et al., Reference Crow, Engels and Denniston1990; Barton, Reference Barton1992). The source population is assumed to be stable in gametic and genotypic frequencies. At the gametophyte stage, pollen and ovules are subject to natural selection (gametic selection) before combined to produce seeds. The same mating system in each population is assumed, with an arbitrary selfing rate. The plant life cycle follows a sequence of events: pollen and ovules generation, pollen flow, selection at the gametophyte stage, mixed mating, seed flow and selection at the sporophyte stage.

Fig. 1. A recipient population of a hermaphrodite plant species is subject to the influences of constant immigration of pollen and seeds from a source population. The source population is assumed to be stable in gametic and genotypic frequencies. At the gametophyte stage, chloroplast DNA (cpDNA) in conifers and nuclear DNA (nDNA) can be dispersed through pollen flow. At the sporophyte stage, all three genomes (mitochondrial DNA (mtDNA), cpDNA and nDNA), can be dispersed through seed flow. The present model considers the effects of unidirectional gene flow on swamping selection in the recipient population.

Consider three diallelic genes with contrasting modes of inheritance (bi-parental, paternal and maternal). Again, the three-gene cytonuclear system can naturally occur in some gymnosperms where chloroplast and mitochondrial genomes are paternally and maternally inherited, respectively (Mogenson, Reference Mogenson1996). When the effects of one organelle genome are neglected, a two-gene cytonuclear system can occur between nuclear and chloroplast genomes in angiosperms or between nuclear and mitochondrial genomes in both angiospersms and gymnosperms. Selection strength can be either strong or weak, and cytonuclear epistasis at either the gametophyte or the sporophyte stage, or both, is allowed in theory.

3. Exact simulation model

Let A ₁ and A ₂ be the two alleles on the diploid nuclear genomes, B ₁ and B ₂ be the two alleles on the haploid organelle genome with paternal inheritance, and C ₁ and C ₂ be the two alleles on the haploid organelle genome with maternal inheritance. These genes are selectively non-neutral, and population fitness is decided by the joint cytonuclear genes. In the recipient population, let $p_{A_{i} A_{j} B_{k} C_{l} }$ , $p_{A_{i} A_{j}}$ , $p_{B_{k}$ and $p_{C_{l}}$ be the frequencies of cytonuclear genotype A_iA_jB_kC_l , nuclear genotype A_iA_j , organelle genotypes B_k and C_l (i,j,k,l=1,2; i⩽j), respectively. In order to extend the definitions of genotypic cytonuclear LD by Asmussen et al. (Reference Asmussen, Arnold and Avise1987), the genotypic cytonuclear LD in the four-gene case, $D_{A_{i} A_{j} B_{k} C_{l}}$ , can be expressed as

(1)

$\eqalign{D_{A_{i} A_{j} B_{k} C_{l} } \equals \tab p_{A_{i} A_{j} B_{k} C_{l} } \minus p_{A_{i} A_{j} } D_{B_{k} C_{l} } \cr \tab \minus p_{B_{k} } D_{A_{i} A_{j} C_{l} } \minus p_{C_{l} } D_{A_{i} A_{j} B_{k} } \comma}$

where $D_{B_{k} C_{l}}$ is the LD between B_k and C_l (= $p_{B_{k} C_{l}} \minus p_{B_{k}} p_{C_{l}}$ ), $D_{A_{i} A_{j} C_{l}}$ is the LD between A_iA_j and C_l (= $p_{A_{i} A_{j} C_{l}}\minus p_{A_{i} A_{j} p_{C_{l}}$ ) and $D_{A_{i} A_{j} B_{k}}$ is the LD between A_iA_j and B_k (= $p_{A_{i} A_{j} B_{k}} \minus p_{A_{i} A_{j}} p_{B_k}}$ ). The above expression can be readily derived using Bennett's method (Bennett, Reference Bennett1954). Some properties of cytonuclear LD are $\sum _{i\comma j \equals \setnum{1}\semi i \les j}^{\setnum{2}} D_{A_{i} A_{j} B_{k} C_{l} } \equals \sum _{i\comma j \hskip1pt \equals \hskip1pt \setnum{1}\semi i \les j}^{\setnum{2}}\, D_{A_{i} A_{j} B_{k} } \equals \sum _{i\comma j \hskip1pt \equals \hskip1pt \setnum{1}\semi i \hskip1pt \les \hskip1pt j}^{\setnum{2}}\ D_{A_{i} A_{j} C_{l} } \equals \sum _{k \hskip1pt \equals \hskip1pt \setnum{1}}^{\setnum{2}} \ D_{A_{i} A_{j} B_{k} } \equals <$> <$>\sum _{l \equals \setnum{1}}^{\setnum{2}} D_{A_{i} A_{j} C_{l} } \equals \sum _{k \equals \setnum{1}}^{\setnum{2}} D_{B_{k} C_{l} } \equals \sum _{l \equals \setnum{1}}^{\setnum{2}} D_{B_{k} C_{l} } \equals 0$ . The following relationships between gametic and genotypic cytonuclear LDs hold: $D_{A_{i} B_{k}} \equals D_{A_{i} A_{i} B} \plus D_{A_{1} A_{2} B_{k}} \sol 2$ , $D_{A_{i} C_{l}} \equals D_{A_{i} A_{i} C_{l}} \plus D_{A_{1} A_{2} C_{l}} \sol 2$ and $D_{A_{i} B_{k} C_{l}} \equals D_{A_{i} A_{i} B_{k} C_{l}} \plus <$> <$>D_{A_{1} A_{2} B_{k} C_{l} \sol 2$ .

Similarly, let $Q_{A_{i} A_{j} B_{k} C_{l}}, Q_{A_{i} A_{j}}, Q_{B_{k}}$ and Q_Cl be the frequencies of cytonuclear genotype A_iA_jB_kC_l , nuclear genotype A_iA_j , cytoplasmic genotypes B_k and C_l (i, j, k, l=1, 2; i⩽j) in the source population, respectively. Let $\bars{D}_{B_{k} C_{l} }$ , $\bars{D}_{A_{i} A_{j} C_{l} }$ and $\bars{D}_{A_{i} A_{j} B_{k} }$ be the LDs between B_k and C_l , between A_iA_j and C_l and between A_iA_j and B_k , respectively. $\bars{D}_{A_{i} A_{j} B_{k} C_{l} }$ and some properties about cytonuclear LD in the source population can be derived in the same way as in the recipient population.

In the gametophyte stage, let $w_{A_{i} B_{k}}$ and $w_{A_{i} C_{l}}$ be the fitness for gametes A_iB_k in pollen and A_iC_l in ovules, respectively. The average fitness in pollen and ovules, denoted by w _P and w _O, respectively, can be calculated using the conventional method $w_{\rm P} \equals <$> <$>\sum _{i \equals \setnum{1}}^{\setnum{2}} \sum _{k \equals \setnum{1}}^{\setnum{2}} w_{A_{i} B_{k} } p_{A_{i} B_{k} }^{\ast } \quad {\rm and}\quad w_{\rm O} \equals \sum _{i \equals \setnum{1}}^{\setnum{2}} \sum _{l \equals \setnum{1}}^{\setnum{2}} w_{A_{i} C_{l} } p_{A_{i} C_{l} }^{\ast }$ , where $p_{A_{i} B_{k}^{\ast}}$ and $p_{A_{i} C_{l}^{\ast}}$ are the gametic frequencies after pollen dispersal. Let $r_{A_{i} B_{k}} \equals w_{A_{i} B_{k}} \sol \lpar w_{A_{1} B_{k}} \plus w_{A_{2} B_{k}}$ ), the relative ratio of gametic fitness in pollen, and $r_{A_{i} C_{l}} \equals w_{A_{i} C_{l}} \sol ( w_{A_{1} C_{l}} \plus w_{A_{2} C_{l}})$ , the relative ratio of gametic fitness in ovules. In the sporophyte stage, let $w_{A_{i} A_{j} B_{k} C_{l}}$ be the fitness for A_iA_jB_kC_l (i, j=1, 2, i⩽j; k, l=1, 2). The average fitness in the sporophyte stage, w, can be calculated by $w \equals \sum _{i \equals \setnum{1}}^{\setnum{2}} \sum _{j \equals \setnum{1}\comma j \les i}^{\setnum{2}} \sum _{k \equals \setnum{1}}^{\setnum{2}} \sum _{l \equals \setnum{1}}^{\setnum{2}} w_{A_{i} A_{j} B_{k} C_{l} } p_{_{{A_{i} A_{j} B_{k} C_{l} }} }^{\ast }$ , where $p_{A_{i} A_{j} B_{k} C_{l}^{\ast}}$ is the genotypic frequency after seed dispersal.

Let α (0⩽α⩽1) be the selfing rate. Following the life cycle mentioned above, the cytonuclear genotypic frequency after selection in the sporophyte stage, denoted by $p_{A_{i} A_{i} B_{k} C_{l}^{\ast \ast}}\ \lpar \equals w_{A_{i} A_{j} B_{k} C_{l}} p_{A_{i} A_{j} B_{k} C_{l}^{\ast}} \sol w \rpar$ for A_iA_iB_kC_l (i, k, l=1, 2), can be expressed as

(2)

$\eqalign{ p_{A_{i} A_{i} B_{k} C_{l} }^{\ast \ast } \equals \tab \lpar 1 \minus m_{\rm S} \rpar {{w_{A_{i} A_{i} B_{k} C_{l} } } \over w} \cr \tab\times \left( {\lpar 1 \minus \alpha \rpar {{w_{A_{i} B_{k} } w_{A_{i} C_{l} } } \over {w_{\rm O} w_{\rm P} }}\lpar \lpar 1 \minus m_{\rm P} \rpar p_{A_{i} B_{k} } p_{A_{i} C_{l} } } \right. \cr \tab \plus m_{\rm P} Q_{A_{i} B_{k} } p_{A_{i} C_{l} } \rpar \cr \tab \left. { \plus \alpha \lpar p_{A_{i} A_{i} B_{k} C_{l} } \plus r_{A_{i} B_{k} } r_{A_{i} C_{l} } p_{A_{\setnum{1}} A_{\setnum{2}} B_{k} C_{l} } \rpar } \Bigg\right) \cr \tab \plus m_{\rm S} {{w_{A_{i} A_{i} B_{k} C_{l} } } \over w}Q_{A_{i} A_{i} B_{k} C_{l} }. \hskip74pt$

Similarly, the general expression for $p_{A_{1} A_{2} B_{k} C_{l}} ^{\ast \ast}$ (k, l=1, 2 ) is derived as

(3)

$\eqalign{ p_{A_{\setnum{1}} A_{\setnum{2}} B_{k} C_{l} }^{\ast \ast } \equals \tab \lpar 1 \minus m_{\rm S} \rpar {{w_{A_{\setnum{1}} A_{\setnum{2}} B_{k} C_{l} } } \over w} \cr \tab \times \left( {\lpar 1 \minus \alpha \rpar {{w_{A_{\setnum{1}} B_{k} } w_{A_{\setnum{2}} C_{l} } } \over {w_{\rm O} w_{\rm P} }}\lpar \lpar 1 \minus m_{\rm P} \rpar p_{A_{\setnum{1}} B_{k} } p_{A_{\setnum{2}} C_{l} } } \right. \cr \tab \plus m_{\rm P} Q_{A_{\setnum{1}} B_{k} } p_{A_{\setnum{2}} C_{l} } \rpar \cr \tab \plus \lpar 1 \minus \alpha \rpar {{w_{A_{\setnum{2}} B_{k} } w_{A_{\setnum{1}} C_{l} } } \over {w_{\rm O} w_{\rm P} }}\lpar \lpar 1 \minus m_{\rm P} \rpar p_{A_{\setnum{2}} B_{k} } p_{A_{\setnum{1}} C_{l} } \cr \tab \plus m_{\rm P} Q_{A_{\setnum{2}} B_{k} } p_{A_{\setnum{1}} C_{l} } \rpar \cr \tab \left. { \plus \alpha \lpar r_{A_{\setnum{1}} B_{k} } r_{A_{\setnum{2}} C_{l} } \plus r_{A_{\setnum{2}} B_{k} } r_{A_{\setnum{1}} C_{l} } \rpar p_{A_{\setnum{1}} A_{\setnum{2}} B_{k} C_{l} } } \Bigg\right) \cr \tab \plus m_{\rm S} {{w_{A_{\setnum{1}} A_{\setnum{2}} B_{k} C_{l} } } \over w}Q_{A_{\setnum{1}} A_{\setnum{2}} B_{k} C_{l} } .\hskip69pt$

The above general model has several advantages. First, it allows an arbitrary level of selfing rate and can be used to assess the critical migration rate for any mating system. Second, no assumptions are imposed on the mode of joint selection for cytonuclear genotypes. Selection can be either weak or strong, or either the presence of cytonuclear epistasis or a purely linear additive–cytonuclear–viability model. Third, the general model considers natural selection at either the gametophyte or the sporophyte stage or both (Tanksley et al., Reference Tanksley, Zamir and Rick1981; Clark, Reference Clark1984). This may be of significance for some plant species where genes expressed in the sporophyte stage (~60% of structural genes) can also be expressed and potentially subject to selection in the gametophyte stage (Mulcahy et al., Reference Mulcahy, SariGorla and Mulcahy1996). Selection at the gametophyte stage can modify the critical migration rates for seeds and pollen. Finally, the model allows different cytonuclear systems, such as the system of nuclear and paternal (or maternal) organelle genomes or the system of sole organelle genomes, depending on the type of system in question.

Although the recursion expression for each allele frequency can be derived from the above general model, the migration rates of seeds and pollen are difficult to solve as the function of genotypic and allelic frequencies and selection coefficients. Given a set of parameters, the above exact model can be applied to searching for the critical migration rates of seeds and pollen in the cytonuclear system. The simulation results are not separately presented but are used to check the analytical approximations under weak selection.

4. Analytical model under weak selection

In order to derive the analytical expressions for the critical migration rates of seeds and pollen, a linear–additive–viability model with weak selection is considered. Let the gametic fitness be $w_{A_{i} B_{k}} \equals 1 \plus s_{A_{i}} \plus s_{B_{k}}$ and $w_{A_{i} C_{l}} \equals 1 \plus s_{A_{i}} \plus s_{C_{l}}$ , where $s_{A_{i}$ , $s_{B_{k}}$ and $s_{C_{l}}$ are the selection coefficients for alleles A_i , B_k and C_l , respectively. Similarly, let $w_{A_{i} A_{j} B_{k} C_{l}} \equals 1 \plus <$> <$>s_{A_{i} A_{j}} \plus s_{B_{k}} \plus s_{C_{l}}$ , where $s_{A_{i} A_{j}}$ is the selection coefficient for nuclear genotype A_iA_j . Under weak selection, all terms containing the second or higher orders of the selection coefficients are neglected. The immigration rates of seeds and pollen are assumed to be small as well so that the balance between the effects of migration and selection can be attained. The terms containing the second or higher orders of the migration rate (m _P², m _S² or m _Sm _P or higher orders) or the products of the migration rate and selection coefficient (sm _P or sm _S) are neglected. For simplicity, the notation for alleles and subscripts is changed by A for A ₁ and a for A ₂, …, and c for C ₂. The Appendix gives the recursion equations for allele frequencies and cytonuclear LD under a general mixed mating system.

Throughout this section, suppose that genotype AABC is fixed in the source population, and that genotype aabc is initially fixed in the recipient population. Frequencies for A, B and C are equal to unity in the source population, without cytonuclear LD, similar to the single locus case investigated by Barton (Reference Barton1992) for the nuclear system. In order to examine the swamping effects of migration, alleles A, B and C that are adaptive to the source population are assumed to be maladaptive to the recipient population although other soft selection schemes can be examined in different scenarios. Two selection schemes for the migrating nuclear genes in the recipient population are considered: heterozygote disadvantage where the heterozygotes are less fit than the homozygotes (s_AA =0, s_Aa =−s ₂, and s_aa =0), and directional selection where allele A is maladaptive (s_AA =−2s ₂, s_Aa =−s ₂ and s_aa =0). Selection in the gametophyte stage is set as s_A =−s ₁ and s_a =0 with directional selection, but is not considered in the scheme of heterozygote disadvantage. Selection coefficient for the organelle genomes is set as −s_B for B and 0 for b, and −s_C for C and 0 for c in both the gametophyte and sporophyte stages. The consequence for the migration swamping effects is that alleles A, B and C are fixed in the recipient population. In the following, the critical migration rates are analytically examined in three different cytonuclear systems. In each system, two specific mating systems (α=0 and 1) are firstly presented, and a mixed mating system is then examined.

(i) Nuclear and maternally inherited organelle genome system

(a) Random mating

The theory in this subsubsection is applicable to the nuclear and mitochondrial genome system in flowering plants or to the nuclear and chloroplast genome system in angiosperms with complete outcrossing (α=0). In the presence of heterozygote disadvantage, the change for the nuclear allele frequency, Δp_A =p_A ^**−p_A , is derived from eqn (A1) in the Appendix:

(4)

$\rmDelta p_{A} \equals \lpar m_{\rm S} \plus m_{\rm P} \sol 2\rpar p{}_{a} \minus \lpar p_{a} \minus p_{A} \rpar s_{\setnum{2}} p_{A} p_{a} \minus s_{C} D_{AC}.$

The change for maternal organelle allele C, Δp_C =p_C ^**−p_C , can be derived from eqn (A4) in the Appendix:

(5)

$\rmDelta p_{C} \equals m_{\rm S} p_{c} \minus 2s_{C} p_{C} p_{c} \minus \lpar p_{a} \minus p_{A} \rpar s_{\setnum{2}} D_{AC}.$

Note that 2s_C in the above equation comes from selection in both the gametophyte and sporophyte stages, and it becomes s_C in the absence of gametic selection. It can be seen that the frequencies of alleles A and C can consistently increase as long as their changes are greater than zero (Δp_A>0 and Δp_C>0), leading to the eventual fixation of alleles A and C in the recipient population.

Suppose that cytonuclear LD is mainly maintained by migration, the recombination rate (=1/2) and the mating system. This consideration is similar to LD between nuclear sites in the hybrid zone studies where LD can be approximated by the balance between migration and recombination (Kruuk et al., Reference Kruuk, Baird, Gale and Barton1999; Barton & Shpak, Reference Barton and Shpak2000; Hu, Reference Hu2005). Here, the mating system plays a role similar to recombination in the nuclear system in maintaining cytonuclear LD. A high selfing rate (or a high outcrossing rate) enhances (or erodes) cytonuclear LD. From eqn (A7) in the Appendix, the steady-state cytonuclear LD is approximated by D_AC≈2m _Sp _ap _c, which is of the order similar to m _S.

Substituting D_AC into eqn (4) at the steady state and neglecting the term with the order of migration times selection yield

(6)

$m_{\rm S} \plus m_{\rm P} \sol 2 \equals \lpar p_{\rm a} \minus p_{\rm A} \rpar s_{\setnum{2}} p_{\rm A}.$

The critical migration rate for joint seeds and pollen is (m _S+m _P/2)*=s ₂/8, essentially the same as Barton's result (Barton, Reference Barton1992, p. 552) except that migration here refers to the compounded seed and pollen flow. For a given selection coefficient s ₂, the sum of m _S and m _P is a constant. A simple complementary relationship exists between seed and pollen flow in contributing to the total critical migration rate. Similarly, substituting D_AC into eqn (5) at the steady state and neglecting the term with the order of migration times selection yield

(7)

$m_{\rm S} \equals 2s_{C} p_{C}.$

The critical migration rate for seeds is m _S*=2s _C, the maximum value of function 2s_Cp_C . It can be seen that the effects of cytonuclear LD can be neglected under random mating with weak selection.

In order to check the above analytical results, exact simulations are conducted according to the general model described in the preceding section. The critical migration rates are calculated using the binary search method detailed by Barton (Reference Barton1992, p. 557). Given the settings of selection coefficients, an analytical estimate of the migration rate is used as the starting value and then adjusted by the binary search procedure. When the difference between the upper (starting fixation of migrated alleles) and lower (sufficiently close to fixation of migrated alleles) values is smaller than their average divided by 20, the average of the upper and lower values is used as the estimate, ensuring the estimate within 5% of the true value. The dynamics for allele frequency is considered to reach steady state when its changes within 10 consecutive generations are sufficiently small (<10⁻⁸; Crow et al., Reference Crow, Engels and Denniston1990).

Figure 2 a shows the comparisons between the exact simulation and analytical approximation results for the maternal organelle gene. Results indicate that slight biases exist when selection coefficients are large, but good agreement between them exists when selection coefficients are small. Figure 2 b shows the comparisons between the exact simulation and analytical approximation results for the nuclear gene. Since seed and pollen migration rates are compounded, the critical migration rates of seeds are estimated, given a migration rate of pollen, or the critical migration rates of pollen are estimated, given a migration rate of seeds. In each case, the analytical approximation performs well for the compounded migration rates of seeds and pollen (Fig. 2 b).

Fig. 2. (a) The critical migration rate of seeds as a function of selection strength for the maternal organelle gene. The dashed line with opened circles gives the analytic approximation, m _S*=2s_C from eqn (3). The solid line with closed circles is the exact simulation result. (b) A comparison of estimates of the critical migration rate from exact simulation and from analytical approximation. The dashed line with opened circles is the analytical approximation, (m _S+m _P/2)*=s ₂/8 from eqn (4). The line with closed circles is the exact simulation result for the critical migration rate of pollen, given a set of migration rates of seeds. The line with closed triangles is the exact simulation result for the critical migration of seeds, given a set of migration rate of pollen. The selection coefficient in each case is set as s ₂=−0·08.

The relationships between m _S* and (m _S+m _P/2)* are complex in fixing the immigrated maternal and nuclear alleles. m _S* for the maternal organelle gene may be smaller than (m _S+m _P/2)* in the presence of weaker selection against allele C than against nuclear allele A. If the migration rate of seeds for both nuclear and maternal organelle genes is equal to m _S*, the migration rate of pollen may be estimated as (1/4−4γ)s ₂, where γ=s_C/s ₂. When the selection against allele C is much stronger than selection against allele A, m _S* may be greater than (m _S+m _P/2)*, and maternal organelle allele C is expected to be fixed earlier than nuclear allele A.

A concordance between nuclear and maternal allele frequencies is possible when certain conditions regarding selection strength, migration and the mating system are met. When a coincidence between cytonuclear allele frequencies occurs, let p_A =p_C =p. The recursion equation for the change of p can be approximated by combining eqns (A1) and (A4) in the Appendix. At the steady state, we obtained 2m _S+m _P/2=2(1+2γ−2p)ps ₂. The critical value for compounded migration rate of seeds and pollen (m _S+m _P/4)* is derived as (1+2γ)²s ₂/16, which is difficult to numerically check using the exact simulation model described in the preceding section.

With directional selection, the effects of cytonuclear LD are negligible in altering allele frequency. The steady-state equation for the frequency of maternal organelle allele remains the same as eqn (7), yielding the same critical migration rate, i.e. m _S*=2s_C . The steady-state equation for the nuclear allele frequency becomes

(8)

$m_{\rm S} \plus m_{\rm P} \sol 2 \equals \lpar 1 \plus \rho \rpar s_{\setnum{2}} p_{A} \comma$

where ρ=s ₁/s ₂, the ratio of selection coefficients between the gametophyte and sporophyte stages. The critical migration rate for compounded seeds and pollen flow is (m _S+m _P/2)*=(1+ρ)s ₂, greater than the results in the scheme of heterozygote disadvantage. It reduces to the result obtained by Barton (Reference Barton1992, p. 552) in the absence of gametic selection (ρ=0). Here, m _S* and the critical migration rate for compounded seed and pollen flow, (m _S+m _P/2)*, are independently responsible for swamping maternal and nuclear alleles. m _S* may be greater than (m _S+m _P/2)* under certain conditions. Exact simulations confirm a very good performance for the analytical estimates of critical migration rate of pollen, given a migration rate of seeds, or the analytical estimates of critical migration rate of seeds, given a migration rate of pollen (data not given here).

When a coincidence between cytonuclear allele frequencies (p_A =p_C =p) occurs, the steady-state equation for p can be approximated by combining eqns (A1) and (A4) in the Appendix, i.e. 2m _S+m _P/2=(1+ρ+2γ)ps ₂. The critical migration rate for compounded seed and pollen flow becomes (m _S+m _P/4)*=(1+ρ+2γ)s ₂/2, unequal to the results in the scheme of heterozygote disadvantage.

(b) Complete selfing

Complete selfing (α=1) removes the impacts of pollen flow. Cytonuclear LD is mainly generated by seed flow and selfing. From eqn (A7) in the Appendix, D_AC is approximated by p_ap_c . In the scheme of heterozygote disadvantage, the change for the frequency of allele C is

(9)

$\rmDelta p_{C} \equals m_{\rm S} p_{c} \minus s_{C} p_{C} p_{c} \minus s_{\setnum{2}} D_{AaC} \sol 2.$

In the above equation, the selection component comes from the selection in the sporophyte stage. From eqn (A11) in the Appendix, the genotypic cytonuclear LD can be approximated by $D_{AaC} \equals 2\lpar 1 \minus \alpha \rpar D_{AC} \sol \lpar 2 \minus \alpha \rpar \plus \rmLambda _{\setnum{1}} \lpar s_{ \bullet } \comma m_{ \bullet } \rpar \comma$ where $\rmLambda _{\setnum{1}} \lpar s_{ \bullet } \comma m_{ \bullet } \rpar$ is the function of the first-order migration rate and selection coefficients. The third term on the right-hand side of eqn (9) is negligible according to the assumption of weak selection. Thus, the critical migration rate for seeds is m _S*=s_C , a half value of the results under random mating due to the removal of selection effects in the gametophyte stage.

From eqn (A1) in the Appendix, the steady-state equation for the nuclear allele frequency becomes

(10)

$0 \equals m_{\rm S} p{}_{a} \minus s_{\setnum{2}} \lpar p_{a} \minus p_{A} \rpar p_{Aa} \sol 4 \minus s_{C} D_{AC}.$

According to eqn (A2) in the Appendix, the steady-state heterozygosity at the nuclear site can be approximated by $p_{Aa} \equals 2\lpar 1 \minus \alpha \rpar p_{A} p_{a} \sol \lpar 1 \minus \alpha \sol 2\rpar \plus <$> <$>\rmLambda _{\setnum{2}} \lpar s_{ \bullet } \comma m_{ \bullet } \rpar$ , where $\rmLambda _{\setnum{2}} \lpar s_{ \bullet } \comma m_{ \bullet } \rpar$ is the function of the first-order migration rate and selection coefficients. Substituting p_Aa and D_AC in eqn (10) yields the same steady-state equation as the maternal allele frequency, with the critical migration rate for seeds, m _S*=s_C , irrespective of the selection pressure at the nuclear site. This indicates the coincidence between maternal and nuclear allele frequencies, i.e. p_A =p_C =p, with approximately the same speed towards fixation.

Figure 3 a shows the same trajectory for the change of migrated maternal (C) and nuclear (A) allele frequencies in the recipient population, calculated from the exact general model. The migrated alleles gradually replace the pre-existed alleles until fixation. Figure 3 b shows a good agreement between the simulation results and the analytical approximation m _S* under the complete selfing.

Fig. 3. (a) A coincidence for the change of migrated maternal organelle (C) or nuclear (A) allele frequencies was observed from the exact general model in the complete selfing system, with the parameter settings s_C =0·02 and m _S=0·02. (b) Critical migration rate of seeds as a function of selection strength. The dashed line with opened circles gives the analytical approximation. The closed triangles are the exact simulation results.

With directional selection at the nuclear site, the steady-state equation for the maternal organelle allele frequency is different from eqn (9),

(11)

$0 \equals m_{\rm S} p_{c} \minus s_{C} p_{C} p_{c} \minus 2s_{\setnum{2}} D_{AC}.$

By substituting D_AC into eqn (11), the critical migration rate for seeds is derived as m _S*=(2+γ)s ₂. The steady-state equation for the nuclear allele frequency from eqn (A1) in the Appendix after substituting D_AC by p_ap_c yields the same critical migration rate. Simulation results confirm that m _S*=(2+γ)s ₂ is in very good agreement with the estimate from the exact model (data are not given here). Furthermore, a coincidence between maternal and nuclear allele frequencies occurs with this critical migration rate. Compared with the results for heterozygote disadvantage, a higher critical migration rate is needed for swamping selection with directional selection.

(c) Mixed mating system

From eqn (A7) in the Appendix, the steady-state cytonuclear LD can be approximated by

(12)

$D_{AC} \equals {{2m_{\rm S} } \over {1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S} }}p_{a} p_{c}.$

Equation (12) basically reflects the balance between the effects of migration and selfing that generate cytonuclear LD and the effects of recombination (1/2) that erodes cytonuclear LD, analogous to LD in the nuclear system except for the inclusion of the mating system effects (Hu, Reference Hu2005, p. 121). This is an intermediate case of the above two specific cases. In random mating (α=0), eqn (12) can be approximated by D_AC≈2m _Sp_ap_c . In the complete selfing case (α=1), eqn (12) can be approximated by D_AC =p_ap_c . For many plant species with the mixed mating system, the condition of 1−α>>(1+α)m _S is appropriate. Numerical assessment indicates that plants of wide range of selfing rates meet this condition, given that migration rate is of the order similar to weak selection. Under this case, cytonuclear LD can be further simplified as D_AC≈2m _Sp_ap_c/(1−α). However, the condition of 1−α<<(1+α)m _S is applicable to the plant species with a predominant selfing system, and this leads to the approximation of D_AC =2p_ap_c/(1+α). For a more accurate calculation, these conditions are not separately addressed.

In the scheme of heterozygote disadvantage, substitution of D_AC into eqn (A4) in the Appendix yields the steady-state equation for the maternal organelle allele frequency:

(13)

$\eqalign{\tab \left( {1 \minus {{4\lpar 1 \minus \alpha \rpar } \over {\lpar 2 \minus \alpha \rpar \lpar 1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S} \rpar }}s_{\setnum{2}} p_{a} \lpar p_{a} \minus p_{A} \rpar } \right)\cr \tab \quad m_{\rm S} \equals \lpar 2 \minus \alpha \rpar s_{C} p_{C}.}$

The maximum value of the right-hand side of eqn (13) is (2−α)s_C . The allele frequency p_C consistently increases until the fixation of allele C as long as the left-hand side of eqn (13) is greater than (2−α)s_C . However, the left-hand side of eqn (13) as a function of p_a indicates that an array of migration rate of seeds can be generated to meet the above condition. Note that searching for the critical migration rate m _S* is different from searching for the maximum m _S as the function of p_A and p_C . Here, consider the critical migration rate of seeds at the point p_a =1/4, where −p_a (p_a−p_A ) has a maximum value of 1/8. Given the selfing rate and selection coefficients, the critical migration rate of seeds can be numerically solved (e.g., using the iterative method) from

(14)

$\hskip-2.2pt \left( {1 \plus {{\lpar 1 \minus \alpha \rpar s_{\setnum{2}} } \over {2\lpar 2 \minus \alpha \rpar \lpar 1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S}^{\ast } \rpar }}} \right)m_{\rm S}^{\ast } \equals \lpar 2 \minus \alpha \rpar s_{C}.$

Figure 4 a shows that the estimates of critical migration rate of seeds from the exact model with the binary search method are in good agreement with the results predicted from eqn (14) under various selfing rates.

From eqn (A1) in the Appendix and the use of the approximation for D_AC , the steady-state equation for the nuclear allele frequency can be written as

(15)

$\eqalign{ \tab \left( {1 \minus {{2s_{C} } \over {1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S} }}p_{c} } \right)m_{\rm S} \plus {{1 \minus \alpha } \over 2}m_{\rm P} \cr \tab \quad \equals \lpar 1 \minus \alpha \rpar \left( {1 \plus {{\alpha ^{\setnum{2}} } \over {2 \minus \alpha }}} \right)s_{\setnum{2}} \lpar 1 \minus 2p_{A} \rpar p_{A}. \cr}$

This compounded critical migration rate changes with the mating system. The maximum value for the right-hand side of eqn (15) (as the function of p_A ) is ${\textstyle{1 \over 8}}\lpar 1 \minus \alpha \rpar \lpar 1 \plus {\textstyle{{\alpha ^{\setnum{2}} } \over {2 \minus \alpha }}}\rpar s_{\setnum{2}}$ at p_A =1/4. The proportion of seed flow in the left-hand side of eqn (15) is the function of p_c . Thus, the critical migration rate for the compounded seed and pollen flow is expressed as

(16)

$\eqalign{ \tab \left( {\left( {1 \minus {{2s_{C} } \over {1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S} }}} \right)m_{\rm S} \plus {{1 \minus \alpha } \over 2}m_{\rm P} } \right)^{\ast } \cr \tab \quad \equals {1 \over 8}\lpar 1 \minus \alpha \rpar \left( {1 \plus {{\alpha ^{\setnum{2}} } \over {2 \minus \alpha }}} \right)s_{\setnum{2}}. \cr}$

Given the migration rate of seeds (or pollen), the critical migration rate of pollen (or seeds) can be calculated from eqn (16). Given a mating system and selection coefficients, the right-hand side of eqn (16) is constant. Numerical analysis shows that the migration rate of pollen required to satisfy eqn (16) reduces with an increase in migration rate of seeds, i.e. a negative but not complementary relationship between seed and pollen flow. Figure 4 b shows the estimates of the critical migration rate of pollen under a fixed migration rate of seeds and various selfing rates. Estimates from the exact simulation results are slightly higher than those predicted from eqn (16) although the similar changing patterns with the selfing rate exist between them.

When eqns (14) and (16) are jointly considered, the maximum value of the right-hand side of eqn (14) may be greater than the maximum value of the right-hand side of eqn (16) except for s ₂>>16s_C . This strict condition s ₂>>16s_C is applicable to the maternal alleles whose selection coefficients are much smaller than those of nuclear alleles. The more frequent situation is that the migration rate of seeds from eqn (14) that leads to the fixation of allele C can also lead to the fixation of nuclear allele A, without the need of pollen flow, which has been confirmed through exact simulations (data not given here).

Fig. 4. (a) Critical migration rate of seeds as a function of selfing rate for the maternal organelle allele. The dashed line gives the analytical approximation from eqn (14). The line with closed circles is the exact simulation result. In each case, parameter settings are the migration rate of pollen m _P=0·01, the selection coefficient s_C =0·02 and s ₂=0·08. (b) Critical migration rate of pollen as a function of selfing rate for the nuclear allele. The dashed curve with closed triangles gives the analytical approximation from eqn (16). The curve with closed circles is the exact simulation result. In each case, parameter settings are the migration rate of seeds m _S=0·001, the selection coefficient s_C =0·01 and s ₂=0·04.

When a coincidence between cytonuclear allele frequencies (p_A =p_C =p) occurs, the steady-state equation for p can be obtained by combining eqns (A1) and (A4) in the Appendix:

(17)

$\eqalign{ \tab \lpar 1 \plus f_{\setnum{1}} s_{\setnum{2}} \rpar m_{\rm S} \plus {{1 \minus \alpha } \over 4}m_{\rm P} \cr \tab \quad \equals {1 \over 2}p\left( {\lpar 2 \minus \alpha \rpar s_{C} \minus \lpar 1 \minus \alpha \rpar \left( {1 \plus {{\alpha ^{\setnum{2}} } \over {2 \minus \alpha }}} \right)\lpar 2p \minus 1\rpar s_{\setnum{2}} } \right)\comma \cr}$

where $f_{\setnum{1}} \hskip-1pt \hskip-1pt \equals \minus \left( {p\gamma \minus {\textstyle{{2\lpar 1 \minus \alpha \rpar } \over {2 \minus \alpha }}}p\lpar 2p \minus 1\rpar } \right){\textstyle{1 \over {1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S} }}}.$ Let $\lambda _{\setnum{1}} \equals<$> <$> {\textstyle{{\lpar 2 \minus \alpha \rpar ^{\setnum{2}} \gamma } \over {\lpar 1 \minus \alpha \rpar \lpar 2 \minus \alpha \plus \alpha ^{\setnum{2}} \rpar }}}$ . It can be shown that the maximum value of the right-hand side of eqn (17) is ${\textstyle{1 \over {16}}}\lpar 1 \minus \alpha \rpar \left( {1 \plus {\textstyle{{\alpha ^{\setnum{2}} } \over {2 \minus \alpha }}}} \right)\lpar 1 \plus \lambda _{\setnum{1}} \rpar ^{\setnum{2}} s_{\setnum{2}}$ at the point p=(1+λ₁)/4. Similarly, the minimum migration rate of seeds can be obtained by setting the maximum f _1max in the left-hand side of eqn (17), i.e. $f_{\setnum{1}\max } \equals <$> <$>{\textstyle{{1 \minus \alpha } \over {4\lpar 2 \minus \alpha \rpar }}}\left( {1 \minus {\textstyle{{\lpar 2 \minus \alpha \rpar \gamma } \over {2\lpar 1 \minus \alpha \rpar }}}} \right)^{\setnum{2}} {\textstyle{1 \over {1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S} }}}$ . The critical migration rate of seeds or pollen can be numerically calculated from the following equation:

(18)

$\eqalign{ \hskip-2pt\left( {\lpar 1 \plus f_{\setnum{1}\max } s_{\setnum{2}} \rpar m_{\rm S} \plus {{1 \minus \alpha } \over 4}m_{\rm P} } \right)^{\ast } \equals \tab {1 \over {16}}\lpar 1 \minus \alpha \rpar \left( {1 \plus {{\alpha ^{\setnum{2}} } \over {2 \minus \alpha }}} \right) \cr \tab \times \lpar 1 \plus \lambda _{\setnum{1}} \rpar ^{\setnum{2}} s_{\setnum{2}} . \hskip22pt$

With the change in the mating system and selection coefficients, a negative but not complementary relation exists between seed and pollen flow in contributing to the compounded critical migration rate. Figure 5 shows the change of the critical migration rate of pollen, predicted by the above equation, with the selfing rate. These estimates are unequal to those under the discordance case. Generally, a high critical migration rate of pollen is needed for species with a high selfing rate, given a fixed migration rate of seeds and selection strengths.

Fig. 5. Critical migration rate of pollen as a function of the selfing rate under the coincidence between the nuclear and maternal organelle allele frequencies. The curve with opened circles is the result estimated from eqn (16) in the scheme of heterozygote disadvantage, with the selection coefficients s ₂=0·04, s_C =0·01, and the migration rate m _S=0·001. The curve with closed circles is the result estimated from eqn (21) with directional selection, with the selection coefficients s ₁=s ₂=0·01, s_C =0·01, and the migration rate m _S=0·01.

With directional selection at the nuclear site, the steady-state equation for the change of maternal organelle allele frequency is derived as

(19)

$\left( {1 \minus {{2\lpar 1 \minus \alpha \rpar \lpar 1 \plus \rho \rpar \plus 4\alpha } \over {1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S} }}s_{\setnum{2}} p_{a} } \right)m_{\rm S} \equals \lpar 2 \minus \alpha \rpar s_{C} p_{C}.$

The critical migration rate for seeds, m _S*, can be numerically calculated from eqn (19) by letting p_a =1 and p_C =1, i.e.

$\left( {1 \minus {{2\lpar 1 \minus \alpha \rpar \lpar 1 \plus \rho \rpar \plus 4\alpha } \over {1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S}^{\ast } }}s_{\setnum{2}} } \right)m_{\rm S}^{\ast } \equals \lpar 2 \minus \alpha \rpar s_{C}.$

It can be shown that m _S* is greater with directional selection than in the scheme of heterozygote disadvantage, eqn (14). Exact simulation results confirm that the analytical approximations from eqn (19) perform very well (data not shown here). The critical migration rate of seeds generally reduces with the selfing rate.

Similarly, the critical migration rate for the nuclear gene can be calculated from eqn (A1) in the Appendix,

(20)

$\eqalign{ \tab \hskip-5pt\left( {\left( {1 \minus {{2s_{C} } \over {1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S} }}} \right)m_{\rm S} \plus {{1 \minus \alpha } \over 2}m_{\rm P} } \right)^{\ast } \equals {{2\lpar 1 \minus \alpha \rpar } \over {2 \minus \alpha }}s_{\setnum{1}} \cr \tab \quad \plus \left( {1 \plus \alpha \minus {{\alpha ^{\setnum{2}} \lpar 1 \minus \alpha \rpar } \over {2 \minus \alpha }}} \right)s_{\setnum{2}} . \hskip94pt$

A negative but not complementary relation exists between seed and pollen flow in contributing to the compounded critical migration rate, given a mixed mating system and selection coefficients. Figure 6 demonstrates that good analytical approximations from eqn (20) can be obtained in comparison with the exact simulation results under various selfing rates. The critical migration rate of pollen increases with the selfing rate, given a fixed migration rate of seeds.

Fig. 6. Critical migration rate of pollen as a function of selfing rate for the nuclear allele. The dashed curve with closed triangles gives the analytical approximation from eqn (20). The curve with closed circles is the exact simulation result. In each case, parameter settings are the migration rate of seeds m _S=0·01, the selection coefficient s_C =0·01 and s ₁=s ₂=0·01.

When a coincidence between nuclear and maternal organelle allele frequencies occurs, the critical migration rate for the compounded seed and pollen flow can be derived by combining eqns (A1) and (A4) in the Appendix,

(21)

$\eqalign{ \tab \left( {\left( {1 \minus f_{\setnum{2}} s_{\setnum{2}} } \right)m_{\rm S} \plus {{1 \minus \alpha } \over 4}m_{\rm P} } \right)^{\ast } \equals {{1 \minus \alpha } \over {2 \minus \alpha }}s_{\setnum{1}} \cr \tab \quad \plus {1 \over 2}\left( {1 \plus \alpha \minus {{\alpha ^{\setnum{2}} \lpar 1 \minus \alpha \rpar } \over {2 \minus \alpha }}} \right)s_{\setnum{1}} \plus {1 \over 2}\lpar 2 \minus \alpha \rpar s_{C} . \cr}$

where

$f_{\setnum{2}} \equals {{1 \plus \alpha \plus \gamma \plus \lpar 1 \minus \alpha \rpar \rho } \over {1 \minus \alpha \plus \lpar 1 \plus \alpha \rpar m_{\rm S} }}.$

Similarly, given the migration rate of pollen (or seeds), critical migration rate of seeds (or pollen) can be estimated from the above equation. The critical migration rate of pollen may be greater with direction selection than that in the scheme of heterozygote disadvantage (e.g. Fig. 5). This critical migration rate of pollen (or seeds) is also unequal to that under the discordance between cytonuclear allele frequencies.

(ii) Nuclear and paternally inherited organelle genome system

The theory in this subsubsection is applicable to the nuclear and chloroplast genome system in conifer tree species. The difference from the preceding subsection is that both seed and pollen flow may contribute to swamping selection for the paternal organelle genes.