Skip to main content
    • Aa
    • Aa

Genetic causes and consequences of the breakdown of self-incompatibility: case studies in the Brassicaceae


The genetic consequences of inbreeding is a subject that has received thorough theoretical attention and has been of interest to empirical biologists since the time of Darwin. Particularly for species with genetically controlled mechanisms to promote outcrossing (self-incompatibility or SI systems), it is expected that high levels of genetic load should accumulate through sheltering of deleterious recessive mutations. Nevertheless, transitions to selfing are common across angiosperms, which suggests that the potentially negative consequences of reduced heterozygosity and genetic diversity are balanced by other factors, such as reproductive assurance. This mini-review focuses on empirical research in the Brassicaceae to emphasize some of the genetic consequences of shifts to inbreeding in terms of mechanisms for loss of SI, changes in genetic diversity following loss of SI, and inbreeding depression in relation to outcrossing history. Despite the long history of theoretical attention, there are still some surprisingly large gaps in our understanding in each of these areas. Rather than providing a complete overview, examples are drawn predominantly from published and emerging data from Arabidopsis thaliana and its relatives to highlight recent progress and remaining questions. We are currently on the brink of major breakthroughs in understanding due both to advances in sequencing technology and a shift in focus from crop plants to natural populations, where critical factors such as population structure, phylogeography, demographic history, partial compatibility and individual variation can be taken into account when investigating the nature of the selective forces regulating mating system evolution.

Corresponding author
Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

J. D. Austin , S. C. Lougheed , L. Neidrauer , A. A. Chek & P. T. Boag (2002). Cryptic lineages in a small frog: the post-glacial history of the spring peeper, Pseudacris crucifer (Anura: Hylidae). Molecular Phylogenetics and Evolution 25, 316329.

M. F. Bailey & D. E. McCauley (2006). The effects of inbreeding, outbreeding and long-distance gene flow on survivorship in North American populations of Silene vulgaris. Journal of Ecology 94, 98109.

D. Balaña-Alcaide , S. E. Ramos-Onsins , Q. Boone & M. Aguadé (2006). Highly structured nucleotide variation within and among Arabidopsis lyrata populations at the FAH1 and DFR gene regions. Molecular Ecology 15, 20592068.

S. C. H. Barrett & D. Charlesworth (1991). Effects of a change in the level of inbreeding on the genetic load. Nature 352, 522524.

A. J. Bateman (1955). Self-incompatibility in angiosperms. III. Cruciferae. Heredity 9, 5368.

J. Bechsgaard , T. Bataillon & M. H. Schierup (2004). Uneven segregation of sporophytic self-incompatibility alleles in Arabidopsis lyrata. Journal of Evolutionary Biology 17, 554561.

J. S. Bechsgaard , V. Castric , D. Charlesworth , X. Vekemans & M. H. Schierup (2006). The transition to self-compatibility in Arabidopsis thaliana and evolution within S-haplotypes over 10 myr. Molecular Biology and Evolution 23, 17411750.

A. Berr , A. Pecinka , A. Meister , G. Kreth , J. Fuchs , F. R. Blattner , (2006). Chromosome arrangement and nuclear architecture but not centromeric sequences are conserved between Arabidopsis thaliana and Arabidopsis lyrata. The Plant Journal 48, 771783.

S. Billiard , V. Castric & X. Vekemans (2007). A general model to explore complex dominance patterns in plant sporophytic self-incompatibility systems. Genetics 175, 13511369.

N. Brugière , Y. Cui , Y.-M. Bi , M. Arnoldo , L. Jackman & S. J. Rothstein (2000). Molecular genetics of self-incompatibility in Brassica napus. Annals of Botany 85, 133140.

J. W. Busch (2005 a). Inbreeding depression in self-incompatible and self-compatible populations of Leavenworthia alabamica. Heredity 94, 159165.

J. W. Busch (2005 b). The evolution of self-compatibility in geographically peripheral populations of Leavenworthia alabamica (Brassicaceae). American Journal of Botany 92, 15031512.

D. L. Byers & T. R. Meagher (1992). Mate availability in small populations of plant species with homomorphic sporophytic self-incompatibility. Heredity 68, 353359.

D. Cabrillac , J. M. Cock , C. Dumas & T. Gaude (2001). The S-locus receptor kinase is inhibited by thioredoxins and activated by pollen coat proteins. Nature 410, 220223.

D. E. Carr & M. R. Dudash (2003). Recent approaches into the genetic basis of inbreeding depression in plants. Philosophical Transactions of the Royal Society of London, Series B 358, 10711084.

V. Castric & X. Vekemans (2004). Plant self-incompatibility in natural populations: a critical assessment of recent theoretical and empirical advances. Molecular Ecology 13, 28732889.

B. Charlesworth & D. Charlesworth (1999). The genetic basis of inbreeding depression. Genetical Research 74, 329340.

D. Charlesworth (1988). Evolution of homomorphic sporophytic self-incompatibility. Heredity 60, 445453.

D. Charlesworth (2003). Effects of inbreeding on the genetic diversity of populations. Philosophical Transactions of the Royal Society of London, Series B 358, 10511070.

D. Charlesworth (2006 a). Balancing selection and its effects on sequences in nearby genome regions. PLoS Genetics 2, 379384.

D. Charlesworth (2006 b). Evolution of plant breeding systems. Current Biology 16, R726R735.

D. Charlesworth & P. Awadalla (1998). Flowering plant self-incompatibility: the molecular population genetics of Brassica S-loci. Heredity 81, 19.

D. Charlesworth & B. Charlesworth (1979 a). Evolution and breakdown of S-allele systems. Heredity 43, 4155.

D. Charlesworth & B. Charlesworth (1979 b). Evolutionary genetics of sexual systems in flowering plants. Proceedings of the Royal Society of London, Series B 205, 513530.

D. Charlesworth & B. Charlesworth (1987). Inbreeding depression and its evolutionary consequences. Annual Review of Ecology and Systematics 18, 237268.

D. Charlesworth & B. Charlesworth (1990). Inbreeding depression with heterozygote advantage and its effect on selection for modifiers changing the outcrossing rate. Evolution 44, 870888.

D. Charlesworth & X. Vekemans (2005). How and when did Arabidopsis thaliana become highly self-fertilising? Bioessays 27, 472476.

D. Charlesworth & S. I. Wright (2001). Breeding systems and genome evolution. Current Opinion in Genetics & Development 11, 685690.

D. Charlesworth & Z. Yang (1998). Allozyme diversity in Leavenworthia populations with different inbreeding levels. Heredity 81, 453461.

D. Charlesworth , M. T. Morgan & B. Charlesworth (1990). Inbreeding depression, genetic load, and the evolution of outcrossing rates in a multi-locus system with no linkage. Evolution 44, 14691489.

D. Charlesworth , E. E. Lyons & L. B. Litchfield (1994). Inbreeding depression in two highly inbreeding populations of Leavenworthia. Proceedings of the Royal Society of London, Series B 258, 209214.

D. Charlesworth , P. Awadalla , B. K. Mable & M. H. Schierup (2000). Population-level studies of multiallelic self-incompatibility loci, with particular reference to Brassicaceae. Annals of Botany 85, 227239.

D. Charlesworth , C. Bartolomé , M. H. Schierup & B. K. Mable (2003 a). Haplotype structure of the stigmatic self-incompatibility gene in natural populations of Arabidopsis lyrata. Molecular Biology and Evolution 20, 17411753.

D. Charlesworth , X. Vekemans , V. Castric & S. Glemin (2005). Plant self-incompatibility systems: a molecular evolutionary perspective. New Phytologist 168, 6169.

D. Charlesworth , E. Kamau , J. Hagenblad & C. L. Tang (2006). Trans-specificity at loci near the self-incompatibility loci in Arabidopsis. Genetics 172, 26992704.

T. Chookajorn , A. Kachroo , D. R. Ripoll , A. G. Clark & J. B. Nasrallah (2004). Specificity determinants and diversification of the Brassica self-incompatibility pollen ligand. Proceedings of the National Academy of Sciences of the USA 101, 911917.

M. J. Clauss , H. Cobban & T. Mitchell-Olds (2002). Cross-species microsatellite markers for elucidating population genetic structure of Arabidopsis and Arabis (Brassicaceae). Molecular Ecology 11, 591601.

C. R. Darwin (1876). The Effects of Cross and Self-fertilization in the Vegetable Kingdom. London: Murray.

S. Glémin , E. Bazin & D. Charlesworth (2006). Impact of mating systems on patterns of sequence polymorphism in flowering plants. Proceedings of the Royal Society of London, Series B 273, 30113019.

S. V. Good-Avila & A. G. Stephenson (2002). The inheritance of modifiers conferring self-fertility in the partially self-incompatible perennial, Campanula rapunculoides L. (Campanulaceae). Evolution 56, 263272.

C. Goodwillie , S. Kalisz & C. G. Eckert (2005). The evolutionary enigma of mixed mating systems in plants: occurrence, theoretical expectations, and empirical evidence. Annual Review of Ecology and Systematics 36, 4779.

D. R. Goring (2000). The search for components of the self-incompatibility signalling pathway(s) in Brassica napus. Annals of Botany 85, 171180.

D. R. Goring & S. J. Rothstein (1992). The S-locus receptor kinase gene in a self-incompatible Brassica napus line encodes a functional serine/threonine kinase. The Plant Cell 4, 12731281.

D. R. Goring & J. C. Walker (2004). Self-rejection: a new kinase connection. Science 303, 14741475.

J. Hagenblad , J. Bechsgaard & D. Charlesworth (2006). Linkage disequilibrium between incompatibility locus region genes in the plant Arabidopsis lyrata. Genetics 173, 10571073.

K. Hatakeyama , T. Takasaki , G. Suzuki , T. Nishio , K. Watanabe , A. Isogai , (2001). The S receptor kinase gene determines dominance relationships in stigma expression of self-incompatibility in Brassica. The Plant Journal 26, 6976.

P. W. Hedrick & S. T. Kalinowski (2000). Inbreeding depression in conservation biology. Annual Review of Ecology and Systematics 31, 139162.

B. C. Husband & D. W. Schemske (1996). Evolution of the magnitude and timing of inbreeding depression in plants. Evolution 50, 5470.

B. Igic , L. Bohs & J. R. Kohn (2006). Ancient polymorphism reveals unidirectional breeding system shifts. Proceedings of the National Academy of Sciences of the USA 103, 13591363.

P. Jarne & D. Charlesworth (1993). The evolution of the selfing rate in functionally hermaphrodite plants and animals. Annual Review of Ecology and Systematics 24, 441466.

E. Kamau & D. Charlesworth (2005). Balancing selection and low recombination affect diversity near the self-incompatibility loci of the plant Arabidopsis lyrata. Current Biology 15, 17731778.

K. Karkkainen , H. Kuittinen , R. van Treuren , C. Vogl , S. Oikarinen & O. Savolainen (1999). Genetic basis of inbreeding depression in Arabis petraea. Evolution 53, 13541365.

A. Kawabe , B. Hansson , J. Hagenblad , A. Forrest & D. Charlesworth (2006). Centromere locations and associated chromosome rearrangements in Arabidopsis lyrata and A. thaliana. Genetics 173, 16131619.

L. F. Keller & D. M. Waller (2002). Inbreeding effects in wild populations. Trends in Ecology and Evolution 17, 230241.

R. Kimura , K. Sato , R. Fujimoto & T. Nishio (2002). Recognition specificity of self-incompatibility maintained after the divergence of Brassica oleracea and Brassica rapa. The Plant Journal 29, 215223.

M. A. Koch & C. Kiefer (2006). Molecules and migration: biogeographical studies in cruciferous plants. Plant Systematics and Evolution 259, 121142.

M. A. Koch & M. Kiefer (2005). Genome evolution among cruciferous plants: a lecture from the comparison of the genetic maps of three diploid species – Capsella rubella, Arabidopsis lyrata subsp. petraea, and A. thaliana. American Journal of Botany 92, 761767.

H. Kuittinen , A. A. de Haan , C. Vogl , S. Oikarinen , J. Leppala , M. Koch , (2004). Comparing the linkage maps of the close relatives Arabidopsis lyrata and Arabidopsis thaliana. Genetics 168, 15751584.

M. Kusaba , K. Dwyer , J. Hendershot , J. Vrebalov , J. B. Nasrallah & M. E. Nasrallah (2001). Self-incompatibility in the genus Arabidopsis: characterization of the S locus in the outcrossing A. lyrata and its autogamous relative A. thaliana. Plant Cell 13, 627643.

M. Kusaba , C. W. Tung , M. E. Nasrallah & J. B. Nasrallah (2002). Monoallelic expression and dominance interactions in anthers of self-incompatible Arabidopsis lyrata. Plant Physiology 128, 1720.

R. Lande & D. W. Schemske (1985). The evolution of self-fertilization and inbreeding depression in plants. I. Genetic models. Evolution 39, 2440.

Å. Lankinen & W. S. Armbruster (2007). Pollen competition reduces inbreeding depression in Collinsia heterophylla (Plantaginaceae). Journal of Evolutionary Biology 20, 737749.

D. A. Levin (1996). The evolutionary significance of pseudo self-fertility. American Naturalist 148, 321332.

F. Liu , L. Zhang & D. Charlesworth (1998). Genetic diversity in Leavenworthia populations with different inbreeding levels. Proceedings of the Royal Society of London, Series B 265, 293301.

P. Liu , S. Sherman-Broyles , M. E. Nasrallah & J. B. Nasrallah (2007). A cryptic modifier causing transient self-incompatibility in Arabidopsis thaliana. Current Biology 17, 734740.

D. G. Lloyd (1979). Some reproductive factors affecting the selection of self-fertilization in plants. American Naturalist 113, 6779.

E. E. Lyons & J. Antonovics (1991). Breeding system evolution in Leavenworthia: breeding system variation and reproductive success in natural populations of Leavenworthia crassa (Cruciferae). American Journal of Botany 78, 270287.

B. K. Mable & A. Adam (2007). Patterns of genetic diversity in outcrossing and selfing populations of Arabidopsis lyrata. Molecular Ecology 16, 35653580.

B. K. Mable , M. H. Schierup & D. Charlesworth (2003). Estimating the number, frequency, and dominance of S-alleles in a natural population of Arabidopsis lyrata (Brassicaceae) with sporophytic control of self-incompatibility. Heredity 90, 422431.

B. K. Mable , A. V. Robertson , S. Dart , C. Di Berardo & L. Witham (2005). Breakdown of self-incompatibility in the perennial Arabidopsis lyrata (Brassicaceae) and its genetic consequences. Evolution 59, 14371448.

J. H. McDonald & M. Kreitman (1991). Adaptive protein evolution at the Adh locus in Drosophila. Nature 351, 652654.

T. Mitchell-Olds (2001). Arabidopsis thaliana and its wild relatives: a model system for ecology and evolution. Trends in Ecology and Evolution 16, 693699.

M. T. Morgan (2001). Consequences of life history for inbreeding depression and mating system evolution in plants. Proceedings of the Royal Society of London, Series B 268, 18171824.

K. Murase , H. Shiba , M. Iwano , F. S. Che , M. Watanabe , A. Isogai , (2004). A membrane-anchored protein kinase involved in Brassica self-incompatibility signaling. Science 303, 15161519.

J. B. Nasrallah (2000). Cell-cell signaling in the self-incompatibility response. Current Opinion in Plant Biology 3, 368373.

J. B. Nasrallah , P. Liu , S. Sherman-Broyles , R. Schmidt & M. E. Nasrallah (2007). Epigenetic mechanisms for breakdown of self-incompatibility in interspecific hybrids. Genetics 175, 19651973.

M. E. Nasrallah & D. H. Wallace (1967). Immunogenetics of self-incompatibility in Brassica oleracea L. Heredity 22, 519527.

M. E. Nasrallah , P. Liu & J. B. Nasrallah (2002). Generation of self-incompatible Arabidopsis thaliana by transfer of two S locus genes from A. lyrata. Science 297, 247249.

M. E. Nasrallah , P. Liu , S. Sherman-Broyles , N. A. Boggs & J. B. Nasrallah (2004). Natural variation in expression of self-incompatibility in Arabidopsis thaliana: implications for the evolution of selfing. Proceedings of the National Academy of Sciences of the USA 101, 1607016074.

M. O'Brien , C. Kapfer , G. Major , M. Laurin , C. Bertrand , K. Kondo , (2002). Molecular analysis of the stylar-expressed Solanum chacoense small asparagine-rich protein family related to the HT modifier of gametophytic self-incompatibility in Nicotiana. The Plant Journal 32, 985996.

M. Paetsch , S. Mayland-Quellhorst & B. Neuffer (2006). Evolution of the self-incompatibility system in the Brassicaceae: identification of S-locus receptor kinase (SRK) in self-incompatible Capsella grandiflora. Heredity 97, 283290.

F. X. Picó , N. J. Ouborg & J. M. van Groenendael (2003). Fitness traits and dispersal ability in the herb Tragopogon pratensis (Asteraceae): decoupling the role of inbreeding depression and maternal effects. Plant Biology 5, 522530.

N. Prigoda , A. Nassuth & B. K. Mable (2005). Phenotypic and genotypic expression of self-incompatibility haplotypes in Arabidopsis lyrata suggests unique origin of alleles in different dominance classes. Molecular Biology and Evolution 22, 16091620.

O. Savolainen , C. H. Langley , B. P. Lazzaro & H. Fréville (2000). Contrasting patterns of nucleotide polymorphism at the alcohol dehydrogenase locus in the outcrossing Arabidopsis lyrata and the selfing Arabidopsis thaliana. Molecular Biology and Evolution 17, 645655.

D. W. Schemske & R. Lande (1985). The evolution of self-fertilization and inbreeding depression in plants. II. Empirical observations. Evolution 39, 4152.

M. H. Schierup , X. Vekemans & D. Charlesworth (2000). The effect of subdivision on variation at multi-allelic loci under balancing selection. Genetical Research 76, 5162.

M. H. Schierup , J. Bechsgaard , L. H. Nielsen & F. B. Christiansen (2006). Selection at work in self-incompatible Arabidopsis lyrata: mating patterns in a natural population. Genetics 172, 477484.

S. Sherman-Broyles , N. Boggs , A. Farkas , P. Liu , J. Vrebalov , M. E. Nasrallah , (2007). S locus genes and the evolution of self-fertility in Arabidopsis thaliana. The Plant Cell 19, 94106.

H. Shiba , M. Iwano , T. Entani , K. Ishimoto , H. Shimosato , F.-S. Che , Y. Satta , A. Ito , Y. Takada , M. Watanabe , A. Isogai & S. Takayama (2002). The dominance of alleles controlling self-incompatibility in Brassica pollen is regulated at the RNA level. The Plant Cell 14, 491504.

H. Shiba , M. Kenmochi , M. Sugihara , M. Iwano , S. Kawasaki , G. Suzuki , (2003). Genomic organization of the S-locus region of Brassica. Biosciences, Biotechnology and Biochemistry 67, 622626.

K. K. Shimizu & M. Purugganan (2005). Evolutionary and ecological genomics of Arabidopsis. Plant Physiology 138, 578584.

K. K. Shimizu , J. M. Cork , A. L. Caicedo , C. A. Mays , R. C. Moore , K. M. Olsen , (2004). Darwinian selection on a selfing locus. Science 306, 20812084.

S. L. Stone , M. Arnoldo & D. R. Goring (1999). A breakdown of Brassica self-incompatibility in ARC1 antisense transgenic plants. Science 286, 17291731.

S. Takayama & A. Isogai (2003). Molecular mechanism of self-recognition in Brassica self-incompatibility. Journal of Experimental Botany 54, 149156.

K. F. Thompson & J. P. Taylor (1966). Non-linear dominance relationships between S alleles. Heredity 21, 345362.

M. K. Uyenoyama & D. M. Waller (1991). Coevolution of self-fertilization and inbreeding depression. III. Homozygous lethal mutations at multiple loci. Theoretical Population Biology 40, 173210.

X. Vekemans , M. H. Schierup & F. B. Christianssen (1998). Mate availability and fecundity selection in multiallelic self-incompatibility systems in plants. Evolution 52, 1929.

J. L. Wang , W. G. Hill , D. Charlesworth & B. Charlesworth (1999). Dynamics of inbreeding depression due to deleterious mutations in small populations: mutation parameters and inbreeding rate. Genetical Research 74, 165178.

S. G. Weller & A. K. Sakai (1999). Using phylogenetic approaches for the analysis of plant breeding system evolution. Annual Review of Ecology and Systematics 30, 167199.

S. I. Wright , B. Lauga & D. Charlesworth (2002). Rates and patterns of molecular evolution in inbred and outbred Arabidopsis. Molecular Biology and Evolution 19, 14071420.

S. I. Wright , B. Lauga & D. Charlesworth (2003). Subdivision and haplotype structure in natural populations of Arabidopsis lyrata. Molecular Ecology 12, 12471263.

S. I. Wright , C. B. K. Yau , M. Looseley & B. C. Meyers (2004). Effects of gene expression on molecular evolution in Arabidopsis thaliana and Arabidopsis lyrata. Molecular Biology and Evolution 21, 17191726.

S. I. Wright , J. P. Foxe , L. DeRose-Wilson , A. Kawabe , M. Looseley , B. S. Gaut , (2006). Testing for effects of recombination rate on nucleotide diversity in natural populations of Arabidopsis lyrata. Genetics 174, 14211430.

S. I. Wright , G. Iorgovan , S. Misra & M. Mokhtari (2007). Neutral evolution of synonymous base composition in the Brassicaceae. Journal of Molecular Evolution 64, 136141.

K. Yogeeswaran , A. Frary , T. L. York , A. Amenta , A. H. Lesser , J. B. Nasrallah , (2005). Comparative genome analyses of Arabidopsis spp.: inferring chromosomal rearrangement events in the evolutionary history of A. thaliana. Genome Research 15, 505515.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Genetics Research
  • ISSN: 0016-6723
  • EISSN: 1469-5073
  • URL: /core/journals/genetics-research
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 1
Total number of PDF views: 6 *
Loading metrics...

Abstract views

Total abstract views: 47 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 24th March 2017. This data will be updated every 24 hours.