Mathematical & Computational Biology

Population genetics: coalescence rate and demographic parameters inference

10.24072/pcjournal.285 - Peer Community Journal, Volume 3 (2023), article no. e53.

Get full text PDF Peer reviewed and recommended by PCI

Inferring the demographic history of species is a great challenge in population genetics. This history is classically represented as a history of size changes, ignoring population structure. We present here the work carried out over the last decade around the concept of IICR (Inverse Instantaneous Coalescence Rate), which makes it possible to link, on the one hand, the history of the true population size for a panmictic population, and the inferred size, sometimes called "effective size", when structure is taken into account. We show that population structure can lead to misinterpretations of some demographic history inference results, we propose a framework for inferring structure-specific demographic parameters (number and size of subpopulations, migration rates), and we analyze the link between IICR and some form of selection modeling on genetic sequences.

Published online:
DOI: 10.24072/pcjournal.285
Keywords: population genetics; demographic history inference; coalescent theory; coalescence rate; population structure; IICR
Mazet, Olivier 1; Noûs, Camille 2

1 Université de Toulouse, Institut National des Sciences Appliquées de Toulouse, Institut de Mathématiques de Toulouse, Toulouse, France
2 Laboratoire Cogitamus, France
License: CC-BY 4.0
Copyrights: The authors retain unrestricted copyrights and publishing rights
     author = {Mazet, Olivier and No\^us, Camille},
     title = {Population genetics: coalescence rate and demographic parameters inference},
     journal = {Peer Community Journal},
     eid = {e53},
     publisher = {Peer Community In},
     volume = {3},
     year = {2023},
     doi = {10.24072/pcjournal.285},
     language = {en},
     url = {}
AU  - Mazet, Olivier
AU  - Noûs, Camille
TI  - Population genetics: coalescence rate and demographic parameters inference
JO  - Peer Community Journal
PY  - 2023
VL  - 3
PB  - Peer Community In
UR  -
DO  - 10.24072/pcjournal.285
LA  - en
ID  - 10_24072_pcjournal_285
ER  - 
%0 Journal Article
%A Mazet, Olivier
%A Noûs, Camille
%T Population genetics: coalescence rate and demographic parameters inference
%J Peer Community Journal
%D 2023
%V 3
%I Peer Community In
%R 10.24072/pcjournal.285
%G en
%F 10_24072_pcjournal_285
Mazet, Olivier; Noûs, Camille. Population genetics: coalescence rate and demographic parameters inference. Peer Community Journal, Volume 3 (2023), article  no. e53. doi : 10.24072/pcjournal.285.

Peer reviewed and recommended by PCI : 10.24072/pci.mcb.100150

Conflict of interest of the recommender and peer reviewers:
The recommender in charge of the evaluation of the article and the reviewers declared that they have no conflict of interest (as defined in the code of conduct of PCI) with the authors or with the content of the article.

[1] Arredondo, A.; Mourato, B.; Nguyen, K.; Boitard, S.; Rodríguez, W.; Mazet, O.; Chikhi, L. Inferring number of populations and changes in connectivity under the n-island model, Heredity, Volume 126 (2021) no. 6, pp. 896-912 | DOI

[2] Arredondo, S. A. Inférence d’histoires démographiques de populations structurées et application à l’évolution humaine, PhD thesis, Toulouse, INSA (2021)

[3] Boitard, S.; Arredondo, A.; Chikhi, L.; Mazet, O. Heterogeneity in effective size across the genome: effects on the inverse instantaneous coalescence rate (IICR) and implications for demographic inference under linked selection, Genetics, Volume 220 (2022) no. 3 | DOI

[4] Bonhomme, M.; Chevalet, C.; Servin, B.; Boitard, S.; Abdallah, J.; Blott, S.; SanCristobal, M. Detecting Selection in Population Trees: The Lewontin and Krakauer Test Extended, Genetics, Volume 186 (2010) no. 1, pp. 241-262 | DOI

[5] Chakraborty, R. Analysis of Genetic Structure of Populations: Meaning, Methods, and Implications, Human Population Genetics, Springer US, Boston, MA, 1993, pp. 189-206 | DOI

[6] Charlesworth, B. Effective population size and patterns of molecular evolution and variation, Nature Reviews Genetics, Volume 10 (2009) no. 3, pp. 195-205 | DOI

[7] Chikhi, L.; Rodríguez, W.; Grusea, S.; Santos, P.; Boitard, S.; Mazet, O. The IICR (inverse instantaneous coalescence rate) as a summary of genomic diversity: insights into demographic inference and model choice, Heredity, Volume 120 (2017) no. 1, pp. 13-24 | DOI

[8] Chikhi, L.; Bruford, M. W.; Beaumont, M. A. Estimation of Admixture Proportions: A Likelihood-Based Approach Using Markov Chain Monte Carlo, Genetics, Volume 158 (2001) no. 3, pp. 1347-1362 | DOI

[9] Chikhi, L.; Sousa, V. C.; Luisi, P.; Goossens, B.; Beaumont, M. A. The Confounding Effects of Population Structure, Genetic Diversity and the Sampling Scheme on the Detection and Quantification of Population Size Changes, Genetics, Volume 186 (2010) no. 3, pp. 983-995 | DOI

[10] Ewens, W. On the concept of the effective population size, Theoretical Population Biology, Volume 21 (1982) no. 3, pp. 373-378 | DOI

[11] Ewing, G. B.; Jensen, J. D. The consequences of not accounting for background selection in demographic inference, Molecular Ecology, Volume 25 (2015) no. 1, pp. 135-141 | DOI

[12] Excoffier, L. Analysis of Population Subdivision, Handbook of Statistical Genetics, John Wiley & Sons, Ltd, Chichester, 2004 | DOI

[13] Gossmann, T. I.; Woolfit, M.; Eyre-Walker, A. Quantifying the Variation in the Effective Population Size Within a Genome, Genetics, Volume 189 (2011) no. 4, pp. 1389-1402 | DOI

[14] Griffiths, R. C.; Marjoram, P. An ancestral recombination graph In: In Progress in population genetics and human evolution, Springer (1997), pp. 257-270

[15] Griffiths, R.; Tavaré, S. The age of a mutation in a general coalescent tree, Communications in Statistics. Stochastic Models, Volume 14 (1998) no. 1-2, pp. 273-295 | DOI

[16] Grusea, S.; Rodríguez, W.; Pinchon, D.; Chikhi, L.; Boitard, S.; Mazet, O. Coalescence times for three genes provide sufficient information to distinguish population structure from population size changes, Journal of Mathematical Biology, Volume 78 (2018) no. 1-2, pp. 189-224 | DOI

[17] Harris, N. S.; Rogers, A. R. Genomic regions linked to soft sweeps approximate neutrality when inferring population history from site pattern frequencies, bioRxiv, 2020 | DOI

[18] Hein, J.; Schierup, M.; Wiuf, C. Gene genealogies, variation and evolution: a primer in coalescent theory, Oxford University Press, 2004

[19] Heller, R.; Chikhi, L.; Siegismund, H. R. The Confounding Effect of Population Structure on Bayesian Skyline Plot Inferences of Demographic History, PLoS ONE, Volume 8 (2013) no. 5 | DOI

[20] Herbots, H. Stochastic models in population genetics: genealogy and genetic differentiation in structured populations, PhD thesis (1994)

[21] Hill, W. G.; Robertson, A. The effect of linkage on limits to artificial selection, Genetical Research, Volume 8 (1966) no. 3, pp. 269-294 | DOI

[22] Hobolth, A.; Siri-Jégousse, A.; Bladt, M. Phase-type distributions in population genetics, Theoretical Population Biology, Volume 127 (2019), pp. 16-32 | DOI

[23] Jensen, J. D.; Kim, Y.; DuMont, V. B.; Aquadro, C. F.; Bustamante, C. D. Distinguishing Between Selective Sweeps and Demography Using DNA Polymorphism Data, Genetics, Volume 170 (2005) no. 3, pp. 1401-1410 | DOI

[24] Jiménez-Mena, B.; Hospital, F.; Bataillon, T. Heterogeneity in effective population size and its implications in conservation genetics and animal breeding, Conservation Genetics Resources, Volume 8 (2015) no. 1, pp. 35-41 | DOI

[25] Johri, P.; Riall, K.; Becher, H.; Excoffier, L.; Charlesworth, B.; Jensen, J. D. The Impact of Purifying and Background Selection on the Inference of Population History: Problems and Prospects, Molecular Biology and Evolution, Volume 38 (2021) no. 7, pp. 2986-3003 | DOI

[26] Kingman, J. The coalescent, Stochastic Processes and their Applications, Volume 13 (1982) no. 3, pp. 235-248 | DOI

[27] Li, H.; Durbin, R. Inference of human population history from individual whole-genome sequences, Nature, Volume 475 (2011) no. 7357, pp. 493-496 | DOI

[28] Mazet, O.; Rodríguez, W.; Chikhi, L. Demographic inference using genetic data from a single individual: Separating population size variation from population structure, Theoretical Population Biology, Volume 104 (2015), pp. 46-58 | DOI

[29] Mazet, O.; Rodríguez, W.; Grusea, S.; Boitard, S.; Chikhi, L. On the importance of being structured: instantaneous coalescence rates and human evolution—lessons for ancestral population size inference?, Heredity, Volume 116 (2015) no. 4, pp. 362-371 | DOI

[30] McVean, G. A.; Cardin, N. J. Approximating the coalescent with recombination, Philosophical Transactions of the Royal Society B: Biological Sciences, Volume 360 (2005) no. 1459, pp. 1387-1393 | DOI

[31] Nei, M.; Takahata, N. Effective population size, genetic diversity, and coalescence time in subdivided populations, Journal of Molecular Evolution, Volume 37 (1993) no. 3 | DOI

[32] Peter, B. M.; Wegmann, D.; Excoffier, L. Distinguishing between population bottleneck and population subdivision by a Bayesian model choice procedure, Molecular Ecology, Volume 19 (2010) no. 21, pp. 4648-4660 | DOI

[33] Rodríguez, W.; Mazet, O.; Grusea, S.; Arredondo, A.; Corujo, J. M.; Boitard, S.; Chikhi, L. The IICR and the non-stationary structured coalescent: towards demographic inference with arbitrary changes in population structure, Heredity, Volume 121 (2018) no. 6, pp. 663-678 | DOI

[34] Rogers, A. Estimates of Effective Population Size in Subdivided Populations, Peer Community in Mathematical and Computational Biology (2023) | DOI

[35] Schiffels, S.; Durbin, R. Inferring human population size and separation history from multiple genome sequences, Nature Genetics, Volume 46 (2014) no. 8, pp. 919-925 | DOI

[36] Schrider, D. R.; Shanku, A. G.; Kern, A. D. Effects of Linked Selective Sweeps on Demographic Inference and Model Selection, Genetics, Volume 204 (2016) no. 3, pp. 1207-1223 | DOI

[37] Tajima, F. Evolutionary relationship of DNA sequences in finite populations, Genetics, Volume 105 (1983) no. 2, pp. 437-460 | DOI

[38] Tavaré, S. Part I: Ancestral Inference in Population Genetics, Lectures on Probability Theory and Statistics (Lecture Notes in Mathematics), Volume 1837, Springer Berlin Heidelberg, Berlin, Heidelberg, 2004, pp. 1-188 | DOI

[39] Teixeira, H.; Salmona, J.; Arredondo, A.; Mourato, B.; Manzi, S.; Rakotondravony, R.; Mazet, O.; Chikhi, L.; Metzger, J.; Radespiel, U. Impact of model assumptions on demographic inferences: the case study of two sympatric mouse lemurs in northwestern Madagascar, BMC Ecology and Evolution, Volume 21 (2021) no. 1 | DOI

[40] Wakeley, J. The Coalescent in an Island Model of Population Subdivision with Variation among Demes, Theoretical Population Biology, Volume 59 (2001) no. 2, pp. 133-144 | DOI

[41] Walsh, B.; Lynch, M. Evolution and Selection of Quantitative Traits, Oxford Scholarship Online, 1, Oxford University Press, 2018 | DOI

[42] Waples, R. S. What Is Ne, Anyway?, Journal of Heredity, Volume 113 (2022) no. 4, pp. 371-379 | DOI

[43] Watterson, G. On the number of segregating sites in genetical models without recombination, Theoretical Population Biology, Volume 7 (1975) no. 2, pp. 256-276 | DOI

[44] Whitlock, M. C.; Barton, N. H. The Effective Size of a Subdivided Population, Genetics, Volume 146 (1997) no. 1, pp. 427-441 | DOI

[45] Yang, M. A.; Malaspinas, A.-S.; Durand, E. Y.; Slatkin, M. Ancient Structure in Africa Unlikely to Explain Neanderthal and Non-African Genetic Similarity, Molecular Biology and Evolution, Volume 29 (2012) no. 10, pp. 2987-2995 | DOI

Cited by Sources: