Estimating effective population size from linkage disequilibrium between unlinked loci: theory and application to fruit fly outbreak populations

There is a substantial literature on the use of linkage disequilibrium (LD) to estimate effective population size using unlinked loci. The estimates are extremely sensitive to the sampling process, and there is currently no theory to cope with the possible biases. We derive formulae for the analysis of idealised populations mating at random with multi-allelic (microsatellite) loci. The ‘Burrows composite index’ is introduced in a novel way with a ‘composite haplotype table’. We show that in a sample of diploid size , the mean value of or from the composite haplotype table is biased by a factor of , rather than the usual factor for a conventional haplotype table. But analysis of population data using these formulae leads to estimates that are unrealistically low. We provide theory and simulation to show that this bias towards low estimates is due to null alleles, and introduce a randomised permutation correction to compensate for the bias. We also consider the effect of introducing a within-locus disequilibrium factor to , and find that this factor leads to a bias in the estimate. However this bias can be overcome using the same randomised permutation correction, to yield an altered with lower variance than the original , and one that is also insensitive to null alleles. The resulting formulae are used to provide estimates on 40 samples of the Queensland fruit fly, Bactrocera tryoni, from populations with widely divergent expectations. Linkage relationships are known for most of the microsatellite loci in this species. We find that there is little difference in the estimated values from using known unlinked loci as compared to using all loci, which is important for conservation studies where linkage relationships are unknown.