Estimating probabilistic dark diversity based on the hypergeometric distribution

O_LIThe biodiversity of a site includes the absent species from the region that are theoretically able to live in the sites particular ecological conditions. These species constitute the dark diversity of the site. Unlike present species, dark diversity is unobservable and can only be estimated. Most existing methods to designate dark diversity act in a binary fashion. However, dark diversity is more suitably defined as a fuzzy set--in which the degree of certainty about species membership is expressed as a probability.\nC_LIO_LIWe present a new method to estimate probabilistic dark diversity based on the hypergeometric distribution. The method relies on co-occurrences to infer the strength of the association between pairs of species and assign probabilistic adscription to dark diversity to absent species. We compare it with two established methods to estimate dark diversity (Beals index and favorability correction). To test the methods, we created simulations based on individual agents in which the suitability of each species in each site is known. We compared the ability of the methods to accurately predict suitability and the size of dark diversity, and compared their sensitivity to data availability. Further, we assessed the methods in two real datasets with nested sampling designs.\nC_LIO_LIOur simulations revealed that predictions of the Beals method were extremely sensitive to species frequency, and predicted suitability poorly. The Favorability transformation corrected this relationship, but did still predicted extremely low probabilities for species with very little information. The Hypergeometric method outperformed the Beals and Favorability methods in all considered aspects in the simulations and displayed better characteristics in the real datasets.\nC_LIO_LIProbabilistic consideratiosn of biodiversity will help to acknowledge the uncertainty associated with ecological information. Although the Beals method has been described as the best estimator of dark diversity, it should be preferred only when the goal is to predict future apperances of species. However, studies on dark diversity should focus on the ecological affinities of species. The Hypergeometric method is the most promising method to estimate probabilistic dark diversity and species pool composition based on co-occurrences.\nC_LI