Caruso et al. (Reference Caruso, Luengos Vidal, Guerisoli and Lucherini2016) compared estimates of species occurrence based on camera traps and interviews, concluding that interview data are unreliable. This topic is important given the increasing use of interview data to supplement or substitute for more expensive or intensive surveys (Miller et al., Reference Miller, Nichols, Gude, Rich, Podruzny, Hines and Mitchell2013). Here we detail several methodological considerations in Caruso et al. that led to flawed inference.
Any occupancy analysis needs to define the sampling units, the time over which occupancy is assumed to be closed, and the replicate surveys representing the detection process (Bailey et al., Reference Bailey, MacKenzie and Nichols2014). With respect to sampling units, Caruso et al. paired five cameras with one interview. The camera grid was 2.25 km2 and mean property size for interviews was 6.84 km2; variance of property size was not reported and differences in sampling unit size were not accounted for in the occupancy models. This is an oversight given the positive relationship between occupancy probability and sampling unit size (MacKenzie et al., Reference MacKenzie, Nichols, Royle, Pollock, Bailey and Hines2006), invalidating Caruso et al.’s comparison of occupancy estimates by method. Each camera trap grid was paired with the interview in closest proximity, with distances ranging from < 5 km (n = 19 interviews) to 15–25 km (n = 7), yet Caruso et al. did not control for site-specific differences that could confound the comparison of occupancy estimates by method. In analyses where true occupancy was known, site-level covariates to explain heterogeneity were needed to reduce bias in occupancy estimates, particularly where site-specific differences were large (Miller et al., Reference Miller, Bailey, Grant, McClintock, Weir and Simons2015).
With respect to the time period over which occupancy is assumed to be closed, a critical assumption is that occupancy status at site level remains unchanged over the study period (MacKenzie et al., Reference MacKenzie, Nichols, Royle, Pollock, Bailey and Hines2006). Violation of this assumption can lead to spurious inference of model parameters (Bailey et al., Reference Bailey, MacKenzie and Nichols2014). Caruso et al. conducted interviews over 4 years and did not report the time difference between data collection by interview and camera trap. Such a comparison could lead to specious conclusions given the spatial and temporal stochasticity of natural systems.
With respect to the replicate surveys representing the detection process Caruso et al. used a single interview to represent a site. One sample cannot provide information on detection probability, and the lack of spatial replicates is a violation of statistical sampling. They also failed to incorporate false positives, an important consideration with interview data (Miller et al., Reference Miller, Nichols, Gude, Rich, Podruzny, Hines and Mitchell2013) that substantially reduces estimation bias (Miller et al., Reference Miller, Bailey, Grant, McClintock, Weir and Simons2015).
In the statistical analysis, Caruso et al. used a correlation of nominal ones and zeros to compare methods by assigning presence and absence to camera grids that did or did not capture 1+ photos of the target species, and did the same for single interviews that did or did not report species presence. This oversimplified approach fails to extricate absence from non-detection, as camera traps and people are imperfect detectors, and does not account for variation and uncertainty in occupancy estimates. Lastly, the most important figure of Caruso et al. (Fig. 2a), which provides occupancy estimates by method and species, does not provide standard error bars for interviews.
Caruso et al. addressed a difficult issue but their comparison would have benefited from a more robust sampling design incorporating site-specific differences and false positives, and by better elucidating the detection process. Failure to do so calls their conclusions into question.
