OBSERVER VARIATION IN VEGETATION CONDITION ASSESSMENTS:
IMPLICATIONS FOR BIODIVERSITY
Emma Gorrod, School of Botany, University of Melbourne
David Keith, NSW Department of Environment and Conservation
Vegetation condition assessments are tools that quantify the value of a patch of vegetation for biodiversity. They are based on the assumption that vegetation structural attributes act as surrogates for the habitat requirements of all indigenous plant and animal species. Examples of vegetation condition assessments include Habitat Hectares and BioMetric, which are used in Victoria and NSW respectively to determine offset requirements for the clearing of native vegetation. While the research has been conducted on both indices, this presentation concerns only the latter.
A vegetation condition assessment using the BioMetric method involves field assessment of ten vegetation structural and compositional attributes, such as % cover of the overstorey and total length of logs, within a quadrat. There are many reasons why different observers might record different values for these attributes in the field. Relevant literature suggests it’s likely that there will be variation amongst observers even if potential sources of variation are controlled for.
In order to characterise observer variation, I conducted a field trial in which ten observers independently conducted BioMetric assessments on 20 sites in Cumberland Plain Woodland (CPW). Sites spanned a range of structural and compositional variants. Observers had a range of training and experience, but all had some experience conducting vegetation surveys. The average coefficient of variation across all observers on all sites was 15%. The highest coefficient of variation was 28%, and the largest range of scores was 45 BioMetric points (scored out of 100). The smallest variation occurred on the two lowest ranking sites.
In order to understand the operation of the BioMetric tool given the nature of observer variation, a simulation was conducted in which 200 hypothetical observers assessed 20 hypothetical sites in CPW. The true values for each of the attributes measured in BioMetric were set to represent a range of structural and compositional variants of CPW. The field data was used to inform the distribution of observer estimates around the true value for each attribute, using a binomial distribution for the bounded variables and a scaled beta distribution for the unbounded variables. Initially, the simulation was unbiased, with the most likely observer estimate set to the true value. Boxplots of the total BioMetric score for the 200 hypothetical observers showed:
- For the lowest ranking sites with BioMetric scores up to about 30, the median value was close to, or on, the true value.
- For sites with true scores of 30 to 50, the true value sat within the second quartile of observations, indicating slight underestimation.
For the highest ranking sites with scores greater than 70, the true value consistently sat within the first quartile of observations. The true value of the highest ranking site was only correctly scored by the observers in the upper extreme of the first quartile.
The implication is that observers are likely to underestimate the true value, particularly of high quality sites.
A simulation in which all attributes were likely to be underestimated by 10% by observers (i.e. the most likely observer value was 10% lower than the true value) was then conducted. The resulting boxplots showed that the median of 200 hypothetical observers was close to the true value for lower ranking sites, and underestimation of high ranking sites was exacerbated relative to the unbiased simulation. In this case, all observers underestimated the value of the highest ranking site.
If the assumption that vegetation condition is a surrogate for the value of a site to biodiversity is right, then it is likely that the BioMetric tool results in underestimation of the value of high ranking sites for biodiversity. In practical terms, when we’re comparing management options and determining offset requirements, higher quality sites are likely to appear equivocal to lower quality sites, when in fact they are better.
In order to further investigate the implications of observer variation for biodiversity, future work will involve investigating the source of underestimation in the BioMetric tool and simulating offset scenarios.