Sampling Design Considerations: Where and When to Sample

The NPS recognizes the importance of collecting data in a scientifically credible manner so that they can be used to address current and future management issues. All parks and their contractors and cooperators should use certain "good sampling practices" so that data meet the purpose for which they were collected and withstand scrutiny by critics. Sample sizes will almost always be limited by shortages of funding and personnel, and it is critical to be able to make inferences to larger areas from data collected at relatively few sampling locations.

In February 2000, a panel of statisticians developed guidance for designing a sampling framework for monitoring natural resources in parks. Recommendations of this panel are presented in the following documents:

Download Guidance for the Design of Sampling Schemes for Inventory and Monitoring in National Parks (S. Fancy, March 2000)

Download Examples of Park Sampling Designs (S. Fancy, March 2000)

Download "Examples Illustrating the Design and Analysis of Monitoring Surveys in National Parks" (P. Geissler, May 2001)

Download Sample Designs for National Park Monitoring (P. Geissler and T. McDonald, April 2003)

Estimation of Change from Survey Data Results and Summary of a Workshop Held 11-12 March 2003 At The USGS Patuxent Wildlife Research Center, Laurel MD.

A good example of how a network of parks has designed a scientifically sound, efficient sampling scheme for a variety of terrestrial and aquatic resources is presented in the Sampling Design Chapter for the Heartland Network.

See also:

Summary of a Statistical Workshop at Olympic NP (A. Woodward and K. Jenkins, April 2001)

Statistical Methods for Adaptive Management Studies This is a good manual on statistical methods by the British Columbia Ministry of Forests Research Program.

Statistical Techniques for Sampling and Monitoring Natural Resources by Schreuder et al. 2004.

Statistical/Modeling Tools for Design and Analysis of Conservation Monitoring Data.

Sampling Design Recommendations

A summary of key elements of the recommendations for designing a sampling scheme are as follows:

1. Some sort of probability sample should always be taken to avoid bias. Conceptually, the target population (usually the entire park) is divided into sampling units such that every point in the park is included in a sampling unit, but not in more than one. The sampling design is used to select a probabilistic sample of the sampling units. As a result, statistical estimates of population attributes can be produced with an estimate of their reliability. Probability samples occur when each unit in the target population has a known, non-zero probability of being included in the sample, and always include a random component (such as a systematic sample with a random start). The credibility of data that are not collected using these principles is easily undermined.
   
2. Statistical, design based inferences can only be made to areas that have a chance of being included in the sample. If study plots are chosen to be close to roads, design-based inferences can only be made to areas near roads. Since the NPS's mission is to protect resources in the entire park, sampling should be designed so that robust inferences can be made to the entire park and not some easily accessible portion of it. Model based inferences and professional judgement can be used to infer values in portions of a park that had no probability of being included in the sample. However, accuracy of model based inferences and professional judgement is only as good as the model and the decision making process of the individual providing the professional judgement, and models and judgement-based information can often be easily discredited by critics. Areas of the park that are too inaccessible or unsafe to sample can be simply excluded from the program, but then no inference can be made about resources in these areas.
   
3. Judgement sampling, using "representative" sites selected by experts, should be avoided. If there is no controversy, judgement sampling sounds good at the beginning, but "representative" sites may come back to haunt you in the future because they are easily discredited by critics and may produce biased, unreliable information.
   
4. Panel members supported a general framework of first spreading samples out over the entire park or target population, and then increasing the sampling intensity in areas of special interest. Simple random sampling is not recommended because you may select a sample that is not spatially balanced, and because we are often interested in species or other park resources that occur in limited areas and we want to make sure we include adequate samples in those areas. Samples can be spread out over the area of interest by using some sort of grid or cell design or a tesselation procedure. Within this overall design, areas of special interest such as rare habitats can be sampled with higher frequencies using either stratification or the more general approach of defining the cells corresponding to the areas of interest and varying their selection probabilities (the unequal selection probability approach). In either case, the areas are then sampled disproportionate to their availability so that adequate samples are taken from each. This unequal sampling probability approach accomplishes most of the advantages of stratification, but avoids some of the problems of stratification that are mentioned below. An overall framework based on this design that allows for including site-specific studies and legacy data is presented below.
   
5. A design based on stratification of the park by "habitats" derived from vegetation maps is not recommended because stratum boundaries will change over time, and unless you fix the stratum boundaries forever there will be problems in the future with data analysis and incorporating new information into the design. A vegetation map is a model based on remote sensing data and data collected on the ground at a series of plots; the map boundaries will change as the classification models change or as additional ground-truthing data becomes available. Using these units to define strata will limit (and greatly complicate) long-term uses of the data by restricting future park managers' abilities to include new information into the sampling framework.
   
6. It is legitimate, and better, to delineate areas of special interest such as riparian or alpine areas based on physical characteristics such as terrain, and use these to judiciously define either strata or areas to sample with higher probability.
   
7. Permanent plots that are revisited over time are recommended for monitoring, because the objective is to detect changes over time. Revisiting the same plots removes plot to plot differences from the change estimates, increasing the precision.
   
8. An important step in developing a sampling design for a park is determining the sample size needed to significantly reduce the uncertainty of guessing about the status or trend of a resource and consequently reduce the costs of stewardship. Taking too few samples may increase the costs of stewardship and put resources at risk because important changes are missed or detected too late for management to be effective, whereas taking too many samples will waste time and resources. The sample size that is needed to meet a sampling objective is largely a function of the effect size, which is the amount of change in the resource from one point in time to the next that the manager seeks to detect, and the variability of the resource across space and time. For a statistician to be able to estimate the sample size needed for a particular program, the park manager needs to be able to specify how much change they need to be able to detect, and with what certainty, to affect their management strategies and practices or to confront and mitigate threats to the park in legal and political arenas. For planning purposes, sample size calculations in most statistical texts can be used to obtain a rough idea of the magnitude of the sample needed to produce a confidence interval of a specified width for a particular variable. If a statistical comparison is to be made between two samples, a "rule of thumb" minimum sample size is 6 measurements in each sample. It is useful to think about sampling over space when allocating samples.
   
9. Be sensitive to spatial integrity of the sample! These data will be used for many purposes, and an initial view of the sample on a map will help to clarify the use and limitations of the sample. When a sample is allocated, it is probably a good idea to display the sample on a GIS to ensure that adequate coverage occurs for areas of interest.
   
10. When repeated measurements of the same site are made to determine trend, remember that the precision will increase as the number of years of sampling increases. [The sample size of comparisons is usually the number of plots, which will not increase. However, the precision will increase because the means for each plot become less variable (var=s2/n.)]. There may be considerable intra-year variability in a measure because of small sample sizes, sampling errors and spatial variation, all of which increase needed sample sizes, and yet you may still be able to identify a trend as you increase the number of years of data.
    
11. When designing a monitoring program, remember that it is not necessary to visit all of the selected sites every year. Sampling designs exist that allow for increased spatial coverage though "rotating panel" designs, where each site is sampled every five years, for example, but five times as many sites can be sampled because only 1/5 of them are visited each year. Data from a complex rotating panel design with multiple strata can be difficult, so data analysis needs to be considered when the design is put together.
    
12. Collocation of samples is recommended to allow comparisons among components. For example, in the same stream segments you might sample water quality, aquatic macroinvertebrates, amphibians, and fish. Another example would be to monitor changes in vegetation, birds, mammals, and certain invertebrates at sites that are close to each other.