Text Size
Increase text size
Increase text size

Understanding Margins of Error

All data that are based on samples, such as the American Community Survey (ACS), include a range of uncertainty. Two broad types of error can occur: sampling error and non-sampling error. Non-sampling errors can result from mistakes in how the data are reported or coded, problems in the sampling frame or survey questionnaires, or problems related to non-response or interviewer bias. The Census Bureau tries to minimize non-sampling errors by using trained interviewers and by carefully reviewing the survey's sampline methods, data processing techniques, and questionnaire design.

 

Sampling error occurs when data are bassed on a sample of a population (approximately 1 in 40 households) rather than the full population. The uncertainty is basedon the fact that the:

 

  1. Estimates of characteristics from the sample data can differ from those that would be obtained if the entire population were surveyed.
  2. Estimates from one subset or sample of the population can differ from those based on a different sample from that same population.

 

Sampling error is related to sample size. Specifically, the larger the sample size, the smaller the uncertainty or sampling error for estimates based on the survey sample. This relationship between sample size and sampling error is the reason that ACS estimates for smaller geographic areas are based on multiple years of data. Combining ACS data from multiple years increases the sample size and in turn reduces the uncertainty or error associated with ACS esimates.

 

The Census Bureau publishes all their estimates with a 90-percent confidence interval, which define a range expected to contain the true value of an estimate with a level of confidence of 90 percent. In other words, we can be 90 percent certain that the range established by the margin of error contains the population value. If an esimate is reported as 2500 and the margin of error is 100 then we can say that with 90% certainty that the "true" value is between 2400 and 2600 (the estimate + and - the MOE). The range is also known as the confidence interval.

 

For example, the ACS Data Profile for Colorado, shown below, shows that an estimated 969,768 married-couple families resided in the state in 2009.

 

 

Margin of Error


By adding and subtracting the margin of error from the point estimate, we can calculate the 90-percent confidence interval for that estimate:


969,768 - 10,889 = 958,879 (Lower-bound interval)

969,768 + 10,889 = 980,657 (Upper-bound interval)


Therefore, we can be 90 percent confident that the true number of married-couple families in Colorado in 2009 falls somewhere between 958,879 and 980,657.

 

The MOE around an estimate is important because it helps you draw conclusions about the data. Small differences between two estimates maynot be statistically significant if the confidence intervals of those estimates overlap. However, the Census Bureau cautions data users not to rely on overlapping confidence intervals as a test for statistical significance, because this method will not always produce accurate results. Instead, the Census Bureau recommends conducting statistical significance tests.

 


How Big is Too Big?

 

Margins of Error are useful in assessing the reliability of estimates and whether the differences between estimates are significant. In general, the larger the margin of error relative to the size of the estimate, the less reliable is the estimate. There are no hard and fast rules about the usability of an estimate based on the size of the MOE; it all depends on the context and how the estimate will be used.

 

Example: Estimate of the Median Household Income for Alamosa County using ACS 5yr 2005-2009 data-

Estimate = $32,362

Margin of Error = $3,699

Relative size = MOE/Estimate = 11.4%

 

The margin of error relative to the estimate is 11.4%. In general, we are suggesting for users:

  • Always consider the context and how the estimate will be used,
  • Do not use estimates when the MOE is larger than 50% of the estimate, and
  • Use with caution when the MOE is 20% to 50% of the estimate.

 

These are general guidelines only and will depend on the question being asked and the imortance of exactness. In Colorado many of our communities are small will therefore, due to size, have larger margins of error. Sometimes uncertain information is better than no information at all, and other times it is worse. What is important is to report the margin of error when reporting the estimate so that other user/readers will understand the uncertainty related to the estimates. Visit our Resources page for a downloadable guide to Understanding MOE.