Precision of Disability Estimates for Southeast Asians in the American Community Survey 2008-2010 Microdata

Carlos Siordia, Vi Donna Le
Preventive Medicine and Community Health, University of Texas Medical Branch

Abstract
Detailed social data about the United States (US) population were collected as part of the US decennial Census until 2000. Since then, the American Community Survey (ACS) has replaced the long form previously administered in decennial years. The ACS uses a sample rather than the entire US population, and therefore only estimates can be created from the data. This investigation computes disability estimates, standard error, margin of error, and a more comprehensive “range of uncertainty” measure for non-Latino-whites (NLW) and four Southeast Asian groups. Findings reveal that disability estimates for Southeast Asians have a much higher degree of imprecision than for NLW. Within Southeast Asian groups, Vietnamese have the highest level of certainty, followed by the Hmong. Cambodian and Laotian disability estimates contain high levels of uncertainty. Difficulties with self-care and vision contain the highest level of uncertainty relative to ambulatory, cognitive, independent living, and hearing difficulties.

Keywords: Southeast Asians; disability; ACS; population estimates; USA

Introduction

The American Community Survey (ACS), collected by the Census Bureau, is a primary source of population-level disability information in the United States (US). The ACS influences the distribution of millions of dollars, including funding that directly affects people with disabilities. For instance, in the 2008 fiscal year, 184 federal domestic assistance programs used ACS datasets to guide the distribution of $416 billion federal funding.¹ Data derived from the ACS are used to apportion money for handicapped facilities in mass transit systems and are used in state-level planning for future eligible Medicare and Medicaid recipients (federally funded healthcare insurance).^2,3 The six questions (see Appendix A) in the ACS reflect how disability is conceptualized in the International Classification of Functioning, Disability, and Health.⁴ The questions use a functional perspective to measure “difficulties” with six different areas of daily living so as to assess the disabled and nondisabled population in the US.⁴

Pre-constructed tabulations on the prevalence of disability amongst the different racial-ethnic groups in the US are available.⁵ From the cited source, about 6% of Asians and 13% of NLWs in the US are said to have reported having a disability (i.e., difficulty completing one or more of the six tasks). This data source appropriately warns non-technical readers that caution should be used when comparing population-level measures on specific groups when the statistics are based on small sample sizes and when the margin of error (MOE) is large relative to the given estimate. MOE is a measure of the accuracy in the estimate.

When using sample survey-based estimates to determine population characteristics, the representativeness of the sample—as is the level of precision in the estimate—is of great importance for producing sufficiently meaningful measurements. In this sense, data are said to be unbiased only to the degree that survey-based estimates do not systematically deviate from the ‘true’ population value. The formation of probability samples assumes that all units in the universe have a known, nonzero probability of being selected in the sample—the basic assumption validating the use of inference to generalize from the sample to the target population. In the case of disability estimates for specific populations, the term “estimate” signals that a sophisticated approximation is being made based on a sample and statistical inference.⁶ This means that the estimate contains some uncertainty.

Because accurate information is the primary goal of the US Census Bureau, they produce statistically rigorous data products. The agency provides extensive documentation of their procedures, thereby making imprecision (a measure of sampling error) measurable. Sampling error can be quantitatively estimated with ACS products because replicate weights are provided to public microdata users. Sampling error is the variation between samples drawn from target population where the same random procedure for selection is used. Large variation between samples signals high sampling error and thus that the survey-based estimate may significantly deviate from the true population value. Sampling error operates as a function of sample size, variability in the measure of interest, and the methodology employed in the production of the estimates. Sampling error can be measured using the standard error (SE) of the estimate—sometimes referred to as the MOE. SE measures represent how closely the survey-based estimate approximates the average result of all theoretically possible samples.

This brief report fundamentally argues that before comparisons on disability prevalence between racial-ethnic groups can be made, researchers must evaluate if the “range of uncertainty” (i.e., level of imprecision in the estimate) allows for such comparisons. Accordingly, this exploratory project compares the level of uncertainty in disability estimates between non-Latino-White (standard reference group in the US) and the following Southeast Asian groups: Vietnamese, Cambodian, Hmong, and Laotians. These Southeast Asian groups are selected because they provide a significantly smaller sub-population than the NLWs and may thus have larger variability between the systematically selected samples from the target population (i.e., larger sampling error and thus imprecision).

Methods

This analysis uses microdata from the ACS 2008-2010 Public Use Microdata Sample (PUMS) file.⁷ ACS PUMS files allow researchers the flexibility to prepare customized variables and tabulations where MOEs and SE around an estimate can also be computed using replicate weights in the microdata. A full discussion on the estimation of MOEs and SEs is given elsewhere.⁸ Confidence intervals on disability estimates, using the standard MOE approach, are provided to show the reader the upper and lower bounds of the estimate—where a 90% confidence level is assured in the measure. To provide a more standardized metric for comparing the precision of disability estimates across the five groups, the following equation is used to calculate the range of uncertainty (RU): [(SE*3) ÷ x]*100, where x is the estimate. As RU increases, the level of imprecision in the estimate tends to increase.

In addition to these measures, the percent of allocated cases is measured. The ACS PUMS files contain an allocation “flag”—a dichotomous variable of whether or not each response was observed or allocated. An allocation occurs when missing items or inconsistent data are replaced through assignment (fixed using within-person information) or allocation (fixed using outside-person information). For the sake of simplicity, both are referred to as allocations here. Greater details on allocation procedures are available elsewhere.⁹ The percent allocated is determined using the following equation: (weighted allocated count ÷ total weighted population)*100. This measure is given to remind the reader that nonsampling error is not accounted for in the RU measure. Nonsampling error is the practically unmeasurable error that may be created in each stage of the survey process—as when data collection is taking place. In this particular case, nonsampling error is created as a result of nonresponse (when no responses are given) and measurement (when illogical responses are given) errors.

MOEs, SEs, RUs, and percent allocated are calculated for the following disability items: self care, hearing, vision, independent living, ambulatory, and cognitive. For a detailed list of disability related questions, please see Appendix A. The variation of uncertainty in disability estimates in the Southeast Asian groups is examined since the detection of a “Central Asian” group is problematic with the data being used. For example, the dataset does not include any individuals residing in the US and who report being born in Kyrgyzstan, Tajikistan, or Turkmenistan—nor are there any ancestry codes for these places. This may due to the ‘true’ absence of such individuals or to editing protocols that recode their reported place of birth and ancestry into different labels. Please note that previous work has used the US Census Bureau’s definitions with ACS data to create a “South Central Asia” group that includes: Afghanistan; Bangladesh; Bhutan; India; Iran; Kazakhstan; Kyrgyzstan; Maldives; Nepal; Pakistan; Sri Lanka; Tajikistan; Turkmenistan; and Uzbekistan.¹¹

Since sample size matters in both weighed counts (using person-weights) and unweighted counts (using actual number of observations). We provide their details. In our analytic sample, NLWs have a weighted count of 148,734,362 and an unweighted count of 4,866,427; Vietnamese have a weighted count of 1,235,633 and an unweighted count of 34,141; Cambodians have a weighted count of 188,749 and an unweighted count of 4,714; Hmong have a weighted count of 129,282 and an unweighted count of 2,779; and Laotians have a weighted count of 164,432 and an unweighted count of 3,945. A rate of inflation (RI) is the average number of persons each individual represents and can be computed by dividing the weighted count by the unweighted number. We note that NLWs have an RI of 30, followed by Vietnamese (36), Cambodians (40), and Laotians (41). Hmong have the highest RI—where, on average, each Hmong survey respondent represent 47 other Hmong.

Results

Since our specific aim is to evaluate the level of precision in the various disability items across the different groups, we focus our discussion on comparing RUs across groups. NLWs are a standard reference group when investigating the US population, therefore we begin by highlighting that their RUs range from a low 1% to 2% (see Table 1). By comparison, the Vietnamese have RUs ranging from 11% to 14%. From Table 2, we note that Cambodians have RUs ranging from 22% to 44%, Hmong from 30% to 46%, and Laotians from 24% to 93%. As is clear from these numbers, relative to NLWs, all Southeast Asian groups have larger degrees of uncertainty in their disability estimates. Amongst the Southeast Asians, the Vietnamese have the least level of uncertainty in their disability estimates. Cambodians have the largest level of imprecision in the vision item and Hmong in the self care item. The self care and vision disability items amongst Laotians have a comparatively high level of imprecision.

Table 1: Weighted number of Non-Latino-White and Vietnamese with disabilities and corresponding estimate precision measures

	Disable	SE	MOE	LCL	UCL	RU	A
	Non-Latino-White
Ambulatory	14,439,768	26,505	43,600	14,396,168	14,483,368	1%	3%
Cognitive	8,361,648	20,346	33,468	8,328,180	8,395,116	1%	3%
Hearing	8,224,412	19,041	31,322	8,193,090	8,255,734	1%	3%
Independent	9,932,839	36,643	60,278	9,872,561	9,993,117	1%	3%
Self-care	5,585,330	19,082	31,389	5,553,941	5,616,719	1%	3%
Vision	4,162,395	22,080	36,321	4,126,074	4,198,716	2%	3%
	Vietnamese
Ambulatory	58,565	2,508	4,126	54,439	62,691	13%	5%
Cognitive	59,223	2,088	3,435	55,788	62,658	11%	5%
Hearing	31,431	1,509	2,483	28,948	33,914	14%	4%
Independent	58,259	2,248	3,698	54,561	61,957	12%	5%
Self-care	29,384	1,489	2,450	26,934	31,834	15%	5%
Vision	24,377	1,111	1,827	22,550	26,204	14%	4%

	Disable	SE	MOE	LCL	UCL	RU	A
	Cambodian
Ambulatory	11,795	1,747	2,873	8,922	14,668	44%	5%
Cognitive	14,459	1,070	1,761	12,698	16,220	22%	5%
Hearing	4,743	530	872	3,871	5,615	34%	3%
Independent	12,589	1,266	2,082	10,507	14,671	30%	4%
Self-care	4,721	746	1,226	3,495	5,947	47%	5%
Vision	5,691	1,103	1,815	3,876	7,506	58%	4%
	Hmong
Ambulatory	6,518	652	1,072	5,446	7,590	30%	6%
Cognitive	7,457	883	1,452	6,005	8,909	36%	6%
Hearing	4,267	425	700	3,567	4,967	30%	4%
Independent	7,147	808	1,329	5,818	8,476	34%	6%
Self-care	3,431	526	866	2,565	4,297	46%	6%
Vision	4,229	538	885	3,344	5,114	38%	4%
	Laotian
Ambulatory	7,814	631	1,037	6,777	8,851	24%	4%
Cognitive	8,479	1,248	2,053	6,426	10,532	44%	4%
Hearing	4,333	830	1,365	2,968	5,698	57%	3%
Independent	9,718	1,244	2,047	7,671	11,765	38%	4%
Self-care	4,038	1,044	1,718	2,320	5,756	78%	4%
Vision	4,214	1,307	2,150	2,065	6,364	93%	3%

Abbreviations: A, percent allocated = (total weighted population ÷ weighted allocated count)*100; Disable, number of people reporting a “difficulty” with the disability related item; LCL, low limit of 90% confidence interval = (Disability – MOE); MOE, margin of error; RU, range of uncertainty = [(SE*3) ÷ Disability]*100; SE, standard error; UCL, upper limit of 90% confidence interval= (Disability + MOE).

Although it was not the primary focus, we note that disability item allocations for NLWs are at 3%, while allocations ranged from 4% to 5% for Vietnamese, 3% to 5% for Cambodians, 4% to 6% for Hmong, and 3% to 4% for Loatians. These numbers indicate that in general, allocations were more prevalent for all disability items in Southeast Asian groups than NLWs.

Conclusions

We find that precision levels on the estimates of disability for four Southeast Asian groups are much lower than for NLWs. Since the range of uncertainty qualitatively differs by disability item and Asian group, comparisons of disability prevalence—where ACS data is used—should be avoided or done with great caution. Future work should explore if and how language, amongst Southeast Asian groups, may affect self-reporting of disabilities and how it may impact the rates of allocation with the survey since both introduce imprecision in the estimate.¹⁰ ACS survey-based disability estimates could potentially affect the quality of services Southeast Asian groups receive, therefore, more research should be undertaken to understand how precision can be improved.

This brief report adds to the literature by poignantly signaling the variability of imprecision on survey-based disability estimates. The level of precision matters and should be accounted for when comparing survey-based estimates between sub-populations. A high level of accuracy in disability estimates is important because they influence US federal funding aimed at aiding the already underserved disabled population.

Unfortunately, investigating small-size sub-populations limits the ability to produce large samples. Under such circumstances, the reduction of imprecision in population estimates may be attempted through improvements in sampling methodologies, survey design, and the administering of the questionnaire. Testing possible solutions is crucial, because most data sources currently used to develop disability estimates make use of sample rather than entire target populations. Because the precision of disability estimates may impact the quality and accuracy of governmental funding, and because the health of the individuals is so closely linked with access to healthcare, more research is needed.

References

1. Reamer AD. Surveying for Dollars: The Role of the American Community Survey in the geographic distribution of federal funds. Washington D.C.: Metropolitan Policy Program at Brookings; 2010.

2. MacDonald H. The American Community Survey: warmer (more current), but fuzzier (less precise) than the Decennial Census. Journal of the American Planning Association. 2006;72:491-503.

3. Swanson DA, McKibben JN. New Directions in the Development of Population Estimates in the United States? Population Research and Policy Review. 2010;29:797-818.

4. Brault M, Stern S, Raglin D. Evaluation Report Covering Disability. In: American Community Survey Content Test Report P.4. Washington D.C.: U.S. Census Bureau; 2007.

5. Erickson W, Lee C, von Schrader S. Disability Statistics from the 2010 American Community Survey (ACS). Ithaca, NY: Cornell University Rehabilitation Research and Training Center on Disability Demographics and Statistics (StatsRRTC); 2012.

6. US Census Bureau. A Compass for Understanding and Using American Community Survey Data: What Researchers Need to Know. Washington D.C.: US Government Printing Office; 2009.

7. US Census Bureau. A Compass for Understanding and Using American Community Survey Data: What PUMS Data Users Need to Know. Washington D.C.: US Government Printing Office; 2009.

8. US Census Bureau. Design and Methodology American Community Survey. Washington D.C.: US Government Printing Office; 2009.

9. Siordia C, Young, R. Methodological Note: Allocation of Disability Items in the American Community Survey. Disability and Health Journal. Forthcoming 2012.

10. Griffin D. Requests for Alternative Language Questionnaires. American Community Survey Discussion Paper. Washington D.C.: U.S. Government Printing Office; 2006.

11. Batalova J. Asian Immigrants in the United States. Washington D.C.: Migration Policy Institute; 2011.

Appendix A

Exact question wording for disability-related items in the American Community Survey

Self care difficulty

Because of a physical, mental, or emotional condition, does this person have difficulty doing errands alone such as visiting a doctor’s office or shopping?

Hearing difficulty

Is this person deaf or does he/she have serious difficulty hearing?

Vision difficulty

Is this person blind or does he/she have serious difficulty seeing even when wearing glasses?

Independent living difficulty

Does this person have difficulty dressing or bathing?

Ambulatory difficulty

Does this person have serious difficulty walking or climbing stairs?

Cognitive difficulty

Because of a physical, mental, or emotional condition, does this person have serious difficulty concentrating, remembering, or making decisions?