ACS and COVID-19: Guidance for Using the PUMS with Experimental Weights

Due to the effects of the COVID-19 pandemic on 2020 ACS data collection and data quality, the Census Bureau has made several adjustments to the data products that contain 2020 data. They did not release their standard data products for the 2020 ACS 1-year data, including the standard Public Use Microdata Sample (PUMS). Instead, they released the 2020 ACS 1-year data products with experimental entropy-balance weights designed to account for the impact of the COVID-19 pandemic on the data quality. For the 2020 ACS 5-year data, the Census Bureau revised its methodology for weighting households by using their standard weighting methodology for the 2016-2019 portion of the data and the entropy-balance weighting methodology for the 2020 portion of the data. The experimental weights were also made available for the 2019 ACS 1-year data. Users can apply both standard weights (HHWT and PERWT) and experimental weights (EXPWTH and EXPWTP) to analyses utilizing the 2019 ACS 1-year PUMS file.

IPUMS USA continues to harmonize the ACS single- and mutli-year PUMS files that contain 2020 data on the IPUMS USA website, however, the Census Bureau encourages users to proceed with caution when using samples that contain 2020 data.

This page provides a brief overview of the impacts of the COVID-19 pandemic on data collection and resulting data quality, the development of the experimental entropy-balance weights, and guidelines for using the ACS PUMS files that contain 2020 data. See the reference list included on this page for links to additional documentation providing greater detail about the 2020 ACS 1-year data products, the experimental entropy-balance weights, and other ACS PUMS samples that contain 2020 data. The information below applies to samples including the 2020 ACS sample (e.g. 2017-2021 ACS 5-year sample)

TABLE OF CONTENTS
Impact of the COVID-19 Pandemic on Data Collection and Data Quality
Development of the Experimental Weights
Guidelines for How and When to Use the ACS PUMS Files that Contain 2020 Data and the Experimental Weights
Reference List

Impact of the COVID-19 Pandemic on Data Collection and Data Quality

The COVID-19 pandemic significantly disrupted ACS data collection strategies, affecting the 2020 ACS 1-year PUMS file. Data collection typically occurs using two strategies: the mailout strategy and the computer-assisted personal interviewing (CAPI) strategy).

The mailout strategy includes five phases: an initial mailing, a reminder letter, a questionnaire package, a reminder postcard, and a final reminder. Each of these phases were affected at different times and to different degrees from March 2020 through the end of the year, resulting in large impacts on the ACS response rate. The CAPI operation strategy was also affected, such that field operations were suspended or reduced for a majority of the 2020 data collection year. Data collection for group quarters was especially disrupted by the COVID-19 pandemic; in-person visits to group quarter facilities were suspended or greatly reduced from March 2020 through the end of the year, and telephone interviews were not conducted due to logistical constraints. For a more detailed description and timeline of how these data collection strategies were impacted, see section two of the working paper: An assessment of the COVID-19 pandemic's impact on the 2020 ACS 1-year data. The 2020 ACS had significant variability in response rates for both housing units and group quarters across the 2020 data collection year, and had the lowest overall response rate in the history of the ACS.

Due to data collection disruptions and resulting data quality issues, the Census Bureau determined that the resulting sample did not meet their data quality standards, and they would need to take additional measures to adjust for the impacts of the pandemic on the 2020 data file. Data quality issues they found included:

  • Lower coverage rates for underrepresented populations (e.g., Black non-Hispanic, Hispanic respondents)
  • Decreased sample size and response rate between and within states
  • Increased rate of item allocation for all variables (most notably for household level variables)
  • An especially low response rate and survey completion rate for group quarters
  • Variables related to socioeconomic status, building structure, marital status, educational attainment, Medicaid coverage, citizenship, income, and poverty were particularly affected. These data quality issues resulted in biased estimates, especially for historically underrepresented populations.

    Development of the Experimental Weights

    To address the impacts of the COVID-19 pandemic on data collection and resulting data quality, the Census Bureau developed and applied additional weighting methodology to the 2020 ACS 1-year file; the resulting weights are referred to as the experimental weights in the Census Bureau and IPUMS documentation. Broadly, the changes to the weighting process addressed the following components:

  • Constitution of Sample Panel Groups
  • Over-Estimation of Vacant Housing Units
  • Under-Estimation of Occupied Housing Units
  • Increase in Non-Response Rates
  • Decrease in Group Quarters Response Rates
  • For a more detailed description of the adjustments made to the weighting process, please see section three of the working paper, An assessment of the COVID-19 pandemic's impact on the 2020 ACS 1-year data.

    Adjusting for the high level of non-response bias was one of the most important aspects of developing the experimental weights. To address the non-response bias, the Census Bureau drew from administrative, third-party, and decennial census data to develop entropy balance weights using methodology tested on the Annual Social and Economic Supplement (ASEC) of the Current Population Survey (CPS). Once developed, the entropy balance weights were first tested against standard benchmarks, such as the distribution of income and earnings, and were found to successfully reduce the non-response bias in the 2020 sample. The entropy balance weights were then applied to the 2019 ACS 1-year file for evaluation on a more standard ACS data sample. The large changes between 2019 and 2020 that were observed prior to applying these weights reduced to normal levels, suggesting that the entropy balance weights were reducing the abnormal effects of the pandemic on the 2020 ACS 1-year data file without changing the normal level of expected change. For more information about the development and assessment of the entropy balance weights, see the working paper, Addressing nonresponse bias in the American Community Survey during the pandemic using administrative data.

    The U.S. Census released experimental weights for the 2019 ACS 1-Year file to allow users to utilize both standard weights and experimental weights in their analyses. These weights are available for both the 2019 1-Year ACS and 2020 1-Year ACS under the variable names EXPWTH and EXPWTP.

    Overall, the experimental weights developed for the 2020 ACS 1-year data products address the known data quality issues, especially issues resulting from the high level of non-response during the 2020 data collection year. Users should still proceed with caution when using the 2020 ACS 1-year data products. The next section provides guidelines for using the 2020 ACS 1-year PUMS file and experimental weights.

    Guidelines for How and When to Use ACS PUMS Files that Contain 2020 Data and the Experimental Weights

    Considerations for using the 2020 ACS 1-year PUMS file:

  • The IPUMS-harmonized 2020 ACS 1-year PUMS file is formatted like the other ACS PUMS samples and can be accessed in the same way on the IPUMS USA website.
  • The 2020 1-year file will not be included among the default samples on the samples selection list.
  • The sample size in the 2020 ACS 1-year PUMS file is smaller than previous ACS 1-year PUMS files; not all states represent one percent of the 2020 population. See the American Community Survey 2020 1-year experimental PUMS file ReadMe for more information.
  • The Census Bureau advises against making comparisons between the 2020 ACS 1-year PUMS file and any other ACS or Census data products. While the Census Bureau addressed the known data quality issues present in the 2020 data file, the full extent of data quality issues is not yet known and may not be adequately accounted for in the experimental weights. See Comparing 2020 American Community Survey Data for more information about this guideline.
  • The Census Bureau suggests that users who typically use the ACS 1-year estimates to allocate funds, conduct program evaluations, make decisions, or for any other program purposes use an alternative data source (e.g., the 2019 ACS 1-year, the 2015-2019 ACS 5-year). See 2020 American Community Survey 1-Year Estimates: What You Need To Know for a flowchart that helps users determine whether or not their application of the data may be impacted by the changes to the 2020 ACS 1-year file.
  • Considerations for using other ACS PUMS files that contain 2020 data:

  • In the 2020 ACS 5-year PUMS file, the Census Bureau revised its methodology for weighting households by using their standard weighting methodology for the 2016-2019 portion of the data and the entropy-balance weighting methodology for the 2020 portion of the data. The Census Bureau will continue to adjust it's weighting methodology for all multiyear samples that contain 2020 data. See Modifications to the 5-year estimation methodology containing data collected in 2020 for more information.
  • The weighting adjustments in the 2020 5-year file resulted in larger coefficients of variation than usual for some key estimates, and the Census Bureau encourages users to proceed with caution when using variables that have large margins of error. See Increased margins of error in the 5-year estimates containing data collected in 2020 for more information.
  • The Census Bureau encourages users to proceed with caution when comparing data products containing 2020 data with other years.
  • The 2020 PRCS 5-year PUMS file was released and is available on the IPUMS USA website to download, despite the fact that the 2020 PRCS 1-year PUMS file was not released. The Census Bureau applied the weighting methodology developed for the 2020 ACS 1-year sample to the PRCS 5-year sample to adjust for data quality concerns. See Puerto Rico Community Survey multiyear accuracy of the data (5-year 2016-2020) for more information.

    Considerations for using the PUMS experimental entropy-balance weights:

  • IPUMS USA applied the standard weight variable names to the experimental weights in 2020, so that the weights use the same name as the standard weights available in other ACS IPUMS samples: HHWT, REPWT, PERWT, REPWTP.
  • New variables were created for the experimental weights (EXPWTH and EXPWTP) to accomodate both standard weights and experimental weights for the 2019 ACS following the release of the 2019 experimental weights in June of 2022.
  • The experimental weights should be used in the same way as the standard weights for other ACS PUMS samples. See Sample Designs and Sample Weights for more information about weights in the ACS PUMS samples.
  • Exercise caution when applying the weights to lower levels of geography. The weights were designed to produce estimates for state and larger counties, not PUMAs.
  • To calculate standard errors and/or margin of error, use the successive difference replication (SDR) formula. The Census Bureau is not publishing design factors for the 2020 1-year PUMS experimental weights file. See the American Community Survey 2020 1-year experimental PUMS file ReadMe for more information.
  • Variance estimates may be smaller than those for previous ACS PUMS samples.
  • Reference List

    2020 ACS 1-Year Experimental Data Release Overview
    2020 American Community Survey 1-year Experimental Data Release
    An Assessment of the COVID-19 pandemic's impact on the 2020 ACS 1-year data
    Collecting American Community Survey Data From Group Quarters Amid the Pandemic
    Addressing nonresponse bias in the American Community Survey during the pandemic using administrative data
    Comparing 2020 American Coummunity Survey Data
    2020 American Community Survey 1-Year Estimates: What You Need To Know
    American Community Survey 2020 1-year experimental PUMS file ReadMe
    Defining "Use With Caution": How we're navigating new census bureau data
    Puerto Rico Community Survey multiyear accuracy of the data (5-year 2016-2020)
    Modifications to the 5-year estimation methodology containing data collected in 2020
    Increased margins of error in the 5-year estimates containing data collected in 2020
    2019 1-Year PUMS with Experimental Weights