Tag Archives: Census API

Data Analytics Labs: Resources & Methods to Create & Apply Insight

.. it has always been that an important first step in Data Analytics was developing or acquiring the key data to be analyzed.  Having the right subject matter data for the right geography and time frame are essential prerequisites.  The Internet and pervasive expanding needs and solutions for data integration has made Data Analytics easier and more accessible — and often more technically challenging.  The Data Analytics Lab, reviewed here, offers a foundational and support framework to address the continually evolving Data Analytics needs. See more about the Data Analytics Lab in the related Web section.

Data Analytics involves more than the ability to perform statistical analyses on data, though this is an important element. Data Analytics encompasses a spectrum of data and the ability to develop, access, integrate and effectively use those data. The platform of data on which we operate is constantly in change. New and different types of data emerge. Older data become obsolete. Required subject matter are often not available at the required geographic granularity. Often the biggest challenge is in linking data for analysis.

Role of Geographic Information Systems (GIS)
GIS software is one on many Data Analytics tools. It is an important tool for many reasons. One main reason is the enabling ability to flexibly visually examine subject matter for different types of geography simultaneously. The following view shows patterns of economic prosperity in the Atlanta region (metro shown with bold boundary). The thematic pattern is median household income by block group. Higher income areas are shown in blue/green and lower income areas are shown in orange/red. Counties are shown with black boundaries.

The next view is a zoom-in to the pointer location in the map shown above. This view has a higher transparency setting enabling a see-though effect to view highways and related ground infrastructure. Block groups appear with black boundaries. The pointer is at the county boundary, a slightly bolder boundary.

Data Analytics Labs (DAL) help participants develop a capacity to create map views like those shown above and perform geospatial analyses. This is more than learning how to operate GIS software; more than how to acquire shapefiles and build a GIS project. It is about selecting and acquiring the right subject matter data and then integrating those data with geometry (shapefiles, etc.) and assembling the composite files for analysis. DAL learning enables participants to develop informative and relevant maps view — and the right types of different geographies (including vintages) to use.

Data Analytics Labs are set up within universities, government agencies and not-for-profit organizations to address these needs. Contact us to discuss how a Lab can be developed within your organization.

A Data Analytics Lab provides tools/resources for hands-on data analytics applications. In a university setting, the Lab can be open to MBA, MPA, MHA and other graduate students plus multi-disciplinary faculty/researchers. Also in the university setting, some parts of a Lab can fit into existing classes and mesh with other existing programs. There is no physical lab; it is a virtual lab. The resources and vision are initially focused on Census-type geographic-demographic-economic data, expanded to selected Federal statistical data, and how these and primary data are knitted together for analysis and decision-making.

There are no fees to participants. Tools and data are made available by ProximityOne — http://proximityone.com.

Data Analytics Lab Resources & Components
1. Software
    • CV XE GIS software
      – GIS shapefiles and applications compatible with ESRI ArcGIS software
    • API data access/integration software
    • Modeler software
      – cause & effect modeling; estimation & forecasting; impact analysis
    • Related software
2. Data Resources
    • state & national scope ready to use GIS projects
    • augmented TIGER/DMD geography
    • U.S. national scope 5-year demographic-economic estimates & projections
      –  small area demographic-economic estimates/projections to 2020
    • U.S. national scope county/up 2060 demographic projections
      – population projections to 2060 (single year of age x gender x race/origin)
    • ProximityOne Data Services (PDS)
      – multi-sourced Federal , Census and other data resources
3. U.S. by state/metro/county macro models (uses software and data above)
    • enable users to explore alternative model specifications/assumptions
    • produce alternative outcomes; perform impact analyses
4. Data Analytics Lab Sessions
    • variations on weekly Web-based sessions
5. Certificate in Data Analytics Modules
    • structured usage of Data Analytics Certificate program sessions/resources
6. Intern Participation: Data Analytics Tools Development and Applications
    • model specification & use; software development & use; data resource development & use

Forthcoming sections will review examples of Data Analytics Labs, how and where they are operating and experiences of participants.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data.

Mapping Block Group Data

.. demographic-economic patterns and characteristics can often be “masked” when using larger geographic areas such as counties and metros. Using data for small area geography such as census tracts and block groups can help identify clusters and areas of interest. Now, demographic-economic data for block groups (217,000 areas averaging 1,200 population covering the U.S. wall-to-wall) are tabulated annually from the American Community Survey (ACS). There are hundreds of subject matter items updated annually. Without specialized tools, it is very difficult to navigate through the maze of small area data that are available and make effective use of these data for analysis and decision-making.

Use the combination of API tools and GIS tools described in this Web section to visually and geospatially analyze demographic-economic characteristics of block groups. Covering the U.S. wall-to-wall and averaging 1,200 population, block groups are the smallest geographic tabulation area for data from the American Community Survey (ACS 5-year estimates). Block group (BG) data are also available from Census 2010 Summary File 1 (SF1) for the same block group area geography.

Patterns of Median Household Income
— San Francisco, CA by Block Group

Create map views similar to the one shown here for your areas of interest.

… Click graphic for larger view. View developed using CV XE GIS.

Install Tools
There are two software tools involved to create maps such as the one above for any county in the U.S. using block group level data for multiple years, flexibly choosing among more than a thousand subject matter items. Steps to use two tools are described in this Web section. You can perform these operations for any area in the U.S. and use the resulting maps and data in any manner.
• Demographic-Economic Data Extraction (DEDE) API tool
CV XE GIS software
See the corresponding Web sections above to install the Windows-based software. There is no fee to use these tools to perform operations described here. There are no block group subject matter datasets to download. The steps to use these tools are summarized here.

Get Help Using these Resources
The flexibility and breadth of data selection options afforded by access to thousands of subject matter items from multiple statistical programs requires several steps to use the data in GIS applications. Join us in a Data Analytics Lab session for additional assistance. We can go through/discuss any aspect of steps summarized here. There is no fee for the Data Analytics sessions.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data.

Using the Census API

The Census Bureau API is a powerful resource that enables you to flexibly and easily access wide-ranging demographic-economic data down to the census block level. The Census API is a well-designed tool geared more toward application developers. That is, you cannot enter a Web instruction that generates a profile, map or directly usable dataset. ProximityOne has developed tools you can use to do just that — at no cost. This section briefly reviews how these tools can be used. See related Web section.

Direct Use of the Census API
See this summary of how to use the API to retrieve decennial census data.
• See Census 2010 SF1 items available, documentation and examples
• See more about finding the right data.
• Join us in a Web session for overview, step-by-step & Q&A.

Visual Analysis of Block Group Demographic Patterns
The following view of the Chicago area shows patterns of Asian and Hispanic population by block group using Census 2010 Summary File 1 data. This map has been developed using data accessed via the Census API and transitioned for visual analysis using tools reviewed below. In this example, two block group layers are displayed — shown to the left of map in the legend (%Asian & %Hispanic). Color patterns assigned to interval values are shown for each layer in the legend.

Click graphic for larger view
Additional views: Asian aloneHispanic alone

Use no-cost tools available from ProximityOne to develop similar map views for your geography and subject matter. The ProximityOne tools extend the capabilities of the Census API resource by enabling the user to create a dataset and then integrate those data into a Census-sourced shapefile to geospatially analyze the data. Details follow.

1. Use Demographic-Economic Data Extraction (DEDE) Tool
The Demographic-Economic Data Extraction (DEDE), a Windows-based application, is used to programmatically control use of the Census API. The user starts DEDE, opens a geographic specifications file, a subject matter specifications file, and then clicks Run to access the specified data. The data are displayed in a spreadsheet where a row corresponds to a geographic area (block group) and columns correspond to the subject matter fields. Use the File
Optionally, stop here. You now have your dataset in Excel of CSV structure for direct use. The DEDE value-added benefit is the ability to select data by a list of geographic codes, a list of subject matter items and save the API retrieved data as a dataset — these operations are not directly available using the Census API alone.

Census 2010 SF1 Table P5
Census 2010 Summary File 1 Table P5 items for a selected area (census tract in this example) is shown below.

2. Using Census API Data & GIS; Mapping & Geospatial Analysis
After extracting the data using the Census API, visually/geospatially analyze the data using Geographic Information System GIS tools. See more about CV XE GIS software to perform these tasks — available at no fee to ProximityOneUser Group members.
Install CV XE GIS.
• Add your User Group ID to the CV XE GIS settings.
• Use the GeoGateway to add shapefiles to your GIS project.
• Use the CV XE GIS dBMerge to integrate Census API data into the shapefile.
• Make settings relevant to your needs; save your GIS project.

Zoom to areas of interest. Change labeling/colors/appearance. Set queries to filter layers showing only select geography. Add other geography.

ProximityOne User Group
Join the ProximityOne User Group to keep up-to-date with new developments relating to decision-making information resources. Receive updates and access to tools and resources available only to members. Use this form to join the User Group. There is no fee.

Support & DMI Web Sessions
Learn more about using resources described in this section. Join us in a Decision-Making Information Web session. There is no fee for these one-hour Web sessions. Each informal session is focused on a specific topic. The open structure also provides for Q&A and discussion of application issues of interest to participants. We can address your specific questions about tools to visually analyze patterns.