Tag Archives: Application programming interface

State and Regional Decision-Making Information

Organized on a state-by-state basis, use tools and geographic, demographic and economic data resources in these sections to facilitate planning and analysis. Updated frequently, these sections provide a unique means to access to multi-sourced data to develop insights into patterns, characteristics and trends on wide-ranging issues. Bookmark the related main Web page; keep up-to-date.

Using these Resources
Knowing “where we are” and “how things have changed” are key factors in knowing about the where, when and how of future change — and how that change might impact you. There are many sources of this knowledge. Often the required data do not knit together in an ideal manner. Key data are available for different types of geography, become available at different points in time and are often not the perfect subject matter. These sections provide access to relevant data and a means to consume the data more effectively than might otherwise be possible. Use these data, tools and resources in combination with other data to perform wide-ranging data analytics. See examples.

Select a State/Area

Alabama
Alaska
Arizona
Arkansas
California
Colorado
Connecticut
Delaware
D.C.
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Maryland
Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Montana
Nebraska
Nevada
New Hampshire
New Jersey
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Rhode Island
South Carolina
South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
West Virginia
Wisconsin
Wyoming

Topics for each State — with drill-down to census block
Visual pattern analysis tools … using GIS resources
Digital Map Database
Situation & Outlook
Metropolitan Areas
Congressional Districts
Counties
Cities/Places
Census Tracts
ZIP Code Areas
K-12 Education, Schools & School Districts
Block Groups
Census Blocks

Join me in a Data Analytics Lab session to discuss more details about accessing and using wide-ranging demographic-economic data and data analytics. Learn more about using these data for areas and applications of interest.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data. Contact Warren. Join Warren on LinkedIn.

Tip of the Day — Census Tract Data Analytics

.. tip of the day .. a continuing weekly or more frequent tip on developing, integrating, accessing and using geographic, demographic, economic and statistical data. Join in .. tip of the day posts are added to the Data Analytics Blog on an irregular basis, normally weekly. Follow the blog to receive updates as they occur.

This section is focused on tools and methods to access and use census tract demographic-economic measures. Median household income ($MHI), median housing value ($MHV) and other selected items are used to illustrate operations and options.

This section illustrates use of census tract data from the 2014 American Community Survey (ACS1014) 5-year estimates. These are the most comprehensive demographic-economic data from the Census Bureau at the census tract level. These “5-year estimates” are centric to mid-2012. See more about 2010-2021 annual estimates and projections.

Methods described here apply to many other geographies; see related tip sections. See related section on ZIP code applications.

Five data access and use options are reviewed. Each method illustrates how $$MHI, $MHV and other data can be analyzed/used in different contexts.

Option 1 – View the data as a thematic pattern map.
Option 2 – View, compare, rank query data in interactive tables.
Option 3 – Access data using API Tools; create datasets.
Option 4 – View $MHI in structured profile in context of related data.
Option 5 – Site analysis – view circular area profile from a location.

Related sections:
Census tracts main section
Evolution of Census Tracts: 1970-2010
Demographic-Economic Estimates & Projections
Census tract and ZIP code equivalencing
Using census tracts versus ZIP code areas
Single year of age demographics

Option 1. View the data as a thematic pattern map; use the GIS tools:
Patterns of Economic Prosperity ($MHI) by Census Tract … the following graphic shows $MHI for a portion of the Los Angeles metro. Accommodating different demographic-economic thresholds/patterns, different legend color/data intervals are used. The pattern layer is set to 80% transparency enabling a view of earth features. Click graphic for larger view, more detail and legend color/data intervals; expand browser window for best quality view.

– View developed using CV XE GIS and related GIS project.

See details about each option in the related Web page.

Join me in a Data Analytics Lab session to discuss more details about accessing and using wide-ranging demographic-economic data and data analytics. Learn more about using these data for areas and applications of interest.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data. Contact Warren. Join Warren on LinkedIn.

Data Analytics Trends in the Future

.. 10 ways that data analytics will evolve in the next 10 years … Data Analytics help us know more about “where we are now,” trends/factors getting here and alternative future scenarios. Predictive analytical tools help management examine change — what will change where, by how much, and when. Cause and effect models can tell us how change might impact you/us; they help us examine options that might alter outcomes. Geographic Information Systems tools help us knit together disparate geographic, demographic, economic and business data and better understand patterns using visual and geospatial methods.

Data Analytics Origins and Evolvement
Data Analytics had its origins in the 1960s/1970s with the availability of mainframe computing. Statistical packages, such as SAS and SPSS, enabled access to a range of Data Analytics tools by a wider array of potential users. Programming tools became more widely available creating customizable Data Analytics tools. Large scale machine-readable data became both available and processable for the first time.

Data Analytics growth has been hastened by more accurate and detailed digital geographic data and geographic-based data. Data Analytics expanded with PCs/microcomputers growth. The Internet has created new ways to perform Data Analytics applications on-the-fly and reachable by a wider set of users. Data collection tools/methods have expanded. Advances in secondary data development has improved the ability to use cause and effect modeling, forecasting and impact analysis.

Data Analytics Trends During the Next 10 Years

1. Organizations using Data Analytics effectively will experience improved growth opportunities relative to those not using these methods.

2. Data Analytics will continue to evolve as a set of integrated techniques and methods.

3. As Data Analytics use grows, a wider area of STEM-related occupations will expand in business and government.

4. TIGER/Line geographic data and digital map databases will improve in quality and scope enabling better location-based analyses using Data Analytics.

5. Use of API (Application Programming Interface) technology will play a larger role in Data Analytics as RESTmethods become more widely used.

6. Cause and effect, stochastic, simultaneous equation modeling will become more widely used as a core element of Data Analytics.

7. Annual time-series demographic-economic data will become available from the American Community Survey for use in Data Analytics modeling.

8. The role of Geographic Information Systems (GIS) tools, geospatial analysis and visual analysis will become increasingly centric to Data Analytics.

9. GIS tools will integrate more seamlessly with CRM, SCM and other Data Analytics methods.

10. Several challenges may impede Data Analytics evolvement including data linking (geocodes and file structuring) and data distribution architecture that makes data difficult to consume.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data.

Data Analytics & API Applications

Possibly the most well known APIs (Application Programming Interface) are the Google APIs. Among these, the Google Maps API, or its use, might be the most familiar. See about integrating political/statistical geography and Google Maps API. This section reviews different types of APIs.

Data Analytics and APIs
API technology/tools play an important role in accessing and integrating geographic, demographic and economic data. APIs offer two types of benefits. For Web-based applications, APIs provide the ability for on-the-fly access to selected demographic-economic buried in otherwise large datasets that typically otherwise involve downloading and preprocessing. A second important benefit/use is to extract data from those same datasets, reconfigure the structure of the data retrieved by API and create a new dataset structure that lends itself to analytical applications.

ProximityOne Data Analytics sessions review purpose, scope, strengths, advantages & limitations of APIs described in this section. Topics included in the related Web section include:
Federal Geographic-Demographic-Economic Data Access Using APIs
REST APIs
Federal Communications Commission
Bureau of the Census … much more than Census
Bureau of Economic Analysis
Bureau of Labor Statistics
Data Access & Analytics Applications
There are hundreds more APIs available from Federal agencies and wide-ranging sources.

Using APis and GIS: Visualizing Patterns; Geospatial Analysis
The following graphic has been developed first using API tools to create the underlying datasets. While the source data used in this application could have been downloaded and processed in legacy manner, the API tools provide on-the-fly development of the data in a structure required by the GIS software. These data are then integrated into shapefiles for use with the GIS software. Once the datasets are developed, they can be used in a myriad of application, GIS and otherwise.

Patterns of Male Hispanic Population Age 5 Years by ZIP Code
— Houston, TX Area


• Click graphic for larger view with ZIP Code labels and more detail.
• The graphic shows patterns of the Male Hispanic population 5 years of age as of Census 2010.
• The view illustrates how single year of age by gender by race/origin can be visually analyzed.
• See more about these data and single year of age demographics

Examples and More Detail
See the corresponding Web section on Data Analytics & Using APIs. Many See application examples are shown there.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data.

Homeownership Patterns by Census Block

The homeownership rate peaked in America in 2004 at approximately 69.2 percent.  Homeownership is defined as the percent of occupied housing units (households) that are owner-occupied.  The homeownership rate in 2013 is roughly the same as in 1995. The gradual decline continues.

hvr
        Homeownership Rate 1970Q1-2013Q3 not seasonally adjusted; based on CPS/HVS

The focus of this section is on creating thematic pattern map views depicting homeownership by census block for the Washington, DC area.  This section builds on the previous post Mapping Demographic Patterns: Census Blocks.

The starting place is where Mapping Demographic Patterns: Census Blocks left off — the required software and GIS project are installed on the computer.  The next step is to start the CV XE GIS software, open the DC GIS project and set the intervals for the thematic pattern views.

To compute homeownership, the minimum required data items/fields are the number of owner-occupied housing units and the number of housing units. By examining the Census 2010 SF1 table shells (xls), these items are found in Table H4. Tenure — owner/renter occupancy of occupied units (line 9259 in the xls file).  Looking at the  SF1 technical documentation (pdf) matrix section (sequential page 483, numbered page 6-321), it is determined that the items field needed are (H0040002 + H0040003) — owner occupied housing units and H0040001 (occupied housing units).  These items are already loaded into the DC block shapefile dbf.

Intervals are defined for the map view with queries that set blue to blocks with homeownership rate of 65% or more, orange to blocks with rate 50%-65% and red to population blocks with a rate below 50%.  Zero population blocks are set to gray.  The initial view shown below tells the visual story that more of Washington, DC has a homeownership rate below the national average than above the national average — and how these homeownership rate patterns are distributed by block.

Homeownership Rate by Census Block — Washington, DC
dc_ho1

Zoom-in to area east of U.S. Capitol complex
— transparency set to 60% enabling “see through” of color patterns
— mini profile of blue block at pointer
— h0040001: 79 occupied housing units and 54 owner occupied housing units (h0040002+h0040003)

dc_ho2

Similar thematic maps showing patterns of homeownership rate by census block may be created for any area in the U.S.  Procedures to access and use these no cost resources for Census 2010 Summary File 1 census block applications are summarized in the the APIGateway Guide.

In a future post, homeownership rates will be reviewed for all states and metros, 2008 through 2012, using annual American Community Survey (ACS) data.

Using Census 2010 Summary File 1 with API Technology

The Census 2010 Summary File 1 (SF1) contains the most detailed Census 2010 summary statistic data available that is tabulated at the census block  level.  The SF1 data are, and will continue to be, an important data resource throughout the 2010s and beyond. The scope of subject matter includes cross-tabulations of age, sex, households, families, relationship to householder, housing units, detailed race and Hispanic or Latino origin groups, and group quarters.

Skip the downloads!  Using pre-API technology methods, you would download the very large state by state zip files and proceed through a number of steps to use the data.  An alternative would be to use the Census Bureau FactFinder, but use of this tool is infeasible for most types of dataset development operations.

As an example, the Texas Census 2010 SF1 — view these data here (not recommended) — is contained in a 815 MB downloadable zip file. The zip file expands into a set of 48 files comprised of a geographic segment and 47 comma-delimited (CSV) structured files.  The expanded 48 files require 8.5 GB of space, still in CSV structure.  Specialized software is then required to transform these data into usable structures/data. This is for reference/history, we can now skip over using that enormous and complex to use data.

Using API Technology
How things change!  Use of API technology to access these data is reviewed in this section.  By using API-based applications, you can avoid downloading the very large SF1. This section reviews how you can use the APIGateway to develop a census block dataset. Many statistical programs and all popular geographies are supported by this tool, but the focus here is on census block demographics from Census 2010 SF1.

The next series of steps illustrate how to develop the sample dataset shown in the graphic presented below.  The graphic shows a spreadsheet oriented view of rows corresponding to census blocks and columns comprised of a census block geoid (the geocode and structured shorthand by which the geographic area is referenced) followed by columns/fields of selected SF1 items.

Washington DC tract 004100 SF1 extract

Using the CV XE APIGateway with Census 2010 SF1
Follow these steps to run through the demonstration application.  You can then use the APIGateway tool to process on geography and subject matter items of interest to you.

1. Install CV XE GIS Software
Use the CV XE GIS installer to install the software on your Windows computer.  Take all default settings.  More information about CV XE GIS.

2. Get Washington, DC GIS Project Files
Expand http://proximityone.com/dmd/2013_dc_dp.zip to folder c:\cvxe\1.

3. Start CV XE.  With CV XE running, start APIGateway
Use File>APIGateway from main menu bar.  The APIGateway form appears as shown below.

apigateway

The Batch Extraction operation involves two steps.
1 – Click main menu Settings>Batch Operation and specify settings.
2 – Close the Setting form. Click Tools>Batch Extraction to start processing.

The Batch Operations setting form is shown below.
APIGateway Batch Extract Settings

These settings will operate with no modification.  To use these settings,  close the form and click Tools>Batch Extraction to start processing.  The output file c:\cvxe\1\blk_sf1_2010_p003p004.dbf is created (Output Dataset) and contains the fields shown in the Field Names edit box.  The output file created will contain ALL records that meet the criteria of the Control File Query (substr(geoid,1,11)=’11001004100′) and processing blocks contained in the Control File (c:\cvxe\1\tl_2013_11_tabblock_dp.dbf).  The graphic at the top of this section provides a partial view of the file created.

You can use this version of the APIGateway to create your own datasets for any area of the U.S.  The Batch Operation in this version operates only with Census 2010 SF1 block level data.  The full version operates with many types of source data and supports wide ranging geographic levels — including ACS block group level data.

See the APIGateway Guide for more details regarding operations.

Using the Dataset Generated
There are at least two main ways the output dataset can be used.
1 – the block level data may be aggregated/analyzed in a tabular manner.
2 – the dbase version of the output dataset is structured in a manner that can be immediately joined with a shapefile for mapping and geospatial applications.

More about the Sample Dataset
The first row of the dataset shows selected data for census block 11-001-004100-1000.  P0030001 is the fieldname/shorthand for Census 2010 total population.  You can see the list describing these items using the table shells (xls) — see table P3 in the xls file. Field/item.column p0030002 is the White alone population.  Field name spelling is nitpicky; one character missed, incorrect or out of place can cause an error.  See sequential page 184 (numbered page 6-22 in the matrix section)in the SF1 technical documentation (pdf) to view the exact spelling of the field names and an alternative view of the table structure.

Washington DC tract 004100 SF1 extract

See related post regarding Mapping Census Blocks.  The next post on using data generated by the APIGateway will be November 19,2013.

Access New ACS 2012 3-year Data via API Gateway

The American Community Survey 2012 3-year demographic-economic estimates were released by Census on November 14, 2013.  These data may now be accessed using the CV XE API Gateway.  The APIGateway enables access to the data online through a client-server application.  Unlike the Census Bureau American FactFinder, which also provides access to these data, the APIGateway enables knitting together multi-sourced data into profiles and ready to use extractions for viewing the data visually in thematic maps.

As summarized in more detail in the ACS 2012 section, the ACS 2012 3-year data provide demographic-economic estimates, centric to 2011, for geographic areas of 20,000 or more population (documentation, xls table shells).  A complete list of all geographic tabulation areas is available in this spreadsheet (opens Excel spreadsheet).  In comparison to the ACS 2012 1-year estimates (tabulation areas of 65,000 population or more), these 3-year estimates provide the most recent wide-ranging demographic-economic data for all Core-Based Statistical Areas (CBSAs) — metropolitan statistical areas (MSAs) and micropolitan statistical areas (MISAs).

The ACS 2012 5-year estimates will be released on December 17, 2013.  The 5-year estimates provide largely the same scope of subject matter, as for the 1-year and 3-year data, for most Census Bureau demographic tabulation areas above the census block level.  In particular, the 5-year data include all block groups, census tracts, ZIP code areas, cities/places, counties and metros.  Most of the 5-year data will be available using the APIGateway and reviewed in upcoming posts.