Category Archives: Shapefile

Block Group Demographic Data Analytics

.. use tools described here to access block group data from ACS 2016 (or ACS2017 in December 2018) using a no cost, menu driven tool accessing the data via API. Select from any of the summary statistic data. Save results as an Excel file or shapefile. Add the shapefile to a GIS project and create unlimited thematic pattern views. Add your own data. Join us in a Data Analytics Web session where use of the tool with the ACS 2017 data is reviewed.

See related Web section for more details.
– examine neighborhoods, market areas and sales territories.
– assess demographics of health service areas.
– create maps for visual/geospatial analysis of locations & demographics.

Illustration of Block Group Thematic Pattern Map – make for any area

– click graphic to view larger view
– pointer (top right) shows location of Amazon HQ2

Topics in this how-to guide (links open new sections/pages)
• 01 Objective Thematic Pattern Map View
• 02 Install the CV XE GIS software
• 03 Access/Download the Block Group Demographic-Economic Data
• 04 Download the State by Block Group Shapefile
• 05 Merge Extracted Data (from 03) into Shapefile (04)
• 06 Add Shapefile to the GIS Project; Set Intervals
• 07 Viewing Profile for Selected Block Group
• 08 BG Demographics Spreadsheet
• 09 Block Group Demographics GIS Project
• 10 Why Block Group Demographics are Important

Data Analytics Web Sessions
Join me in a Data Analytics Lab session to discuss more details about accessing and using wide-ranging demographic-economic data and data analytics. Learn more about using these data for areas and applications of interest.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data. Contact Warren. Join Warren on LinkedIn.

How the New York Metro is Changing

.. or more precisely, how the New York Metropolitan Statistical Area (MSA) is changing. As of Census 2010 the New York MSA (officially the New York-Newark-Jersey City, NY-NJ-PA MSA) consisted of 20 counties. With the new OMB metropolitan statistical areas defined as of February 2013, the New York MSA became 22 counties, absorbing the Poughkeepsie, NY MSA two counties (Dutchess and Orange). The Poughkeepie MSA was removed from the official MSAs. The delineation remained that way until the new September 2018 delineations when the Census 2010 delineation was restored. Now, the Poughkeepsie, NY MSA exists as a 2 county area and the New York MSA exists as a 20 county area (both as they existed geographically in Census 2010).

These metro-county relationships are shown in the graphic presented below. The Poughkeepsie, NY MSA is shown with the blue cross-hatch to the north and the New York MSA is shown with the salmon color pattern.

– view developed using the CV XE GIS software and related GIS project.
– see the related New York Metro Situation & Outlook report.

What Difference Does it Make?
A lot! First, during the interim period 2013-2018, the Poughkeepsie, NY MSA lost the metropolitan area identity/status as conferred by the OMB delineations. It might have been omitted from size class market development and research analyses. Related, that metro was not included as a tabulation or estimation area of MSAs by Federal statistical agencies. An example of the impact is that the official demographic estimates for the Poughkeepsie, NY MSA developed by the Census Bureau were not tabulated as such and omitted from various statistical reports. Also, the removal of designation and now adding the designation back, creates a hiccup in the time series — affecting both the Poughkeepsie NY MSA and the New York MSA.

Detailed Demographic Profiles for New York MSA and Poughkeepsie, NY MSA
.. click link to view profile.

New York-Newark-Jersey City, NY-NJ-PA MSA
  Bergen County, NJ (34003)
  Essex County, NJ (34013)
  Hudson County, NJ (34017)
  Hunterdon County, NJ (34019)
  Middlesex County, NJ (34023)
  Monmouth County, NJ (34025)
  Morris County, NJ (34027)
  Ocean County, NJ (34029)
  Passaic County, NJ (34031)
  Somerset County, NJ (34035)
  Sussex County, NJ (34037)
  Union County, NJ (34039)
  Bronx County, NY (36005)
  Kings County, NY (36047)
  Nassau County, NY (36059)
  New York County, NY (36061)
  Putnam County, NY (36079)
  Queens County, NY (36081)
  Richmond County, NY (36085)
  Rockland County, NY (36087)
  Suffolk County, NY (36103)
  Westchester County, NY (36119)
  Pike County, PA (42103)

Poughkeepsie-Newburgh-Middletown, NY (CBSA 39100)
  Dutchess County, NY (36027)
  Orange County, NY (36071)

Looking Forward
The September 2018 CBSA delineations define counties that will be used for Census 2020 (likely, there could be yet further changes) — 384 MSAs in the U.S. In the cases of the New York MSA and the Poughkeepsie, NY MSA, it appears that the geography (component counties) used for Census 2010 will be the same as for Census 2020. Going forward, ProximityOne estimates and projections will use the most current vintage of CBSAs.

Data Analytics Web Sessions
Join me in a Data Analytics Lab session to discuss more details about accessing and using wide-ranging demographic-economic data and data analytics. Learn more about using these data for areas and applications of interest.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data. Contact Warren. Join Warren on LinkedIn.

Insights into County/Metro Business Establishment Patterns

.. new data, released this past week, enable us to better assess county and metro business establishment patterns .. these data help businesses understand how sales of their products and services align with markets .. what counties might be underserved? .. where should marketing and sales operations be ramped up or reduced in scope/reallocated? .. what do these tell us us about organization of market territories? how are these characteristics trending by county? Use the this interactive table to examine business establishments characteristics by county and metro by type of business. See more detail in the related Web section about topics covered in this post. 

• U.S. establishments rose 1.2% from 7,663,938 in 2015 to 7,759,807 in 2016.
• First quarter employment was up 2.1% from 124,085,947 to 126,752,238
• Annual payroll was up 2.9% from $6.3 trillion in 2015 to $6.4 trillion in 2016.

Based on the 2016 data (new April 2018), there are 7.3 million establishments in Metropolitan Statistical Areas (MSAs). In these MSAs ..
• 5.3 million establishments have fewer than 10 employees.
• 6,522 of these establishments have 1,000 or more employees.
• 374 of these establishments have 5,000 or more employees.

MSAs by Number of Establishments with 1,000 or more Employees
The following graphic shows Metropolitan Statistical Areas (MSAs) by number of establishments with 1,000 or more employees. Click graphic for larger view view showing metros labeled with number of these establishments. Expand brower to full window for best quality view.
.. View developed with CV XE GIS software and related GIS project.

Business Establishment Characteristics Updated in Metro Reports
Examine 2014, 2015, 2016 mini trend profiles for establishments by 2-digit sector in Metro Situation & Outlook Reports Choose any MSA by clicking column 3 in this table .. then view section 6.3. Examples:
New York .. Los Angeles .. Miami .. Chicago .. Dallas .. Houston .. Denver .. Seattle

Gaining Insights
Gain insights into these types of patterns by county by detailed type of business (NAICS). Use the interactive table below to examine business establishment characteristics for counties and metros of interest. Data reviewed in this section are based on the Census-sourced County Business Patterns released in April 2018. We have integrated current population estimates with the establishment data in the interactive table.

Tools You Can Use
• create maps and geospatially analyze business establishments at 6-digit industry detail with ready-to-use GIS project/tools .. see related section for details.
• Use the APIGateway tools to build custom business establishment datasets.
• Use the interactive table to query/view sort business establishment characteristics by county and metro.

Data Analytics Web Sessions
Join me in a Data Analytics Web Session, every Tuesday, where we review access to and use of data, tools and methods relating to GeoStatistical Data Analytics Learning. We review current topical issues and data — and how you can access/use tools/data to meet your needs/interests.

About the Author
Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data. Contact Warren. Join Warren on LinkedIn.

Location-Based Demographics Update

.. tools you can use to examine characteristics of addresses/locations .. many of us are interested in knowing attributes of addresses or locations. Often knowing address latitude-longitude is important so that the addresses can be viewed on a map .. see below.  Some might need to know what census block, or other geography, in which an address is located .. or what school district is an address located in.  Others need to know demographic-economic attributes of the neighborhood or area where an address is located.  These types of attributes can be obtained for addresses using the Location-Based Demographic (LBD) tools.  The LBD tool has just been made a part of the CV XE GIS software.  The LBD tool is available in all versions of CV XE GIS, including the no fee User Group version. See more about using the LBD tools to look-up and analyze address/location attributes.

Viewing Geocoded Addresses on a Map – automatically
The following view shows addresses geocoded using the LBD tool. Markers show addresses of 27 Trader Joe’s locations in the Los Angeles area. LBD automatically creates a shapefile that is added to your GIS project. The markers are labeled with population ages 18 and over in the corresponding census tract. Marker color/styles reflect different levels of median household income. A separate census tract layer shows patterns of economic prosperity.

Click graphic for larger view. Expand browser window. A mini profile is displayed showing demographic-economic attributes for the marker at pointer.

View the locations without the tract thematic pattern layer:

Make similar views for your addresses.

Get Started Using the LBD Tool
1 – join the User Group .. click here to join (no fee).
2 – run the installer to install on a Windows machine .. requires your userid.
3 – with CV XE GIS running, click Tools>Find Address/LBD
    enter an address .. a form appears showing characteristics of the address.
4 – see more about using the tools on the LDB page.

GeoStatistical Data Analytics Learning Sessions
We are developing a series of “GeoStatistical Data Analytics” (GSDA) Learning Sessions/modules. One of these is focused on using the LBD tools and methods in the broader context of data analytics. We plan to develop the GSDA models for self-guided use by analysts/practitioners as well as in the classroom setting with teacher/student materials. Upcoming blog posts will describe the program in more detail.

Data Analytics Web Sessions
Join me in a Data Analytics Web Session, every Tuesday, where we review access to and use of data, tools and methods relating to GeoStatistical Data Analytics Learning. We review current topical issues and data — and how you can access/use tools/data to meet your needs/interests.

About the Author
Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data. Contact Warren. Join Warren on LinkedIn.

Analyzing Block Group Demographics

.. tools & data to analyze sub-census tract households, education, income, housing, more … Block Groups, subdivisions of census tracts, are the smallest geographic areas for which “richer demographics” are developed by the Census Bureau. Block group demographic-economic estimates, based on Census 2010 geography, are annually updated beginning with American Community Survey (ACS) 2010. The latest ACS estimates for these 217,740 areas covering U.S. wall-to-wall are from ACS 2015. The ACS 2016 update will be released in December 2017.  See the related Web section for more detail about accessing and using block group geography and demographic-economic data.

Patterns of Economic Prosperity by Block Group
The following graphic shows patterns of median household income by block group in the Houston, TX area. Markers show block groups with 10 or more housing units having value of $2 million or more. Markers are labeled with the number of housing units having value of $2 million or more in that block group. Click graphic for larger view, more detail and legend color/data intervals. This map illustrates the geographic level of detail available using block group demographics and the relative ease to gain insights using geospatial data analytics tools.

– View developed using CV XE GIS and related GIS project.

Block Group Demographic-Economic Data & Shapefiles
… selection of key demographic-economic attributes; annual update
… subject matter categories include:
  • Total population>
  • Population by gender iterated by age
  • Population by race/origin
  • Households by type of household
  • Educational attainment by detailed category
  • Household Income by detailed category
  • Housing units by owner/renter occupancy
  • Housing units by units in structure
  • Housing units by detailed value intervals

See the related Web section for a detailed list of items.

Use these Data on Your Computer
Use the above U.S. national scope dataset with your own software or in ready-to-use GIS projects with the CV XE GIS software.

Join me in a Data Analytics Lab session to discuss more details about accessing and using wide-ranging demographic-economic data and data analytics. Learn more about using these data for areas and applications of interest.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data. Contact Warren. Join Warren on LinkedIn.

Life Expectancy Change by County, 1980-2014

.. data and tools to examine changing life expectancy by county. Use the interactive table to examine life expectancy characteristics and related demographics for counties and regions of interest. Use the related GIS project and datasets to examine life expectancy contextually with other geography & subject matter. See details below. These data and tools are part of the ProximityOne health data analytics resources.

Life expectancy is rising overall in the United States, but in some areas, death rates are going in the other direction. These geographic disparities are widening.

Life Expectancy Change by County, 1980-2014
The following graphic shows patterns of the change in life expectancy change from 1980 to 2014. Click graphic for larger view. Expand browser window for best quality view.

– View developed using CV XE GIS and related GIS project.
– see below in this section about using this GIS project.

Life expectancy is greatest in the high country of central Colorado, but in many pockets of the U.S., life expectancy is more than 20 years lower. These data are based on research and analysis by the University of Washington Institute for Health Metrics and Evaluation.

Examining life expectancy by county allows for tracking geographic disparities over time and assessing factors related to these disparities. This information is potentially useful for policymakers, clinicians, and researchers seeking to reduce disparities and increase longevity.

Life Expectancy Change by County, 1980-2014 — drill-down view
— South Central Region
The following graphic shows patterns of the change in life expectancy change from 1980 to 2014. Click graphic for larger view. Expand browser window for best quality view. The larger graphic shows counties labeled with change in life expectancy from 1980-2014.

– View developed using CV XE GIS and related GIS project.
– see below in this section about using this GIS project.

Additional Views — use the GIS project to create your own views
.. click link to view
Alaska
Hawaii
Minneapolis metro

Using the Interactive Table
Use the interactive table to view, rank, compare life expectancy characteristics. This graphic shows California counties ranked on life expectancy change 1980-2014 in descending order. Select states or metros of interest. Click graphic for larger view.

Join me in a Data Analytics Lab session to discuss more details about accessing and using wide-ranging demographic-economic data and data analytics. Learn more about using these data for areas and applications of interest.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data. Contact Warren. Join Warren on LinkedIn.

Creating & Using Location Shapefiles

.. GIS tools and methods to develop and update location shapefiles .. location shapefiles are essential to most GIS applications. Location shapefiles, or point shapefiles, enable viewing/analyzing locations on a map and attributes of these locations such store or customer ID, street address, city, date updated, value, ZIP code and wide-ranging attributes about the location. This section reviews tools and methods to develop and use location shapefiles. See more detail about topics covered in this section in the related Web page.

Viewing/Analyzing Store Locations in the Dallas, TX Area
The following graphic illustrates how store locations can be shown in context of other geography and associated demographic-economic attributes. This view shows store locations (red markers) in context of Dallas city (blue cross-hatch pattern) and broader metro area. Markers shown in this view are based on a location shapefile created using steps described below. The identify tool is used to click on a location and show attributes in a mini-profile.

.. view developed with ProximityOne CV XE GIS and related GIS project.

View the locations contextually with thematic patterns by tract or other geography. Combine views of store, customer, agent, competitor and other location shapefiles.
The following view shows patterns of median household income by census tract.

.. view developed with ProximityOne CV XE GIS and related GIS project.

Development of location shapefiles often starts with a list of addresses. Locations are not always address-oriented; they might be geographically dispersed measurement or transaction locations — having no address assigned. In applications reviewed here, locations are organized as rows in a CSV file. Each CSV file contains like-structured attributes for each location. The example used in this section uses store locations located in the Dallas, TX area.

There are two basic methods used to create location shapefiles: 1) geocoding address-data contained in the source data file or 2) using the latitude-longitude of the location included in the source data file record. The focus here is on option 2 — using the latitude-longitude of the location already present in the source data file.

Creating a Location Shapefile
The process of creating a location shapefile uses the CV XE GIS Manage Location Shapefile feature. With CV running, the process is started with File>Tools>ManageLocationShapefile. The following form appears.

.. ManageLocationShapefile feature/operation in ProximityOne CV XE GIS.

CV XE GIS provides other ways to create location shapefiles:
• Tools>AddShapes>Points — click points on the map window canvas.
• Tools>FindAddress — creates a single point shapefile based on specified address.
• Tools>FindAddress (Batch) — creates a point shapefile based on specified file of address records.
See details in User Guide.

Steps to Create a Location Shapefile
The process of creating the shapefile “C:\cvxe\1\locations1pts.shp” can be viewed by clicking the Run button on the form (with CV running). Two input CSV structured files are required:
• data definition file
• source data file

There are two sets of illustration location input files included with the CV installer:
• locations1_dd.csv and locations1.csv (7 locations in Johnson County, KS)
• locations2_dd.csv and locations2.csv (252 locations in Dallas and Houston)
These files are located in the \1 (typically c:\cvxe\1) folder. The marker/location shapefile used in the map shown above was created using the lcoations2 input files.

Data Definition File
The Data Definition (DD) file is an ASCII/text file structured as a CSV file. It may created with any text editor. The DD file is specific to the source data file. But in the case of recurring source data files for different periods the same DD file might apply to many source data files. There are several rules and guidelines for development of the DD file:
• there is one line/record for each field in the source data file.
• each line/record must be structured in an exact form:
.. each line/record is comprised of exactly 4 elements separated by a comma:
.. 1 field name for subject matter item
– must consist of 1 to 10 characters and include no blanks or special characters
.. 2 field type: C for character, N for numeric
.. 3 field length: an integer specifying the maximum with of the field
.. 4 maximum number of decimals for field (value is 0 for character fields)
The DD File must include three final fields:
LATITUDE,n,12,6
LONGITUDE,n,12,6
GEOID,c,15,0
The structure of these three DD file records must be as shown above. The source data file, described below, must have the LATITUDE and LONGITUDE fields populated with accurate values. The GEOID field may populated with either an accurate value of placeholder value like 0.

Example. Data for each store for the default DD file name “C:\cvxe\1\locations1_dd.csv” include the following fields/attributes:
  NAME,C,45,0
STORE,c,15,0
ADDRESS,c,60,0
CITY,c,40,0
LATITUDE,n,12,6
LONGITUDE,n,12,6
GEOID,c,15,0

Optionally create a DD File using the Create DD File button on the form. Clicking this button will create a DD File containing attributes of the dBase file specified in the associated edit box. The DD File name is created from the dBase file name. If the dBase file name is “c:\cvxe\1\locations1pts.dbf”, the DD File will be named “c:\cvxe\1\locations1pts_dd.csv”.

About the GEOID
The GEOID is a 15 character code which defines the Census 2010 census block containing each location. The GEOID is generally assigned by the ManageLocationShapefile operation and is one of the important and distinctive features of this tool. The GEOID is used to uniquely determine, with the GIS application, any of the following: state, county, census tract, block group, or census block.

The GEOID, as used in this section, is the 15 character Census 2010 geocode for the census block. The GEOID value 481130002011012 (see in location profile in map at top of section) is structured as:
state FIPS code: 48 (2 chars)
county FIPS code: 113 (3 chars)
census tract code 000201 (6 chars)
census block code: 1012 (4 chars) (block group code: 1 — first of 4 characters)

About the Source Data File
The Source Data File is an ASCII/text file structured as a CSV file. It is typically developed by exporting/saving an Excel or dBase file in CSV structure. There are several rules and guidelines for development of the source data file:
• fields must be structured and arranged as defined in the DD File.
• character fields must not contain embedded commas.
• final items in record sequence must be:
.. LATITUDE – must have accurate decimal degree value; 6 digit precision suggested.
.. LONGITUDE- must have accurate decimal degree value; 6 digit precision suggested.
.. GEOID – this may be 0, not assigned or the accurately assigned GEOID value.
– optionally create/rewrite the GEOID used in the new shapefile.

Updates; Combining Vintages of Location Attributes
Location based data might update frequently, even daily. The recommended method to add, update and extend the scope of location-based data is to create new address shapefiles corresponding to different vintages or dates covered. The structure of the files must be the same so that they files can be used together or separately. Suppose there is one set of data covering year to date and a second set of data covering the following month. The ManagePointShapefile operation would be run once for each time period. Two shapefiles would be created. These shapefiles may be added to a GIS project and used separately or in combination to view/analyze patterns.

Join me in a Data Analytics Lab session to discuss more details about accessing and using wide-ranging demographic-economic data and data analytics. Learn more about using these data for areas and applications of interest.

About the Author
— Warren Glimpse is former senior Census Bureau statistician responsible for innovative data access and use operations. He is also the former associate director of the U.S. Office of Federal Statistical Policy and Standards for data access and use. He has more than 20 years of experience in the private sector developing data resources and tools for integration and analysis of geographic, demographic, economic and business data. Contact Warren. Join Warren on LinkedIn.