What Is a Shapefile?
The Shapefile is the de facto standard format for geospatial vector data — geographic features represented as points, lines, and polygons with associated attribute data. Developed by Esri (Environmental Systems Research Institute) in the early 1990s for their ArcView GIS software, the Shapefile format has outlasted multiple generations of GIS software and remains one of the most widely used geospatial data formats in the world despite its age and limitations.
A shapefile stores geographic features such as:
- Points: city locations, weather stations, GPS waypoints, earthquake epicenters
- Polylines: roads, rivers, hiking trails, power lines, flight paths
- Polygons: country boundaries, building footprints, land use zones, lakes, forest areas
- Multipoints/Multilines/Multipolygons: complex features composed of multiple geometries
The Shapefile Is Not a Single File
Despite the singular name, a shapefile is a collection of at least three mandatory files sharing a common base name, plus optional companion files:
| File extension | Content | Required? |
|---|---|---|
.shp |
Geometry — the actual geometric shapes | Required |
.dbf |
Attribute table — feature attributes in dBASE III+ format | Required |
.shx |
Shape index — offsets into the .shp file for random access | Required |
.prj |
Projection — coordinate reference system (WKT format) | Strongly recommended |
.cpg |
Code page — character encoding for the .dbf file | Recommended (UTF-8) |
.sbn / .sbx |
Spatial index — for fast bounding box queries | Optional |
.qix |
QGIS spatial index | Optional |
.atx |
Attribute index | Optional |
.xml |
ISO 19115 metadata | Optional |
This multi-file design is the biggest practical problem with shapefiles. When sharing data, all files must be included — a shapefile missing its .prj file will display in the wrong location on a map (or not at all), and a missing .dbf means no attributes. The convention is to distribute shapefiles as ZIP archives containing all companion files.
The .shp File Structure
The .shp file is a binary file with a file header followed by variable-length records:
File Header (100 bytes)
├── File code: 9994 (big-endian int32)
├── File length in 16-bit words (big-endian int32)
├── Version: 1000 (little-endian int32)
├── Shape type: 1=Point, 3=Polyline, 5=Polygon, 8=Multipoint,
│ 11=PointZ, 13=PolylineZ, 15=PolygonZ, etc.
├── Bounding box: Xmin, Ymin, Xmax, Ymax, Zmin, Zmax, Mmin, Mmax
│ (8 × little-endian float64 = 64 bytes)
Record 1 (variable length)
├── Record header (8 bytes)
│ ├── Record number (big-endian int32, 1-indexed)
│ └── Content length in 16-bit words (big-endian int32)
└── Shape record (content length × 2 bytes)
├── Shape type (little-endian int32)
└── Shape-specific data (coordinates, parts, etc.)
Record 2 ...
Record N ...
Note the bizarre mix of big-endian and little-endian integers in the same file — a historical artifact of the format's 1990s origins.
Coordinate Reference Systems and the .prj File
The .prj file contains the Coordinate Reference System (CRS) in Well-Known Text (WKT) format. This is critical — geographic coordinates mean nothing without knowing what projection and datum they use.
Common CRS examples in .prj format:
# WGS 84 (GPS coordinates, latitude/longitude) — EPSG:4326
GEOGCS["GCS_WGS_1984",
DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],
PRIMEM["Greenwich",0.0],
UNIT["Degree",0.0174532925199433]]
# Web Mercator (Google Maps, OpenStreetMap tiles) — EPSG:3857
PROJCS["WGS_1984_Web_Mercator_Auxiliary_Sphere",
GEOGCS["GCS_WGS_1984",...],
PROJECTION["Mercator_Auxiliary_Sphere"],
UNIT["Meter",1.0]]
Working with Shapefiles in Python
GeoPandas (Recommended)
# pip install geopandas fiona pyproj shapely
import geopandas as gpd
import matplotlib.pyplot as plt
# Read a shapefile
gdf = gpd.read_file('countries.shp')
print(gdf.head())
print(f"CRS: {gdf.crs}") # e.g., EPSG:4326
print(f"Geometry type: {gdf.geom_type.unique()}")
print(f"Bounding box: {gdf.total_bounds}") # xmin, ymin, xmax, ymax
# Attribute operations (just like pandas)
europe = gdf[gdf['CONTINENT'] == 'Europe']
print(f"European countries: {len(europe)}")
print(europe[['NAME', 'POP_EST']].sort_values('POP_EST', ascending=False).head(10))
# Spatial operations
centroid = gdf.centroid # point at center of each feature
buffered = gdf.buffer(0.1) # 0.1 degree buffer
dissolved = gdf.dissolve(by='CONTINENT') # merge by attribute
# Reproject to a different CRS
gdf_mercator = gdf.to_crs(epsg=3857)
# Spatial join — find which country each city is in
cities = gpd.read_file('cities.shp')
cities_with_country = gpd.sjoin(cities, gdf, how='left', predicate='within')
# Plot
fig, ax = plt.subplots(figsize=(15, 10))
gdf.plot(column='POP_EST', cmap='YlOrRd', legend=True, ax=ax)
plt.title('World Population by Country')
plt.savefig('world_population.png', dpi=150, bbox_inches='tight')
# Write a new shapefile
europe.to_file('europe.shp')
# Write to GeoJSON (better for web and APIs)
europe.to_file('europe.geojson', driver='GeoJSON')
# Write to GeoPackage (modern replacement for shapefile)
europe.to_file('europe.gpkg', driver='GPKG')
Fiona (Lower-Level Access)
import fiona
# Inspect a shapefile
with fiona.open('countries.shp') as src:
print(f"Driver: {src.driver}")
print(f"CRS: {src.crs}")
print(f"Schema: {src.schema}")
print(f"Feature count: {len(src)}")
print(f"Bounds: {src.bounds}")
# Iterate features
for feature in src:
name = feature['properties']['NAME']
geom_type = feature['geometry']['type']
coords = feature['geometry']['coordinates']
print(f"{name}: {geom_type}")
break # just first feature
Command-Line with GDAL/OGR
# ogrinfo — inspect a shapefile
ogrinfo -al -so countries.shp
# ogr2ogr — convert between geospatial formats
# Shapefile → GeoJSON
ogr2ogr -f GeoJSON countries.geojson countries.shp
# Shapefile → GeoPackage
ogr2ogr -f GPKG countries.gpkg countries.shp
# Filter features during conversion
ogr2ogr -f GeoJSON europe.geojson countries.shp \
-where "CONTINENT = 'Europe'"
# Reproject while converting
ogr2ogr -f GeoJSON -t_srs EPSG:3857 countries_merc.geojson countries.shp
# Shapefile → CSV (coordinates as WKT)
ogr2ogr -f CSV countries.csv countries.shp -lco GEOMETRY=AS_WKT
The Shapefile's Limitations (and Why GeoPackage Is Better)
Despite its widespread use, the shapefile format has well-known limitations:
| Limitation | Details |
|---|---|
| Multiple files | A shapefile is 3-7+ files, making sharing error-prone |
| Column name length | Maximum 10 characters (from dBASE III) |
| Attribute types | Limited: string (max 254 chars), integer, float, date — no boolean, no array, no JSON |
| File size limit | 2 GB per .shp and .dbf file |
| Single geometry type | One shapefile can only contain points OR lines OR polygons — not mixed |
| No topology | No built-in support for shared edges or node constraints |
| Character encoding | .dbf files default to ASCII; UTF-8 requires an explicit .cpg file |
| No null geometry | NULL geometry is non-standard |
GeoPackage (.gpkg) is the modern OGC standard that addresses all these limitations — it is a single SQLite database file, supports mixed geometry types, has no column name length limit, supports UTF-8 natively, and has no 2 GB file size limit. QGIS, ArcGIS, GDAL, and GeoPandas all support GeoPackage fully.
Despite its limitations, the shapefile format is so entrenched that it will remain in common use for decades. Many government data portals (US Census TIGER files, UK Ordnance Survey, European Union GISCO) distribute data as shapefiles.
Major Shapefile Data Sources
- Natural Earth (naturalearthdata.com): free worldwide vector data at 1:10M, 1:50M, 1:110M scales — country boundaries, coastlines, rivers, populated places
- US Census Bureau TIGER: census blocks, tracts, counties, roads, water bodies
- OpenStreetMap: exported to shapefile via Geofabrik, GeoJSON, or the Overpass API
- GADM (gadm.org): administrative boundaries for every country at multiple admin levels
- USGS: topographic data, land cover, elevation contours
Related conversions
Frequent conversions across the catalogue: