Performance & scale
Spatial indexes, parallelism, memory, and datasets too big to load at once.
Everything on Spatial Workflow about performance & scale, in the order that usually works: understand the idea, then run the task, then fix it when it goes wrong. 34 guides in total.
8 learning paths run through this topic — follow one if you would rather work through a route than pick a single guide.
🧭 Start with the concepts
The ideas the how-to guides below assume you already have.
- Spatial Indexes Explained: R-trees and Why Spatial Joins Are Fast spatial index r-tree explained
- Threads, Processes and the GIL: Parallelism in Python GIS Explained python gil threads processes gis
- Spatial SQL or GeoPandas? Choosing Where the Work Happens spatial sql vs geopandas
- PostGIS Spatial Indexes Explained: GiST and the Two-Phase Filter postgis gist spatial index
- Why GeoPandas Is Slow: The Four Real Bottlenecks why is geopandas slow
- GeoParquet and Columnar Storage Explained: Why It Reads So Much Faster geoparquet explained python
- Nearest-Neighbour Joins Explained: Distance, Ties and Search Radius nearest neighbour join explained
- Lazy Loading Explained: Why xarray Reads Nothing Until You Ask lazy loading xarray explained
- Chunked Arrays and Zarr Explained zarr chunked arrays explained
⚙️ Then do the task
Step-by-step walkthroughs with code you can copy and adapt.
- How to Speed Up Batch GIS Jobs with Parallel Processing in Python parallel processing gis python batch
- How to Cache Pipeline Steps So Only Changed Data Is Reprocessed cache gis pipeline steps python
- How to Reduce GIS File Size in Python Without Wrecking the Data reduce gis file size python
- How to Process a Very Large GeoPackage in Chunks with Python process large geopackage in chunks
- How to Split a Large Layer into Tiles for Batch Processing split layer into tiles python
- How to Read a Large PostGIS Table into Python Without Running Out of Memory read large postgis table python
- How to Build a Small, Fast Python GIS Docker Image small python gis docker image
- How to Replace a Row Loop with Vectorised GeoPandas Operations vectorise geopandas loop
- How to Use the Spatial Index Directly in GeoPandas (sindex) geopandas sindex spatial index
- How to Profile a Slow Python GIS Script and Find the Real Bottleneck profile slow python gis script
- How to Build a Distance Matrix Between Two Layers in Python distance matrix geopandas
- How to Measure Network Distance for Many Origin-Destination Pairs network distance matrix python
- How to Choose Chunk and Tile Sizes That Actually Help choose chunk and tile sizes
- How to Scale a GeoPandas Job with dask-geopandas dask geopandas scale python
🔧 When it breaks
The errors this topic produces, with the cause explained rather than just the command.
- How to Speed Up GeoPandas: Tips for Large Datasets geopandas performance optimization
- Fixing Memory Errors in GeoPandas When Working with Large Files geopandas memory error
- Batch GIS Job Slows Down and Runs Out of Memory: How to Fix It batch job memory leak python
- Parallel GIS Batch Job Hangs or Crashes with multiprocessing: How to Fix It multiprocessing hangs python gis
- OSError: Too Many Open Files in a Batch GIS Job too many open files python gis
- My PostGIS Spatial Query Is Slow: How to Fix It postgis spatial query slow
- GIS Tests Run Out of Memory or Time Out in CI: How to Fix It ci out of memory gis tests
- GeoPandas .apply() Takes Hours: How to Fix It geopandas apply slow
- Reading a COG from a URL Is Slow or Downloads Everything cog read is slow
- dask-geopandas Uses More Memory or Time Than Plain GeoPandas dask geopandas slower than geopandas
- My GeoJSON Is Too Big for the Browser geojson too big for browser
Learning paths through this topic
A curated route rather than a list — the guides in the order that builds on itself.
-
Spatial Statistics
Measure the pattern instead of eyeballing the map.
Open path → -
Networks & Routing
Distance along the street, not across the rooftops.
Open path → -
Spatial SQL & PostGIS
Move the heavy lifting from Python into the database.
Open path → -
GIS Data Cleaning
Turn messy spatial data into analysis-ready datasets.
Open path → -
Batch Processing
Process many files at once, reliably.
Open path → -
Automation & Pipelines
Build repeatable, config-driven GIS pipelines.
Open path → -
Cloud-Native GIS
Read the part you need, not the file it lives in.
Open path → -
Publishing Web Maps
Get the data into a browser without shipping the whole dataset.
Open path →
Related topics
Subjects that keep coming up alongside this one, most overlap first.