Skip to content

earth-osm

One-command to extract infrastructure data from OpenStreetMap

📚 Overview

earth-osm downloads, filters, cleans and exports infrastructure data from OpenStreetMap (OSM). It provides a Python API and a CLI interface to extract data for various infrastructure types, such as power lines, substations, and more.

🌟 Key Features

  • 🔌 Extracts infrastructure data from OSM
  • 📅 Historical data support (specify target dates)
  • 🧹 Cleans and standardizes the data (coming soon)
  • 🚀 No API rate limits (data served from GeoFabrik)
  • 🐍 Provides a Python API
  • 🖥️ Supports multiprocessing for faster extraction
  • 📊 Outputs data in .csv and .geojson formats
  • 🌍 Supports global data extraction

  • 🖱️ Easy-to-use CLI interface

  • 🌐 Dual data sources: GeoFabrik (default) and live Overpass API

🚀 Getting Started

Installation

Install earth-osm using pip (recommended):

pip install earth-osm

Or with conda:

conda install --channel=conda-forge earth-osm

Basic Usage

Extract OSM data using the CLI:

earth_osm extract power --regions benin monaco --features substation line

This command extracts power infrastructure data for Benin and Monaco, focusing on substations and power lines. By default, the resulting .csv and .geojson files are stored in ./earth_data/out.

Load the extracted data using pandas:

import pandas as pd
import geopandas as gpd

# For Pandas
df_substations = pd.read_csv('./earth_data/out/BJ_raw_substations.csv')

# For GeoPandas
gdf_substations = gpd.read_file('./earth_data/out/BJ_raw_substations.geojson')

🛠️ CLI Reference

Extract Command

earth_osm extract <primary> --regions <region1> <region2> ... [options]

Arguments:

  • <primary>: Primary feature to extract (e.g power)

Required Options:

  • --regions: Specify one or more regions using ISO 3166-1 alpha-2, ISO 3166-2 codes, or full names

Tip: A list of regions is available at regions.md

Optional Arguments:

Argument Description Default
--features Specify sub-features of the primary feature All features
--update Update existing data False
--no_mp Disable multiprocessing False (MP enabled)
--data_dir Path to data directory './earth_data'
--out_dir Path to output directory Same as data_dir
--out_format Export format(s): csv and/or geojson ['csv', 'geojson']
--agg_feature Aggregate outputs by feature False
--agg_region Aggregate outputs by region False
--source Data source: geofabrik (default) or overpass geofabrik

🐍 Python API

For more advanced usage, you can use the Python API:

import earth_osm as eo

eo.save_osm_data(
    primary_name='power',
    region_list=['benin', 'monaco'],
    feature_list=['substation', 'line'],
    update=False,
    mp=True,
    data_dir='./earth_data',
    out_format=['csv', 'geojson'],
    out_aggregate=False,
    data_source='geofabrik'  # or 'overpass'
)

For historical data, add the target_date parameter:

from datetime import datetime

# Download historical data for a specific date
eo.get_osm_data(
    region_str='malta',
    primary_name='power',
    feature_name='line',
    target_date=datetime(2020, 1, 1)  # Jan 1, 2020
)

🛠️ Development

To contribute to earth-osm, follow these steps:

  1. (Optional) Install a specific version of earth_osm: 

    pip install git+https://github.com/pypsa-meets-earth/earth-osm.git@<required-commit-hash>

  2. (Optional) Create a virtual environment for Python >=3.10: 

    python3 -m venv .venv
source .venv/bin/activate

  3. Install the development dependencies: 

    pip install git+https://github.com/pypsa-meets-earth/earth-osm.git
pip install -e .[dev]

  4. Read the CONTRIBUTING.md file for more detailed information on how to contribute to the project.

🤝 Community

Join our Discord community to connect with other users and contributors, ask questions, and get support.

📚 Documentation

For more detailed information, check out our full documentation.

Made with ❤️ by the PyPSA meets Earth team

earth-osm logo