This week Overture is releasing datasets that include new addresses across ten countries, new road features in Saudi Arabia and Italy from TomTom, and new building height data from Microsoft and Esri Community Maps contributors. Here's a preview of what you'll find in our October release.

Last month we released v1.0.0 of the Overture schema and “general availability” versions of four themes: base, buildings, divisions, and places. Our addresses theme launched in alpha with 200 million addresses for 14 countries; we anticipate adding more address data each month, country by country, starting with Switzerland in the August release.

The transportation theme remains in beta and we expect to make significant changes over the next couple months. In this post, we'll give you a preview of three key improvements to the transportation schema in the August release, which will be available next week.

Transportation schema changes​

The schema changes we plan to implement over the next few releases will better support routing and map display use cases, and make it easier for our users to integrate Overture transportation data into their pipelines and products. These improvements are necessary if we are to achieve our goal of a GA release in October. Here are the steps we're taking in the August release to move us toward that goal.

Subclasses​

A diagram of the new class and subclass structures

We added a new subclass property that refines the descriptions of segments by:

• clearly specifying the usage of a length of road, and using linear referencing instead of further segmentation to pinpoint a change in how the road is used
• properly classifying detailed information from OpenStreetMap

Subclasses require non-overlapping geometries, which will make it easier for routing engines to consume our road data and generate turn-by-turn directions. The new class and subclass structures allow us to capture more information about roads from our data sources. For example, we’ve been able to move more than 30 million “unknown” road segments from OSM into the new subclasses, laying the groundwork for richer and more detailed map displays with Overture data.

Here's how class, subclasses, and flags are modeled in the Overture schema:

class: footway
subclass_rules:
  - value: sidewalk
    between: [0, 0.6]
  - value: crosswalk
    between: [0.6, 1.0]
road_flags:
  - values: [is_bridge]
    between: [0.2, 0.3]
  - values: [is_under_construction]
    between: [0.3, 0.5]

class: service
subclass_rules:
  - value: driveway
    between: [0, 0.5]

class: primary
subclass: link
subclass_rules:
  - value: link

Connectors​

To better support routing use cases, we added to the segment feature type a new property called connectors: an array of IDs with pre-computed linear reference values that explicitly link segments and connector features via coordinates. The connectors property contains a list of connector features to which a segment feature is physically connected and the relative location of those connector features along the segment. Each connector is a possible routing decision point, meaning it defines a place along the segment in which there is possibility to transition to other segments that share the same connector.

This new property — a replacement for the existing connector_ids property — will reduce complexity and improve accuracy as we calculate the intersections between segment and connector features. We plan to deprecate the connector_ids property in the August release and remove it from the schema in the October release.

Here are examples from the Overture schema comparing the new and deprecated features:

id: overture:transportation:segment:123
type: Feature
geometry:
  type: LineString
  coordinates: [[0, 0], [0.03, 0], [0.10, 0]]
properties:
  theme: transportation
  type: segment
  version: 1
  subtype: road
  class: secondary
  connectors:
    - connector_id: fooConnector
      at: 0
    - connector_id: barConnector
      at: 0.3
    - connector_id: bazConnector
      at: 1
  road_surface:
    - value: paved

id: overture:transportation:segment:123
type: Feature
geometry:
  type: LineString
  coordinates: [[0, 0], [0.03, 0], [0.10, 0]]
properties:
  theme: transportation
  type: segment
  version: 1
  subtype: road
  class: secondary
  connector_ids: [fooConnector, barConnector, bazConnector]
  road_surface:
    - value: paved

Destinations​

Highway signs help us get from point A to Green Bay

Also in support of routing use cases, we added a destinations property to the segment feature type. This property describes the transitions from one segment to another on the way to a specified location. In turn-by-turn routing applications, this is what is known as “follow signs for” — the human-readable directions and signposts along a road, highway, or interstate that get us from point A to point Z, by way of any number of paths in between. We designed destinations with a flexible schema that will allow us to capture and model navigation data from many different sources.

Here's an example of how highway sign information in the picture above is modeled in the destinations schema:

destinations:
  - when:
    heading: forward
    from_connector_id: 08f274370d59a8840436ff3bad31c60e
    to_connector_id: 08f274370d59a8840436ff3bad31c60e
    to_segment_id: 08a274370d59ffff043fff3b802d7aa5
    final_heading: forward
    labels:
     - value: Green Bay
      type: unknown
     - value: I 43 North
      type: route_ref
    symbols:
     - NULL

Transportation splitter tool​

Recently we made our transportation splitter tool available in a public GitHub repository. The tool is an experimental PySpark script and Jupyter notebook that offers multiple ways to split Overture’s transportation features into smaller and simpler segments. Our new transportation guide includes several examples of how to use it. Please take it for a test drive and let us know what you think!

Overture has launched a new browser-based tool to give users a no-code option to interact with our data. The Explore website is now the quickest way to get started with Overture, no SQL required: https://explore.overturemaps.org

Checking out "Lion Crushing a Serpent" in Philadelphia’s Rittenhouse Park

Not a map​

The Explore tool might look like a map, but we prefer to think of it as an x-ray data inspector. We’ve punted on making cartographic decisions in favor of displaying all of Overture’s themes, all at once: addresses, base, buildings, divisions, places, and transportation. Our users are coming at the data with different perspectives and goals, and we want to provide as much information as possible. We also want to stoke your curiosity and inspire you to ask new questions of Overture data.

Seeing all the data all at once might be a bit overwhelming. We recommend zooming to an area of interest, hovering over the layers icon in the upper left corner of the map, and toggling the data themes off and on. You can click on an individual map feature, like the famous "Lion Crushing a Serpent" statue, to inspect its properties. Want to download the data? In the upper right corner of the Explore site, you can click on the Download Visible button to generate a GeoJSON file containing all the data visible in the browser.

The Explore site is currently in beta, and your feedback will inform the features we add in the coming months. If you have questions about the data, or if you want to report a problem with data quality or a glitch in the website, please click on the bug icon and file an issue on GitHub. (Note: we are actively working on fixing known issues with the downloaded data.)

Build with us​

This project grew out of the diverse experiences and skill sets of the engineers working on Overture. We’re using WebAssembly and Rust, specifically geoarrow-rs, to query and download Overture’s GeoParquet files. The front end is React and Maplibre, and we’re generating PMTiles to visualize the data. It’s a serverless site, using all cloud-native data storage formats. We also plan to make PMTiles available with each Overture release. Special thanks to Kyle Barron from Development Seed and Brandon Liu from Protomaps for collaborating with Overture's Developer Advocacy team on this project.

Explore is open source, under an MIT license, and we welcome contributions from the Overture community. Our goal is to give you the basic building blocks for creating your own tools around Overture data. Let us know what you make and we’ll happily share it on our community projects page.

Mapmakers rejoice!​

The May release of Overture Maps includes new high-resolution land cover data and new cartographic schema properties.

Our land_cover layer is vectorized data derived from the European Space Agency’s 2020 WorldCover (10m) rasters. It's similar to the land cover layer in the Daylight map distribution, but Overture Maps added higher-resolution data (zoom level 15) with more detail and land cover classes. You'll find 10 subtypes in the new data: snow, forest, urban, grass, crop, barren, wetland, moss, mangrove, and shrub.

Our May release also includes schema properties that offer cartographic "hints" for optimal use of Overture Maps data in mapmaking. We added min_zoom and max_zoom to define the recommended zooms for each resolution of land cover, using the common “slippy maps” zoom level specification. This is a first step toward improving the user experience for mapmakers. We plan to expand these properties in future releases of Overture Maps data.

Exploring land cover​

In the notebook example below, we'll show you how to extract, process, and visualize land cover data for an area of interest using lonboard and the Overture Maps Python command-line tool. We recommend that you consult this example in the lonboard docs to better understand the methods used here. You can view and download the complete notebook on Notebook Sharing Space.

To follow along, you'll need to have JupyterLab or Jupyter Notebook running and the following dependencies installed:

• lonboard
• overturemaps-py
• pandas
• GeoPandas
• Shapely

import pandas as pd
import geopandas as gpd
import overturemaps 
from shapely import wkb
from lonboard import Map, PolygonLayer
from lonboard.colormap import apply_categorical_cmap

# specify bounding box
bbox =  -78.6429, 39.463, -73.7806, 41.6242

# read in Overture Maps land_cover data type
table = overturemaps.record_batch_reader("land_cover", bbox).read_all()
table = table.combine_chunks()

# convert to dataframe
df = table.to_pandas()

# filter for higher resolution land_cover features
df_h = df[df.cartography.apply(lambda x: x['min_zoom'] == 8)]

# create color map for land_cover subtypes, loosely based on natural-color palette: https://www.shadedrelief.com/shelton/c.html
color_map = {
    "urban": [167, 162, 186],
    "forest": [134, 178, 137],
    "barren": [245, 237, 213],
    "shrub": [239, 218, 182],
    "grass": [254, 239, 173],
    "crop": [222, 223, 154],
    "wetland": [158, 207, 195], 
    "mangrove": [83, 171, 128],
    "moss": [250, 230, 160],
    "snow": [255, 255, 255],  
}

# apply color map to land_cover subtypes
colors = apply_categorical_cmap(df_h.subtype, color_map)

# dataframe to geodataframe, set crs
gdf = gpd.GeoDataFrame(
    df_h, 
    geometry=df_h['geometry'].apply(wkb.loads), 
    crs="EPSG:4326"
)

# create map layer 
layer = PolygonLayer.from_geopandas(
    gdf= gdf[['id','subtype', 'cartography', 'geometry']].reset_index(drop=True),
    get_fill_color=colors,
    get_line_color=colors,
)

#render map
view_state = {
    "longitude": -76.2,
    "latitude": 39.6,
    "zoom": 8,
    "pitch": 65,
    "bearing": 5,
}
m = Map(layer, view_state=view_state)
m

Last week Overture Maps announced the beta release of our schema and data. After months of hard work and steady improvements, we are nearing production-level stability. In a series of posts over the next few weeks -- starting with this one -- we’ll unpack the highlights and improvements you'll see in this release and beyond.

Making geospatial fast​

The first thing you'll notice is how much faster you can pull Overture Maps data out of the cloud. We made this possible by working closely with open-source collaborators on the repartitioning and spatial optimization of our GeoParquet files. Meanwhile, our friends at DuckDB, one of our favorite SQL tools, added a feature that improves the performance of queries with bounding boxes.

SELECT 
*
FROM 
    read_parquet('s3://overturemaps-us-west-2/release/2024-02-15-alpha.0/theme=buildings/type=building/*', filename=true, hive_partitioning=1)
WHERE 
    bbox.minx > -75.60154
    AND bbox.maxx < -74.955763
    AND bbox.miny > 39.867004
    AND bbox.maxy < 40.137992;

SELECT 
*
FROM   
    read_parquet('s3://overturemaps-us-west-2/release/2024-04-16-beta.0/theme=buildings/type=building/*', filename=true, hive_partitioning=1)
WHERE 
    bbox.xmin > -75.60154
    AND bbox.xmax < -74.955763
    AND bbox.ymin > 39.867004
    AND bbox.ymax < 40.137992

We're continuing to make things faster and easier for users. Along with the folks at Development Seed, an Overture Maps Foundation member, we're building special tools for Overture Maps data on top of lonboard, their Python library for visualizing large geospatial datasets in Jupyter. And recently our friends at Wherobots took a comprehensive look at how our use of GeoParquet makes querying and analyzing our data with Apache Sedona very efficient.

As you can see, we're moving forward with the community to iterate on data, software, and specifications with the shared goal of making geospatial fast.

Easier-to-use schema​

Another highlight of the beta release is the transition to an easier-to-use schema for our administrative boundary data. We first explored this idea with the Overture Maps community in February, and after two short months of work, the new divisions schema and data are ready to go. Here's a query to divisions that grabs geometries for all the countries in the world:

SELECT *
   FROM read_parquet('s3://overturemaps-us-west-2/release/2024-04-16-beta.0/theme=divisions/type=division_area/*', filename=true, hive_partitioning=1)
WHERE subtype = 'country';

You can see that the divisions query above is much simpler than a comparable query to admins:

WITH admins AS (
    SELECT *
    FROM read_parquet('s3://overturemaps-us-west-2/release/2024-04-16-beta.0/theme=admins/type=*/*', filename=true, hive_partitioning=1)
  )

  SELECT *
    FROM admins country_locality
    JOIN admins country_area
      ON country_area.locality_id = country_locality.id
   WHERE country_locality.locality_type = 'country';

We plan to deprecate admins by the July release. In the meantime, both admins and divisions will be available to users.

Bridges, islands, waterfalls, and more!​

We added more rich detail to our base layer in this release, including an infrastructure type with familiar features from Facebook’s Daylight map distribution. We also added new subtypes and classes for the land, land_use, and water feature types. You'll find a comprehensive listing of the subtypes and classes for each feature type in our schema reference docs. Ready to make your own map? We have a tutorial to help you get started.

Stay tuned for more highlights​

We'll be back soon with more posts that explore our path from the beta release to production. In the meantime, we invite you to get started with our data and share with us your comments and feedback.

Welcome to the Overture Maps engineering blog! We're excited to tell the technical stories behind this big, exciting, ambitious, intensely collaborative project we’ve been working on for more than a year. The engineers building Overture Maps are eager to connect with our developer community. In the posts we publish in the coming months, you’ll hear from team members who work at Amazon, Meta, Microsoft, TomTom, Esri, Development Seed, Precisely, and more. And we’d like to hear from you: feedback and contributions from folks working with our data and schema are crucial to our success. Thanks for joining us on this journey. We’re glad you’re here.