Help map some sidewalks for cities in the U.S.

United States cities are built for cars, with very few exceptions. From where I am sitting right now, I see this:

carscarscars

Cars zooming by incessantly at 70kph.

Finding your way in an urban space that is designed this way is tricky – and often dangerous – if you are walking or bicycling. Sidewalks are often not present, crossing streets can be very dangerous or even impossible. OSM has great tagging for bike lanes and sidewalks, but I find that these crucial tags are often missing on ways that need them most: the four or six lane urban arterials that you see in the picture above.

As I was sitting here asking myself how on earth I would get back to my hotel (which is 10 minutes away) safely, I thought to myself: ‘we can fix this problem and make the world a bit safer for those who can’t or won’t drive.’

MapRoulette to the rescue!

I created this challenge highlighting all primary and secondary ways that have nosidewalk tag in Tampa, Florida. (I am actually in Sarasota now, south of Tampa, but I already fixed all the ways there so that would be a boring challenge.) The idea is to look at the aerial image in JOSM or iD, see if there is a sidewalk, and add the appropriate tag. Adding sidewalk=no is actually just as important as adding both, right or left. Here is an example way from this challenge:

exampleway

Even zooming further in there is no sight of a sidewalk:

zoomedin

So let’s add that information:

tagging

And upload!

Create a Challenge for your city

The fun part is that you can easily replicate this challenge for your own city. Here’s what to do.

Overpass Turbo

First you head over to Overpass Turbo and run the query that highlights all highway=primary and highway=secondary that have no sidewalk tag:

overpassturbo

You can use my query as a template, replacing the GeocodeArea with the name of your city.

Once you have the results, export them to GeoJSON. Let’s use a gist:

gist1

gist2

You can now click on the gist link and see the result on GitHub as well:

github

We will need the ‘raw’ GeoJSON content, so click on the ‘Raw’ button and copy the link it leads you to.

Geojson2MapRoulette

Next we’ll use a little tool I created to easily turn the contents of a GeoJSON file into a MapRoulette challenge. To get it, head over to the Github repository and follow the instructions to install the tool.

The tool takes its configuration from a YAML file. The samples directory contains an example for this sidewalks challenge you can use as a template:

# the base URL for the MapRoulette server API to call
server: http://dev.maproulette.org/api
#server: "http://localhost:5000/api"
#server: http://maproulette.org/api

# server API admin credentials
user: devuser
password: mylittlesony

# source file or URL. You can give a list of URLs too, all data will be gathered and added to the same challenge.
source_url: https://gist.githubusercontent.com/anonymous/310005dc1dcf08f5c4a7/raw/74aa0d6a845a20511d0c450d715dab23d2a6c0d6/overpass.geojson
# source_file: ....

# source geojson property key to use as your task identifier (optional, will use random UUID if not given)
# identifier_property = ...

# Challenge metadata, see https://gist.github.com/mvexel/b5ad1cb0c91ac245ea3f for background
slug: sidewalks-sarasota
title: Add sidewalks to major roads in Sarasota
instruction: This way has no `sidewalk` tag. Usually you can see from the aerial imagery if there is a sidewalk or not. Please add the appropriate `sidewalk` tagging.
help: "Help make OSM be a better resource for safe, walkable streets! Many primary and secondary roads in the US are not safe for pedestrians if there is no sidewalk. This challenge highlights all `primary` and `secondary` ways that have no [`sidewalk`](https://wiki.openstreetmap.org/wiki/Key:sidewalk) tagging whatsoever. You can help by looking at aerial imagery and adding the appropriate `sidewalk` tagging. `sidewalk=no` is just as important to have as the 'positive' values. Thanks for helping make OSM better!"

The only items you would need to change are the source_url (use the raw GeoJSON github link you just copied), the slug (use sidewalks-YOURSTATE-YOURCITY or something similar – this will be the challenge URL component in MapRoulette) and the title (change the city name).

By default this configuration will post to dev.maproulette.org. If you want to post tomaproulette.org you would need to get in touch with me to get the credentials.

Once you have the YAML config file in order posting to MapRoulette is as simple as:

$ ./geojson2maproulette.py samples/sidewalks-sarasota.yaml --post --activate
Posting 364 tasks...
server alive: True
Updating challenge...
Reconciling tasks...
Done!

Let me know if you need any help with this or if you want me to create a challenge for you!

Facebooktwittergoogle_plus

Check out the new tutorial section

Recently we have launched a video tutorial section on improve.osm.org, to help our users get up to speed with our tools and let you know what are all the possibilities and all the versatility of our products for the community’

So far, we have 2 videos. One is focused on doing a gentle introduction to Improve OSM, some sort of a guided tour of the Improve OSM web site for new users. This is the perfect starting point if you’re new here.

The second one is a step-by-step guide to fixing your first one-way street in OpenStreetMap using the Improve OSM web site. You can see it here.

We’ll prepare more tutorial videos soon. Let us know what other aspects of our products you’d like to be featured in the future.

Facebooktwittergoogle_plus

How we imported Administrative Boundaries for Mexico from INEGI

The INEGI boundaries import project is focused on importing the data of the national, state, municipal and sub-municipal level divisions present in the MGN published by the INEGI in a community monitored process.

One of the current problems in OSM regarding Mexico’s data is the incompleteness of the administrative boundaries for municipalities. Municipalities are the second-level administrative division in Mexico, the first being the state. There are 2456 municipalities, including the ones in Mexico City which are also a second-level division just with a different name – delegations.

The main goal of this process is to enhance the current OSM administrative division coverage of Mexico with open data made available by the government at the end of 2014.

Import Process

The following steps describe the entire workflow we followed to import the boundary data.

  • Step 0 – Reprojection of INEGI dataset

Before any other step, the data released by INEGI has to be reprojected to WGS84 (EPSG:4326), from ITRF92, using QGIS, and saved as .shp file. An important thing to mention is that no simplification of the boundary geometries is considered whatsoever for this or any of the subsequent steps since the geometries are official government data.

  • Step 1 – Conversion to OSM data

Download the state boundary of interest relation from OSM and save it as an .osm file. In QGIS, using Vector > Research Tools > Select by Location, select the INEGI municipalities boundaries that are within the area of interest, in this case Quintana Roo state, and export the selection as .shp file.

Municipalities in Quintana Roo state.
Municipalities in Quintana Roo, as polygons.

The exported features will be polygons. In order to process them, they must be converted to lines in QGIS using the Polygons to Lines option, available in Vector > Geometry tools. Visually, the output will look the same as when the municipalities were polygons.

Municipalities in Quintana Roo, as lines.
Municipalities in Quintana Roo, as lines.

Using ogr2osm the .shp file containing the boundaries as lines is converted into an .osm file.

Before moving forward, the resulting .osm file has to be modified a bit. Using Notepad++, open the file and search and replace <nd ref=’ with <nd ref=’- and <node id=’  with <node id=’-, so the file will be with negative id.

The negative id is important because JOSM will know that this is new data, not yet added to the map.

Next, the .osm file can be converted to an .osm.pbf file using osmosis.

  • Step 2 – Processing

We load the .osm.pbf file from the previous step into an internal tool, called Mexico Split. The tool is designed to eliminate duplicate/overlapping ways by detaching them from their parent polygons and replacing them with a single common way of the two involved polygons.

mxsplit3
Detects overlapping ways and replaces them with a single common way.

Besides this main purpose, the tool also splits any resulting ways longer than 2000 segments in shorter ways, groups the ways in relationships according to the borders they define and adds some predefined tags to these ways and relations.

Tags added to relations:

type=boundary

INEGI:MUNID=<value_from_the_original_polygon>

name=<value_from_the_original_polygon>

Tags added to both ways and relations:

boundary=administrative

admin_level=6

source=INEGI, MGN 2014 v6.2

For example, data for Bacalar municipality contains the following information:

Bacalar municipality in Quintana Roo state.
Bacalar municipality in Quintana Roo state. (click for larger image)
  • Step 3 – Backup and metrics of existing OSM data

We took a backup of the current OSM data previous to the import of the regions that are going to be impacted, using Overpass API. Also, tag related metrics have been recorded – source, population, admin_center, admin_label, wikipedia etc. in order to have an overview of the newly added information.

  • Step 4 – Delete existing data from OSM and upload fresh data

In some cases the states already have some information regarding municipality boundaries (admin_level=6). These will be deleted, but before deletion we take a look at all the features and relations, to have a very good image of what we should put back in map data after the import.

Next, we upload the municipalities on a state by state basis.

  • Step 5 – Clean/verify the newly added data

This is a very important step because we verify the data that we’ve uploaded to make sure that there are no errors and manually re-link the admin_level=6 relations to the admin_level=4 boundaries, where required. Any other manual corrections are done at this step.

output_mSaeg1
An example of the newly added municipalities boundaries in Tabasco.

To ease the process of importing the municipality boundaries, we use the Mexico Import Map paint style for JOSM. It highlights the last node of every way, making it simple to see the length of every way.

Map styles - JOSM default vs. Mexico Import
Map styles – JOSM default vs. Mexico Import. (click for larger image)

The square node also has a certain degree of transparency, so we can see if there is a node under the node. To be able to work in a systematic way, it allows to quickly see duplicated nodes and see the difference between the admin_level=4 and admin_level=6.

Facebooktwittergoogle_plus

Improve OSM Tool Highlight: Cygnus

Improve OSM is all about making OpenStreetMap better with the help of billions of GPS points that users of Scout apps and others contribute automatically. This allows us to identify missing roads, one-way streets and turn restrictions that are not in OSM yet. However, our OSM engineers think about other ways to help make OSM better as well. Looking at what is going on in the OpenStreetMap community, I see a lot of groups and individuals struggle with the challenge of using external data in OpenStreetMap. Many governments now open up data that was closed before. With proper caution and preparation, we may be able to use this data in OSM.

The technical challenge here is to combine the new, external data with what is already present in OSM. In the GIS world this is called conflation. There are some tools out there to help with this task, but they are hard to use. So we started to think about this from a practical point of view and a specific use case: what if you have an external database of just roads, and you want to add only the new roads to OSM?

cygnus-osm-merge

We developed a tool that can do just that, and we called it Cygnus. I wrote about Cygnus on my OSM diary before, announcing it here and then writing about it being used by OSM user MikeN here. We added a link to the Cygnus web site to the ‘Other tools’ section of the ImproveOSM web site.

We made Cygnus as easy to use as possible, but it is still not straightforward. It should not be. Conflation is hard, and what you are doing is importing data into OSM, which should be done with the utmost caution! If you want to get started using Cygnus, please get in touch. We are happy to help!

Facebooktwittergoogle_plus