ImproveOSM with your own GPS data – a Field Report

We launched ImproveOSM about 6 months ago as a way to turn the vast amounts of GPS data that Scout users give us into useful and actionable hints mappers can use to add turn restrictions, missing roads as well as wrong or missing one-way streets. The response has been incredible — since we launched, more than 26 thousand hints have been processed, leading to more than 16 thousand improvements to the map worldwide. I think that is a fantastic result, and we will keep working to make ImproveOSM better based on your feedback.
Initially, we just used our own GPS data to generate the hints. But there is no reason why we couldn’t process any GPS data we can get from other sources. So I was really excited when long time Brazil mapper Wille Marcel got in touch with a cool idea. He worked with the Brazilian Environment Ministry, which collects GPS data of the vehicles that work in environmental monitoring. Most of the data are in rural areas where OSM is much less complete. So this was a perfect fit for ImproveOSM’s missing roads tool.After getting the proper permissions from the agency, Wille sent us the GPS data and we started analyzing it.

sparse-missing-roads

We quickly realized that the GPS data is much less dense than what we are used to working with. Some missing roads were only driven once. Our algorithm, tuned to higher density data, initially only detected a few tiles. We decided to loosen the detection threshold significantly for this particular dataset. After a few iterations of tweaking and testing, we ended up withmore than 5000 tiles containing missing roads based on Wille’s GPS data.

overview

The missing roads in Brazil are on ImproveOSM now, so why not go to the web site or fire up the ImproveOSM JOSM plugin and help the Brazilian community out by adding some missing roads?

If you are in a similar position as Wille and know of a source of free and open GPS data for your country, please get in touch with me so we can look at the data and see if we can include it in ImproveOSM.

We are already working with a number of other folks who have lots of GPS data. Soon, the number of missing roads, one-ways, and turn restrictions in ImproveOSM will be much, much bigger. We are also working on a host of new features, so I hope you will stay tuned to the ImproveOSM blog to be among the first to hear about what we have up our sleeves for ImproveOSM and other OSM related projects we are working on. And follow us on Twitter at @ImproveOSM!

 See also Wille’s post about this collaboration (in Portuguese).

Facebooktwittergoogle_plus

Help map some sidewalks for cities in the U.S.

United States cities are built for cars, with very few exceptions. From where I am sitting right now, I see this:

carscarscars

Cars zooming by incessantly at 70kph.

Finding your way in an urban space that is designed this way is tricky – and often dangerous – if you are walking or bicycling. Sidewalks are often not present, crossing streets can be very dangerous or even impossible. OSM has great tagging for bike lanes and sidewalks, but I find that these crucial tags are often missing on ways that need them most: the four or six lane urban arterials that you see in the picture above.

As I was sitting here asking myself how on earth I would get back to my hotel (which is 10 minutes away) safely, I thought to myself: ‘we can fix this problem and make the world a bit safer for those who can’t or won’t drive.’

MapRoulette to the rescue!

I created this challenge highlighting all primary and secondary ways that have nosidewalk tag in Tampa, Florida. (I am actually in Sarasota now, south of Tampa, but I already fixed all the ways there so that would be a boring challenge.) The idea is to look at the aerial image in JOSM or iD, see if there is a sidewalk, and add the appropriate tag. Adding sidewalk=no is actually just as important as adding both, right or left. Here is an example way from this challenge:

exampleway

Even zooming further in there is no sight of a sidewalk:

zoomedin

So let’s add that information:

tagging

And upload!

Create a Challenge for your city

The fun part is that you can easily replicate this challenge for your own city. Here’s what to do.

Overpass Turbo

First you head over to Overpass Turbo and run the query that highlights all highway=primary and highway=secondary that have no sidewalk tag:

overpassturbo

You can use my query as a template, replacing the GeocodeArea with the name of your city.

Once you have the results, export them to GeoJSON. Let’s use a gist:

gist1

gist2

You can now click on the gist link and see the result on GitHub as well:

github

We will need the ‘raw’ GeoJSON content, so click on the ‘Raw’ button and copy the link it leads you to.

Geojson2MapRoulette

Next we’ll use a little tool I created to easily turn the contents of a GeoJSON file into a MapRoulette challenge. To get it, head over to the Github repository and follow the instructions to install the tool.

The tool takes its configuration from a YAML file. The samples directory contains an example for this sidewalks challenge you can use as a template:

# the base URL for the MapRoulette server API to call
server: http://dev.maproulette.org/api
#server: "http://localhost:5000/api"
#server: http://maproulette.org/api

# server API admin credentials
user: devuser
password: mylittlesony

# source file or URL. You can give a list of URLs too, all data will be gathered and added to the same challenge.
source_url: https://gist.githubusercontent.com/anonymous/310005dc1dcf08f5c4a7/raw/74aa0d6a845a20511d0c450d715dab23d2a6c0d6/overpass.geojson
# source_file: ....

# source geojson property key to use as your task identifier (optional, will use random UUID if not given)
# identifier_property = ...

# Challenge metadata, see https://gist.github.com/mvexel/b5ad1cb0c91ac245ea3f for background
slug: sidewalks-sarasota
title: Add sidewalks to major roads in Sarasota
instruction: This way has no `sidewalk` tag. Usually you can see from the aerial imagery if there is a sidewalk or not. Please add the appropriate `sidewalk` tagging.
help: "Help make OSM be a better resource for safe, walkable streets! Many primary and secondary roads in the US are not safe for pedestrians if there is no sidewalk. This challenge highlights all `primary` and `secondary` ways that have no [`sidewalk`](https://wiki.openstreetmap.org/wiki/Key:sidewalk) tagging whatsoever. You can help by looking at aerial imagery and adding the appropriate `sidewalk` tagging. `sidewalk=no` is just as important to have as the 'positive' values. Thanks for helping make OSM better!"

The only items you would need to change are the source_url (use the raw GeoJSON github link you just copied), the slug (use sidewalks-YOURSTATE-YOURCITY or something similar – this will be the challenge URL component in MapRoulette) and the title (change the city name).

By default this configuration will post to dev.maproulette.org. If you want to post tomaproulette.org you would need to get in touch with me to get the credentials.

Once you have the YAML config file in order posting to MapRoulette is as simple as:

$ ./geojson2maproulette.py samples/sidewalks-sarasota.yaml --post --activate
Posting 364 tasks...
server alive: True
Updating challenge...
Reconciling tasks...
Done!

Let me know if you need any help with this or if you want me to create a challenge for you!

Facebooktwittergoogle_plus

Improve OSM Tool Highlight: Cygnus

Improve OSM is all about making OpenStreetMap better with the help of billions of GPS points that users of Scout apps and others contribute automatically. This allows us to identify missing roads, one-way streets and turn restrictions that are not in OSM yet. However, our OSM engineers think about other ways to help make OSM better as well. Looking at what is going on in the OpenStreetMap community, I see a lot of groups and individuals struggle with the challenge of using external data in OpenStreetMap. Many governments now open up data that was closed before. With proper caution and preparation, we may be able to use this data in OSM.

The technical challenge here is to combine the new, external data with what is already present in OSM. In the GIS world this is called conflation. There are some tools out there to help with this task, but they are hard to use. So we started to think about this from a practical point of view and a specific use case: what if you have an external database of just roads, and you want to add only the new roads to OSM?

cygnus-osm-merge

We developed a tool that can do just that, and we called it Cygnus. I wrote about Cygnus on my OSM diary before, announcing it here and then writing about it being used by OSM user MikeN here. We added a link to the Cygnus web site to the ‘Other tools’ section of the ImproveOSM web site.

We made Cygnus as easy to use as possible, but it is still not straightforward. It should not be. Conflation is hard, and what you are doing is importing data into OSM, which should be done with the utmost caution! If you want to get started using Cygnus, please get in touch. We are happy to help!

Facebooktwittergoogle_plus