Improving OSM in Canada one day at a time

Ever since we started our mapping project in Canada, nearly 8 months ago, we’ve been continuously working on bringing the OSM data to the level where all elements needed for routing get as detailed as possible.

Whether we are talking about the basics of road networks such as geometry, naming or traffic flow direction, to in-depth details like number of lanes, turn lanes, turn restrictions, signposts and even complex relations referring to highways, we edit everything.

Our main focus is oriented towards the Top 5 metro areas: Toronto, Montreal, Ottawa, Vancouver, Calgary. These are the places where we spent the most of our time researching for open data, adding new features, editing existing ones. In order to make sure that the overall state of OSM throughout the entire region of Canada is in navigable ready state, we’ve also included the first 50 cities based on population.

So, let’s see some numbers and graphs because everybody likes those. If we start looking at the numbers for the entire region we can see a significant rise in road geometry that was added, around 3% (25,330 miles) out of the total numbers of miles. The same goes for roads that previously did not have name tags with a rise of little over 3.5% (16,799 miles).

A more significant change can be noticed for features that weren’t extensively mapped before in the area, such as turn restrictions rising from 5254 to 54891, or signposts which hadn’t been mapped under the same standardized method. With the help of OpenStreetCam and Mapillary pictures, we’ve managed to add relevant signpost information increasing the number of nodes well over 68%.

If we break down the numbers for the Top 5 areas, the most noticeable changes can be observed for both Toronto and Montreal where oneway tags and signpost information have been improved.

One of our main goals is to focus not only on quantity but especially on quality. This is why we have multiple tools for integrity checking that are ran periodically on the entire region of Canada. These tools cover a wide variety of cases that are being corrected weekly, such as: road name flip-flops, unconnected ways, smoothness problems, misnamed road, road names having their suffixes or prefixes abbreviated and many more.

We make use of different QA tools (KeepRight/Osmose) to search and track issues in OSM that have either been added by mistake or have remained unedited after large imports. We’re also on the look out to improve way accuracy and fix alignment issues.

An overview of our edits.

Below you can see some examples of our improvements.

Road geometry updates.
Road geometry alignment.
Missing geometry and minor refinements.
Turning loops updates.
Facebooktwittergoogle_plus

Mapping traffic signals and stop signs using MapRoulette

In our journey of improving  the OpenStreetMap we are constantly searching for  open source data. This search is very important and is done before we start improving the map in a new area.

Currently, part of our team is focused on improving the Detroit area. So, before we started mapping we searched for useful geospatial data and we came across open data about traffic signals and stop signs for Wayne County, Detroit. The data can be found here and here.

Traffic signals mapped in OSM
Stop signs mapped in OSM

We filtered out the traffic signals and stop signs that were already in OSM but there is still a significant amount of data that can be added in OSM. (912 – traffic signals and 8755 – stop signs). Due to this, we thought about creating a MapRoulette challenge.

About MapRoulette

MapRoulette is a micro-tasking tool used to fix bugs in OpenStreetMap and to improve it. A user can create tasks by uploading files which contain the location, ways, points with the error that has to be fixed or files with features that are missing from the map and can be added by other users.

When creating a new task, the user gives specific instructions on what steps have to be followed to edit through this tool. Once a user has logged in, he can see on the map the created challenge and the pins which consists of tasks he can solve.

So, given the available data that we found, we created two challenges – one for traffic signals and the other for stop signs. Some general rules for mapping traffic signals and stop signs can be found on the OSM wiki – here and here.

Tags that we use for mapping
  • Stop signs – highway=stop
  • Traffic signals – highway=traffic_signal
Notes
  • If the traffic signal/stop sign is referring to all the highways entering the intersection, we add the traffic signal/stop sign in the intersection point.

  • If the traffic signal/stop sign is not referring to all the highways entering the intersection we add the traffic signal/stop sign before the intersection, where the sign/signal is positioned.
  • We need to add an additional tag if the road is bidirectional:
    • for traffic signals we use the traffic_signals:direction key with the forward or backward values to indicate the affected direction.

    • for stop signs add direction=forward or direction=backward to indicate the affected direction.

The data has been published under Public Domain license.

Everyone who is keen on mapping is welcomed to help us.

Let’s improve OSM together!

Facebooktwittergoogle_plus

OpenStreetCam JOSM plugin – new features

Last week we had released a new version of the OpenStreetCam JOSM plugin. While we are continuously working on improving and fixing existing functionality, we also keep adding new and exciting features.

Map view improvements

This new release introduces a major improvement to the map view. For small zoom levels, we had adopted a similar visualization as in the case of the web and mobile OpenStreetCam applications. Instead of displaying individual photo locations we display ways that have OpenStreetCam data coverage.Segments are colored with purple and have different transparency based on the data coverage: segments that have many images are opaque while the segments that have only a few images are more transparent.

By changing the initial MapView visualization we were able to display OpenStreetCam data starting with zoom level 10. This way we can indicate areas that have street view coverage at a country view level and possible gave a hint to the user where he/she can find an extra source of mapping support.

Starting with zoom level 18 the map view changes and individual photo locations are displayed similarly as in the previous versions of the plugin.

The displayed data type is user configurable and can be changed from the OpenStreetCam plugin preference settings. You can access the preference settings from JOSM ->Preferences -> OpenStreetCam plugin -> MapView settings or from the OpenStreetCam panel by clicking on the preference icon. 

From the MapView settings section, you can change the minimum zoom level at which image locations are displayed, along with the data type change method. By default, the MapView data type is changed automatically.                                                                                               When the “switch manually between segment and image view” option is enabled a new button is visible in the OpenStreetCam panel.

The “data switch” button is enabled starting from zoom 16 and is represented with different icons based on the displayed data type. For segment map view a photo icon is displayed while for image location view a segment icon.

If you click on the button the map view changes from segment view to image location view and vice-versa.  The type of data can be changed manually for zoom levels bigger than 16.

 

Layer and panel improvements

The OpenStreetCam layer and panel default visibility had been improved and previous open/closed states are remembered for future JOSM sessions. After installing the plugin in order to see the OpenStreetCam data you need to open manually the layer and panel. The layer can be opened from Imagery ->OpenStreetCam menu, while the panel from the left side JOSM menu.

We had changed the OpenStreetCam window button panel actions and removed the actions that were not related to the currently selected image. Feedback and filter actions were added to the OpenStreetCam layer menu:

In case you need a refresher:  OpenStreetCam data can be filtered based on date and currently logged in OSM user. Basically you can visualize images that were uploaded after the specified date. You can also visualize only your contributed data.

Nearby photos

An important feature that we had added to the plugin is the nearby photos functionality. This functionality improves the mapping process especially if the selected photo does not contain all the information or if the selected photo has bad quality or has not the right angle.

A nearby photo of a selected photo can be visualized either by clicking on the “Nearby photo” icon or by pressing ALT+N keys. 

If the “Load track on image selection” preference settings option is selected , than also the track corresponding to the nearby photo is loaded.

Nearby photos are computed based on the currently visible photos, if the user moves the map or zooms in the set of nearby photos is recomputed.

A photo is considered nearby if belongs to a different track and it is located to maximum distance from the selected photo.

Photo load on mouse hover

Another important feature that we had added to the latest release allows users to quickly load photos on mouse hover action.  By default this feature is disabled and can be activated from JOSM ->Preferences -> OpenStreetCam plugin -> Image settings.

If this feature is activated, than the small thumbnail image is loaded in the OpenStreetCam panel and remains loaded only it is explicitly un-selected from the map.

A better resolution image is loaded if you click on the image location icon or if the OpenStreetCam panel is maximized.

Upcoming features

The JOSM plugin is work on progress, we are working on improving the usability and plan to add new features from time to time.

We hope that you enjoy the new features! If you have ideas, suggestions or encounter any issue with the plugin during editing sessions please submit either to the GitHub issue page or to the Feedback forum .

Have fun improving the map by using OpenStreetCam images!

Facebooktwittergoogle_plus

3D Scene reconstruction

From 2D images we can extract a limited range of information like width, height and color. These can be useful to determine the regions of interest in our images: street signs, lanes, or even roads.

However, for a more accurate detection, the depth perception
is crucial. Here comes the 3D reconstruction into play. Extracting a 3rd dimension, the depth, we can determine how far from the
camera the regions of interest are and consequently, their shape. This way we can distinguish the road from the obstacles (cars, pedestrians, curbstones) simply because we know that the road has an increasing distance from the camera while the objects have a constant distance (fig. 1).

fig. 1 Depth Map

Continue reading “3D Scene reconstruction”

Facebooktwittergoogle_plus

More and Updated Data for ImproveOSM

ImproveOSM has been updated with many new roads. We processed recent  GPS data from a number of data partners with some great results. A total of 30,000 new missing road tiles were added, over 17000 in Indonesia alone.

Aside from the missing roads, we added 67000 potential missing one-way roads that we detected with high confidence. Internal testing revealed only 6% false positives.

We are happy to continue providing OSM mappers with high quality data about missing things in OSM based on billions of GPS traces. Because ImproveOSM is based on actual drives from people using navigation or mapping software in their vehicles, and we apply a pretty high threshold for number of trips and quality of the GPS data, you can be pretty confident that every ImproveOSM feature will lead you to something you can add to OSM. Even if the aerial imagery is poor.

You should see the new data in your ImproveOSM plugin or on the ImproveOSM web site very shortly. Happy mapping and let us know what you mapped using ImproveOSM!

Facebooktwittergoogle_plus

New Version Of OpenStreetCam Introduces Points

Late last week, we released new versions of the OpenStreetCam apps and the web site. While we continue to make the platform faster and more reliable, we also like to keep adding interesting and fun features from time to time! This new release introduces points and levels. Every time you drive, you earn points. Earn enough points and you level up.

We went back in and calculated points for all your existing trips, so why not head to the newly designed leaderboard and see how you stack up against your fellow cammers? You can also see the leaderboard in the app:

We also enabled leaderboards by country on top of the daily, weekly and monthly rankings.

Your profile screen in the app and on the site will show you exactly how many points you have, how many you earned per trip, and what your current level is.

The new profile screen

More points for unexplored roads

So as you are driving around, you will automatically earn points for every picture recorded. But not all pictures earn you equal points! The less explored a road is, the more points you get — up to 10x the points for roads that have no coverage at all yet!

(This made it possible for me to gain 11k points on a 50 minute drive last week: most of the roads had no coverage yet, so I was getting 10x points for most of the way. )

11k points!

You can see which roads are less covered, or not covered at all yet, in the app. Just look for the roads with lighter or no purple OSC overlay:

Darker streets have better coverage, lighter streets need more.

We calculate the quality of coverage by the number of trips that cover the way as well as the age of the existing trips. This way we encourage each other to always have the most recent imagery available for OpenStreetMap.

We hope you enjoy the new features! Please let us know what you think by writing us at hello@openstreetcam.org.

Facebooktwittergoogle_plus

OSMTime in Cluj featuring MapRoulette

OSMTime is a monthly OSM mapping event organized by Telenav colleague Beata Jancso. Telenav hosts the events in the Cluj-Napoca office  and sponsors with pizza. Usually Bea chooses a theme and sometimes there will also be a speaker with an interesting OSM related topic.

While visiting the Telenav Romania office in Cluj last week, I was lucky to also catch an OSMTime event. The theme of the evening was ‘Mapping Roundabouts using MapRoulette’. Being the person behind MapRoulette, Bea asked me to do a quick introduction. Colleague Bogdan Gliga also presented the metodology he used to detect missing roundabouts from massive amounts of probe data. (He wrote about that topic here as well.)

OSMTime Cluj with Bogdan presenting
OSMTime Cluj with Bogdan presenting

After the presentations and pizza, the 25 or so mappers logged on to MapRoulette to start with the new Missing Roundabouts challenge. Most people had not used MapRoulette before, so I was glad that everyone was getting the hang of it quickly. Most of the problems and questions were not about MapRoulette but about what is a roundabout exactly, and what is the difference between a roundabout and a mini_roundabout and a traffic_circle. (The OSM wiki helps out a little here.)

At the end of the evening, the mappers in the room already made a good dent in the challenge, which has more than 4500 tasks total.

I had a great time, thanks to Bea for organizing the OSMTime events every month and spreading the word. If you are in the Cluj-Napoca area, you may want to subscribe to the OSMTime meetup so you know when the next one takes place. Or look for an OpenStreetMap meetup in your area and meet local mappers!

Facebooktwittergoogle_plus

OpenStreetCam JOSM plugin

The OpenStreetCam JOSM plugin helps the community to improve the map by displaying up to date street view images. Street view images are collected by the OpenStreetCam platform and are available also via the OpenStreetCam web application and map editor.  

Having an extra source of free and open imagery ease the process of remote mapping and allows the users to reflect the reality also in the map. Street view images are helpful for editing map features that are not visible on satellite imagery like traffic signs, house numbers, bus stops, points of interests.

Installation

Install the OpenStreetCam plugin the familiar way, through the JOSM plugin Preferences menu item. After you install the plugin and restart JOSM, you should see the OpenStreetCam layer and panel.

OpenStreetCam layer

After a successful installation the OpenStreetCam layer is available in the layer menu panel and on the main map the image locations are displayed. Image locations are illustrated with blue icons, each icon indicating the image heading.

An image location can be selected by single mouse click action as long as the layer is visible. You can select images even if the OpenStreetCam layer is not the active layer.

OpenStreetCam layer displays data starting with zoom level 14, so in order to see the data you need to zoom in into the desired mapping area.

For Imagery layers the data is loaded as you move the map and zoom in/out. In the case of OSM data the OpenStreetCam layer data is loaded only for the downloaded area.

The plugin saves the open/closed state of the layer. So if you delete the layer then at the next JOSM session the OpenStreetCam layer will not be loaded by default. A previously deleted OpenStreetCam layer can be activated again from the Imagery menu.

OpenStreetCam panel
In the OpenStreetCam panel you can interact with the currently selected image.

The panel along with the image displays basic information such as: OSM username and date of creation.

The panel also has a number of action buttons on the bottom. These are for filtering, next/previous image loading, centering the map, opening image web page and giving feedback. Image related actions are enabled only when the image is showing in the panel.

These features will be discussed in the next sections.

The plugin saves the open/closed state of the panel. So if you delete the panel then at the next JOSM session the OpenStreetCam panel will not be opened by default. If you don’t see the panel you should be able to open it by selecting the OpenStreetCam icon from the left side panel.

Image filtering

The displayed data can be filtered based on the creation time and JOSM user. In order to view only your uploaded images, you need to authenticate in JOSM using OAuth login.

 

By default no filter is set, custom filters can be removed by clicking the Clear button.

Visualizing an image and corresponding track
Individual images can be visualized by clicking on the image icon displayed on the map. The corresponding image is loaded in the OpenStreetCam panel and the corresponding track is displayed on the map.

An OpenStreetCam track is illustrated with a blue directed line. Images belonging to the selected track are illustrated with opaque icons; while other images along the track are illustrated with transparent icons.

Image zoom in/out

The displayed image can be zoomed in and out using the mouse wheel. In an already zoomed in image details can be observed by moving the image left, right, up and down.

Next/Previous image

You can navigate between the previous and next image of a track either from the OpenStreetCam panel by clicking on the Next/Previous button or by pressing Alt-Left arrow/Alt-Right arrow.

If the next or previous image is not visible in the current view, the map is moved automatically and images near the track are downloaded.

Center map to selected image

The map can be re-centered to the selected image location by clicking on the “Location” button from the OpenStreetCam panel. This feature is useful when the map was moved and the selected image location is not visible on the map.

Image web page

The selected image web page can be opened by clicking on the “Globe” button from the OpenStreetCam panel.

Upcoming features

We are working on improving our JOSM plugin and plan to add new exciting features. In the near future we plan to:

  • improve image loading speed by adding caching mechanism
  • allow the user to select easily nearby images to an already selected image
  • improve the map view and suggest street view coverage by displaying OSM ways instead of individual images. We will implement something similar as in the case of the web and mobile applications.

Source code

The source code for the plugin can be found on GitHub .

 

Feedback

Ideas, suggestions and bug reports can be submitted either to the plugin’s GitHub issue page or to the Feedback forum. Other mapper’s idea can be voted there.

We take a look at all incoming ideas, so be sure your input is heard and very much appreciated!

 

Have fun adding missing map features using OpenStreetCam images.

Facebooktwittergoogle_plus

Enhancing OSM Maps using Machine Learning & Big Data

One of our main goals here at Telenav is to constantly improve the maps we are using in our applications and services Having very detailed and accurate maps is of fundamental importance if we want to build high-quality and precise routing applications, ADAS systems or self-driving guidance software. In this post we’re going to talk about how we leveraged our massive datasets of anonymized GPS (probe) data in order to enhance the OSM maps, more specifically how we were able to detect missing roundabouts throughout the world.

The Task

The problem at hand is as follows: Given a dataset of GPS probe data and the current OSM geometries, could we identify missing roundabouts? More precisely, we are searching for geometries that lack a specific tag ( junction = roundabout) identifying them as such.

A relevant case would be the one below, where the geometry clearly defines a roundabout, but that specific tag is missing from the OSM map.

1484816109 1484816464

What we decided to do is to analyze car movement patterns and use this analysis to make inferences about the underlying map topology. In order to solve this not-so-trivial problem we decided to harness the power of Machine Learning. We did this because we are aware of the huge recent developments in this field and of the powers of a well-designed Machine Learning algorithm when combined with huge datasets. 

The Solution

The intuition about why this approach is preferable is obvious when analyzing the available data and how different traffic patterns are when we are in the context of a roundabout compared to the context of a normal intersection. What we have achieved is to teach the algorithm to associate the circular traffic movements having a “hole” in the middle with a roundabout and to associate the evenly spread movements with a normal intersection.

1484816471    1484816475

After successfully developing this “smart” detection algorithm, we have selected from the world map approximately 117 000 potential points, where a roundabout would be likely to exist based on some predetermined criteria. Of course, these are far too many to manually check, so the automated solution is the only one suitable for this job.

The Results

After running those points through the Machine Learning algorithm, it has detected around 9000 missing roundabouts in Europe and North America, as those are the areas for which we have GPS probe data. The massive size of these results which translate to substantial improvements of the OSM map is obvious when visualizing them.

1484816482

1484816492

1484816499

After a quick series of manual testing of a batch of results, we have discovered that the predictions are between 82% – 86% correct, depending on the level of confidence, which proves the efficiency of the Machine Learning oriented solution to this difficult task.

What’s next

In the near future, we plan to release this data to be validated by the OSM community through the MapRoulette platform and we are eager to see the feedback we get. Having acquired even more knowledge in this field, we are now ready to tackle more difficult problems using more advanced Machine Learning and Deep Learning algorithms. This will surely enable us to improve the OSM maps even more.

PS

We have uploaded all our predictions here in CSV format for those of you who are interested in playing with the data.

 

 

Facebooktwittergoogle_plus

Usage tips on the new Improve OSM

In this post we will take a detailed look at the new improveosm.org web site. We recently completely overhauled the application. It is now based on the OSM iD editor. We will walk you through the functionality and give some pointers.

Zoom and layer activation

Just by entering the web application, you already have our Improve OSM layer selected, and also active. A low zoom level displays a heat-map the content of which is modifiable via the left side filtering options. Note the color coded dots, corresponding to the heat-map’s circles.

improve-osm-heatmap

A higher zoom level, meaning a zoom level over 15, will show each individual Improve OSM item. Turn Restrictions are grouped in clusters, if some items represent rules for the exact same intersection. You’ll also notice that different types of Missing Roads have their specific color on the map and are also marked with the respective color in the filter panel. Filtering has the same usage as in the heat-map.improve-osm-transparent-tr

We talked about the Improve OSM layer being active by default, but what exactly does that mean? It means you can use the filters from the side panel and you can also interact with any Improve OSM item on the map. While inactive, the filter panel becomes minimized and any Improve OSM item becomes see-through. You can’t interact with the layer in this inactive state, but you can and will want to interact with all the iD’s map items and editing mechanisms. The SPACE key switches between the active and the inactive states. You can also use the toggle button found in the side panel’s header.

Item selection and status

When the Improve OSM layer is active any item can be selected. After selection, options appear in the side panel allowing you to change the status and add a comment.

You can have a multiple selection by keeping the CTRL key pressed and selecting items with the mouse. Only for the Missing Road items, you can batch select neighboring tiles. For this, keep the SHIFT key pressed and click one of the tiles in the batch. All neighboring tiles in that batch will be selected.

improve-osm-selections-1

improve-osm-selections-2

For Turn Restriction clusters, selecting one will automatically select a single Turn Restriction from the cluster. You can switch to another item in that cluster by selecting it from the list appearing in the side panel.

improve-osm-selections-3

Clicking anywhere on the map area not containing an Improve OSM item, will deselect all selected items.

For Turn Restrictions, you can select one to see its describing arrows and pass numbers, and then have the entire Improve OSM layer inactivated. This way, you’ll still see the describing arrows while being able to interact with the editing iD tools.

Item types

Turn Restrictions – they mark an intersection where a new, unmapped, turn restriction is in place. The green arrow marks the street vehicles came from while entering an intersection, while the red one marks a street used by none or very few of those vehicles to exit the intersection. It’s assumed that the very low percent of passes on the red segment, indicates a strong possibility of a turn restriction being in place. Passes on the red segment are assumed to be traffic rules violations.

Missing Roads – they mark a portion of the map (a tile) that contains GPS output from vehicles, in places no road is mapped. If they are numerous and they represent a clear trail, it’s assumed a missing road is there.

One Ways – they mark a road that is not mapped as a one way street, for which traffic data suggests with high confidence it actually is a one way street.

A typical usage flow

1. Identify an Improve OSM marker (item) and decide if iD editing is required

improve-osm-flow-set-1

2. Press SPACE to inactivate the Improve OSM layer and use the iD’s tools

improve-osm-flow-set-2

3. Make the wanted edit in iD

improve-osm-flow-set-3

4. Press SPACE agai, select the resolved Improve OSM item and change the item’s status to ‘SOLVED’

improve-osm-flow-set-4

We hope you find the new improveosm.org web site useful and are looking forward to hearing your feedback!

Facebooktwittergoogle_plus