OpenStreetView is now OpenStreetCam

This summer, we launched OpenStreetView and received great response both from the OpenStreetMap community and the press.

After only 4 months, you have already contributed almost 12 million images covering 322 thousand kilometers. We have released open source apps, upload and OpenStreetMap editing tools, and are working on many improvements aimed at improving OSM faster than is possible now.

As part of our fast growing public profile, we have also attracted the attention of Google Inc, who holds the ‘Street view’ trademark. They are really interested in OpenStreetView but also expressed concerns about the name creating confusion. Obviously to us this confusion does not exist, but after considering the pros and cons carefully, we decided to change the name.

From now on, OpenStreetView will be known as OpenStreetCam. 

osc-logo-web-380_72

Aside from the name, nothing changes. In fact, we will be launching some pretty cool new features and improvements very soon, so please stay tuned for that. If you have not tried OpenStreetCam yet, why not download the free and open apps for Android or iOS, explore the coverage or start editing with OSC in OpenStreetMap?

Happy OpenStreetCamming!

Facebooktwittergoogle_plus

A glimpse into the future of Mapmaking with OSM

We have over the last 12 months starting to look extensively in how we can leverage AI / Deep Learning to help improve OpenStreetMap and today we want to provide a few details about how we envision the future of making maps and also share more on what we are already doing. We see the emergence of self-driving vehicles as a game-changer and one key requirement for those vehicles are accurate and up-to-date maps. Currently commercial map providers map every region around every 12-24 months – in a costly process with a high precision and high cost vehicle, our goal was to achieve maps that are updated on a minutely basis and with key streets covered at least once every day. This is the goal we set out to solve with OSM in supporting to make it ready for this use-case.
Using OSM for Navigation Maps
At Telenav (and before at skobbler) I’ve been actively involved in OSM for almost 10 years now and it is truly unbelievable how OSM has grown massively in that period from a map that was used mostly by passionate enthusiasts to a map that is used by 100s of millions of users and big companies such as Toyota, Tripadvisor or Apple to just name a few to power their consumer products. Despite this success we have still seen that for navigation maps many additional attributes are needed that are not that well covered in OSM such as Signposts, Speed limits, Turn restricitons or Lane Information is needed to provide the best possible guidance.
 space
rsz_speed-na-1

Speed limit coverage

rsz_1tr-coverage
Turn restriction coverage
 space
rsz_tr-na-1
 Turn restriction coverage United States
 space
What we have done especially to close the turn restriction gap is to use (anonymised) GPS probe data from our millions of customers and from partners like Inrix to detect where there are likely turn restrictions based on turn behaviour. This data is then shared with the community via ImproveOSM and also for the most likely cases we put a high penalty on turns for our customers so they avoid those manoeuvres if possible. This way we have been able to detect 139,181 turn restrictions and increased  coverage in a meaningfull way.
Next step: Higher accuracy with Computer Vision
With Speed Limits, Lanes and Sign Posts it is significantly more tricky as it is not possible to identify those purely from GPS probe data. This is the reason why we started our OpenStreetView project to capture those images as there was no truly open project for Streetlevel Imagery that we could use (when we approached Mapillary they asked for hundreds of thousands of dollars in license fees – which was not an option for us).
In parallel to the OpenStreetView projects we have invested a lot in Computer Vision algorithms and established a cooperation with the Technical University in Cluj to get their over 15 years in the field. Our goal was to use computer vision to automatically build maps based on this images.
In the last year we made very significant progress and now we are able to detect Speed Limits, Turn Signs and Signposts (incl. OCR the text in those signs). Those detections when made will be reviewed by our editors and added directly into OSM.
<Slideshow with our computervision images for detecting turn signs, OCR, speed limits>
   input-picture-1
Input picture
space
   panel-detection-2
Panel deetection
space
glyph-segmentation
Glyph segmentation
space
character-grouping-into-words
Character grouping into words
space
ocr
OCR and classification results
space
We have build a map editor that allows us internally to review those changes and add them with our team of 20+ mappers to OSM.
We have by now added 19,798 map features (turn-restrictions,one-ways, signs) to OSM using this tool, and are adding every week hundreds of new turn restrictions and other signs to the map to make it better.
<MAP EDITOR TOOL SCREENSHOTS>
map-editor-1-cropped
 Map editor tool
space
map-editor-2
 Map editor tool
Advanced level: Create High Accuracy maps (ADAS / HD maps)
The next level for this challenge was to create the high accuracy maps needed by self driving cars and for ADAS (Advanced Driver Assistance System) applications. Those maps need accuracy < 2m which typically OSM doesn’t provide consistently and which is a big challenge to achieve purely based on GPS probes as we learned through a lot of trial and error. We looked into how we can achieve better accuracy and our natural choice was to leverage car data that is available to achieve higher accuracy.  Therefore we integrated our OpenStreetView application via an OBD2 port (which is available on every car manufactured in the last ~20 years) to integrate our phone based data with data coming directly from the car (such as speed, or on some models even with steering wheel angle available via OpenXC). With this we have been able to achieve an accuracy which is 5-10x higher than purely achievable by Phone based GPS and with several passes on one road we can create truly high accuracy maps.P ENHANCEMENTS FROM HARALD>
bildschirmfoto-2016-11-21-um-14-02-43
Trip Enhancement
Our vision of the future of map making:
We believe if enough consumers help recording the necessary images via OpenStreetView maps can be created in near real-time at an unprecedented accuracy. This would be a major enabler for self-driving cars and uptodate navigation systems. In order to make that possible we are also in early stages working with several car manufacturers to use the data from their on-board cameras in the future for those detections and hopefully this way we can use millions of cars from our OEM partners in the future with this technology to enhance maps and share this data with the OSM community to create even higher quality maps than today.
We will over the next few weeks go in this blog deeper into our individual modules that we built for making this future happen and looking also forward to feedback from the community.
<TEAMPICTURE OF TELENAV OSM TEAM>
thumb_img_8533_1024
team-pic-2
space
rsz_img_8680
foto_miriam frederic steve-coast
OpenStreetView team
Facebooktwittergoogle_plus

ImproveOSM now based on iD editor

With the help of ImproveOSM, Telenav’s project to analyze billions of GPS points to detect missing roads, one-ways, and turn restrictions, you have already looked at 60,000 missing road tiles, 15,000 one-way suggestions, and 2,000 turn restriction suggestions since the project launched in September 2015.

Today, the Telenav OSM team has released a completely new version of the ImproveOSM web site. ImproveOSM.org is now entirely based upon the OpenStreetMap iD editor. The new ImproveOSM combines the benefits of the familiar, user-friendly iD editing environment with the power of ImproveOSM detections.

The new ImproveOSM web site based on iD
The new ImproveOSM web site based on iD

The new ImproveOSM web site showing missing roads.

Since the new web site is based on iD, it should look very familiar and you should have little trouble getting started with it. The main difference you will see is that the ImproveOSM version of iD has a special panel, which shows ImproveOSM specific options, actions and information. If you have used ImproveOSM before, these will be familiar to you. You can mark items as solved or invalid and apply filters to determine which detections you see.

I do not want to go into too much detail in this post, but I do have a quick power tip: following up on many requests from you, you can now select multiple missing road tiles more easily by pressing shift and selecting one tile. This will automatically select all adjoining tiles within the current view.

Our goal is to integrate the ImproveOSM functionality into the main iD editor over time. To make that happen, your feedback is really important, so please do not hesitate to report bugs and ideas on the project GitHub page, where the source code will also become available soon.

We hope you enjoy the new ImproveOSM web site and look forward to your feedback! Happy mapping!

Facebooktwittergoogle_plus

How to deal with orphan nodes in OSM?

This is a guest post by one of our Map Analyst team interns, Manuela.


While editing the map I stumbled upon clusters of orphan nodes. Basically, orphan nodes have no tags and are not part of a way. One example here:
nodes

Some online tools report these as bugs/issues (e.g. osmose). You should be careful, though. These may have been created with a reason. Before proceeding to deletion, ask yourself:

  • Were these nodes orphan from the very beginning?
  • Is there just one orphan node in a changeset or are there many?
  • Are they arranged in a special way/shape?

If so, they may be GPS traces that could be used for mapping. And still, even nodes from GPS traces should be deleted, there is no reason to keep orphan nodes in the map AFTER you extracted all the needed information from them.

My advice: research! There are many tools to find out the history of an OSM element. To name a few: object history, WHODIDIT, OSM History Viewer, attic data etc.

You’ll probably find yourself in one of the following situations:

  • You’ve found a newbie that creates orphan nodes by mistake
  • The nodes are correct but the user forgot to add the tags
  • A Redaction bot deleted ways without deleting the corresponding nodes (read more here)

Your options:

  • Contact the creator or the used who made the last change via a changeset comment or private message (in a friendly way, of course)
  • Add relevant tags if that’s the case
  • Delete the nodes or leave a fixme tag for other users that may know the area better than you

How to find orphan nodes

Osmose will report orphan nodes clusters as issues:

orphan

In JOSM, orphan nodes (even isolated ones) can be easily found using the Search tool, with the following queries:

type:node tags:0 -child

or

type:node untagged -child

The -child tells JOSM to select only those nodes that are not part of a way.

As if that wasn’t enough already, I’ve created a map paint style that highlights orphan nodes and dims other elements. This will help you analyze the distribution of the orphan nodes without needing to select them and will help you take the decision to delete or no.

This is the default JOSM map paint style, where you can’t really see the orphan nodes that well.
orphan2
This is how the map looks like after applying the new paint style.

If you want to use this map paint style, you can find the script and a step by step guide on the GitHub page:

https://github.com/manuelabutuc/JOSM-Orphan-nodes-map-paint-style/tree/master

Have fun spotting orphan nodes! But remember to delete them only if you are sure they have no reason to be on the map. If you have any improvement suggestions for this map paint style, feel free to comment below or fork the project.

 

Facebooktwittergoogle_plus

HOTOSM recognition by the President of Mexico in Internet Day 2016

On the Internet Day, May 17 2016 the President of Mexico Enrique Peña Nieto invited fifty citizens who called Digital Leaders (#LideresDigitales) to have a dialogue on the future of technology and Internet in Mexico, I had the opportunity of being among these group of citizens. The President talked about various related topics but especially appreciated the efforts of humanitarian mapping conducted by Humanitarian OpenStreetMap in Hurricane Patricia.

Day of Internet 2016- Dialogue with the President of Mexico
día del internet_3
President’s office twitter account

Why the President of México thanked the efforts from HOTSOM in Hurricane Patricia? Here you will find some details so you know what was done.

On October 23, 2015 Hurricane Patricia threatened to touch the states of Colima, Nayarit and Jalisco with winds up to 325 km/h, authorities of Mexico mentioned “It is very likely that this hurricane is the strongest ever in the Pacific side of our country, since it has records ”

Hurricane Patricia Path from NOAA

Contributors of OpenStreetMap and HOTOSM like Rodolfo Wilhelmy, Humberto Yances, Rafael Avila, Robert Banick, Andres Ortiz, and many others (sorry for not mentioning everyone)  in addition to an army of over 500 mappers of Mexico and the world joined efforts to support this area of the Mexican Pacific with data that could be used for the benefit of the population that could be affected. Fortunately, the hurricane lost strength by touching the coast of Mexico causing minimal damage compared to what was expected.

Quick stats:

  • More than 500 contributors mapped 9,000 kms of roads (5.6k miles of road) + 72,000 buildings in 72hrs
  • It was processed 29,608 km^2 pre-event DigitalGlobe imagery to improved coverage over priority areas.
  • It was analyzed INEGI road data to identify missing roads and road names in OSM data.
  • Mexico Open Data was confirmed by authorities to be used in OpenStreetMap.

All these was possible thanks to the great work HOT members, companies supporting OSM project and the local community in Mexico and the World

Contributors mapping the priority area in 72 hours Gif by Mapbox

The event took place in Los Pinos (The equivalence of the U.S. White House) at the moment the Open Data topic was mentioned, Peña Nieto said he knew someone who had supported the alert for Hurricane Patricia was among the guests so I raised my hand to start the dialogue, the President mentioned “… I just want to thank because it’s an example that illustrates very well what we can achieve and I think that you also use open information.” In my participation  I could give my point of view on the need for Mexico not only upload open data to be the first in quantity of released Open Data but emphasize the need of quality Open Data in order to take better decisions based on them. Also I could mention the importance of Open Mapping and collaboration between Governments and Civil Society so more Mexicans are less harmed by disasters (find the video here).

Miriam_DiadelInternet
Fragment of video from President’s office twitter account

In Mexico the OpenStreetMap community is not as numerous as in other countries but in the last two years a group of collaborators we have joined together to promote the project and increase the local community through massive workshops in Universities and courses for Government Authorities and Civil Society. Much remains to be mapped but I believe we are on the right track.

Miriam Gonzalez

@mapanauta

Facebooktwittergoogle_plus

Happy Mapping Hour – Presentation Import Project INEGI MGN (National Geostatistical Framework)

Last April 6th 100% of the Mexico Telenav’s team (Andrés Ortiz 50% and Miriam Gonzalez 50% 😀 ) presented the results of Import Project INEGI National Geostatistical Framework. The meeting point was the Felina bar on the edge of Condesa and Escandon neighborhood.

Andres_presentando
Andres presenting at Happy Mapping Hour
Miriam_presenta
Miriam presenting in Happy Mapping Hour                                                         Image by @Tlacoyodefrijol

More than 20 people booked and came to the appointment. The project was originally announced in May 2015 with much skepticism because this was the first time a project of such magnitude was taking place in Mexico and the OpenStreetMap community in Mexico at that time was very disperse.

Before_import
Before the Import Project there were 69 valid boundaries                               Image by Ruben @Mapbox

Many import projects have been conducted in many parts of the world, such projects have helped (mostly) to create the map of the world that we have today and Mexico was going to be part of them. People with extensive knowledge in imports formed part of the project including Victor Ramirez, Ernesto Carreras, contributors od OpenStreetMap Puerto Rico and Rafael Avila, a HOTOSM collaborator and expert in African countries imports. At the beginning of the project we realized that there were only 69 valid administrative boundaries (although in the image it looks more than 69,  these lacked the tag SOURCE which made them invalids) and the end of the Import project the team had added 2,457 administrative boundaries with tag Source = INEGI MGN 2014 v6.2

After_Import
After the Import Project there were 2,457 administrative boundaries        Image by Ruben @Mapbox

To the #HappyMappingHour diverse OSM contributors atended such as geographers, developers, archeologists and also Armando Aguiar – INEGI IT Services Director  witnessed how the Open Data Inegi released at the end of 2014 has been in benefit of OpenStreetMap. Let me share some statistics:

Quick Statistics:

Node numbers/Ways/ Deleted relations

  • 500K / 2k / 500

Node numbers/ Ways / Added relations

  • 1000K / ~4k / ~1050

Number of hours dedicated :

  • 250+

NUmber of administrative boundaries added:

  • 2,457

 

Now that the map has de MGN boundaries as a reference mappers as Irk_Ley have been investigating the local laws of the states of Veracruz and have been reviewing historical maps of the Map Library Manuel Orozco. These mappers will be verifying and correcting those limits which have differences with the MGN when they have the backup of the documentation of the local law.

Ancient map of the Papatla, Veracruz region

Here you will find the presentation of #HappyMappingHour and if you want more technical details we suggest you check the following blogs and the wiki.

Here also you will find two Blogs from collaborators in the Import Project:

Blog: My experience in OSM during the MGN Import by Pablo Garcia (OSM user: Irk Ley)

Blog: Import of INEGI Mexico municipalities finished by Andres Ortiz (OSM user: Andresuco)

You can contact them directly if you have any questions or comments for them.

What are the next challenges?

Evaluate data from the National Road Network and create a joint project with Mexico OpenStreetMap community to carry out its import. It is also in the radar create a tool where information from OpenStreetMap in Mexico is a kind of “inspector” to send feedback to INEGI about possible shortcomings or errors can be corrected and improved thanks to contributors OpenStreetMap but first we need more discussions with the local community.

Facebooktwittergoogle_plus

Updates to ImproveOSM JOSM plugin for better usability

The team has been working on some nice updates to the ImproveOSM JOSM plugin. I have been taking the new version for a spin and wanted to report back.

In case you need a refresher: ImproveOSM is a suite of tools (currently a web site and a JOSM plugin) that takes the results of a massive data analysis comparing billions of GPS data points with existing OSM data and displays them in a way that makes it easy for any mapper to improve OSM with missing roads, turn restrictions, and one-way tags.

Missing Roads (red), One-ways (orange) and Turn Restrictions (blue) in the clustered view of the ImproveOSM JOSM plugin.
Missing Roads (red), One-ways (orange) and Turn Restrictions (blue) in the clustered view of the ImproveOSM JOSM plugin. This is the Dallas, Texas area. Imagery from Bing.

The improvements are fairly small but gave me a noticeably nicer workflow, so I thought it would be worth sharing.

The first improvement is that you can now right-click on any of the ImproveOSM layers in the layer panel to access the data filtering options for that layer.

Access the data filters using a right-click on the layer panel
Access the data filters using a right-click on the layer panel.

The data filters let you see part of the data for that layer based on various criteria, such as number of trips, confidence level, status and others. The criteria available vary by layer. Here is the filter window for Missing Roads, for example:

Filter window for missing roads

The filters themselves are not new, but you needed to go to the ImproveOSM panel to access them before. I think this is way quicker.

Another thing I really like is the improved visualization for the turn restrictions. The team made it much easier to see the from-via-to flow of the suggested restriction. The from-segment is now green and the to-segment is red. When selected, the info panel will also display more useful information than before:

The new visualization of the missing turn restriction. The 'from' segment is green, the 'to' segment is red.
The new visualization of the missing turn restriction. The ‘from’ segment is green, the ‘to’ segment is red.
The metadata we display for a turn restriction is now more relevant.
The metadata we display for a turn restriction is now more relevant.

The detailed info panel was improved for the other categories (missing roads and one-ways) as well.

Finally, when you are done mapping an ImproveOSM thing, you can now quickly mark the thing as invalid or solved, without having to enter a comment. We realized that this was not a very efficient workflow. You can still add a comment upon closing the issue, but now it’s easy to do it without, by right-clicking on the ‘solve’ or ‘invalidate’ buttons and selecting the appropriate action.

invalidate-context

These small but meaningful improvements made my work with ImproveOSM in JOSM much more efficient. We are always looking for more ways to make ImproveOSM better. If you have used ImproveOSM and you have a few minutes to spare, I would appreciate it if you filled out this survey. Thanks a lot!

Facebooktwittergoogle_plus

Using PostGIS to answer geodata questions

One of the biggest challenges when working with large sets of data is to find the least costly workflow that you have to follow in order to get the most accurate answers.

Let’s say you have a huge dataset composed of all sorts of geometry features (points, lines, areas etc.) and you want to do a bit of cleaning – because messy and redundant information is no fun!

So you might be thinking “Hmmm… which are the areas that have an unnecessary high density of points?”

The same issue can arise when working with OpenStreetMap data. This can be easily solved using PostGIS and a command line tool that we’ve created and using.

Note: The following steps require a Linux environment, Postgresql 9.x, PostGIS 2.x, Osmosis 0.43+, QGIS 2.12.2+

Getting the data

Download an *.osm.pbf file using command line:

 wget https://s3.amazonaws.com/metro-extracts.mapzen.com/san-francisco_california.osm.pbf

This is the metro extract for San Francisco, provided by Mapzen. Geofabrik is also a very good resource for OSM data extracts.

In the same folder, download SCOPE – databaSe Creator Osmosis Postgis loadEr.

wget https://github.com/baditaflorin/osm-postgis-scripts/blob/master/scope.sh

Make sure to set the file to be executable by using

chmod +x scope.sh

Load the data

Using SCOPE and following the instructions on the screen, load the *.osm.pbf into a database.

SCOPE automatically creates the database with hstore and postgis extensions and the pgsnapshot schema.

Play with the data

Now that you have the data set up, you can easily query it using the DB Manager from QGIS and some PostGIS scripts.

Interesting examples

For example, using the find_duplicate_nodes query, we can see that this building (@20.805088941495338, -104.92877615339032), appears on the same spot 23 times!

duplicate_building

The one next to it (@20.8054225, -104.9278152) appears 22 times!

duplicate_building_2

The node density for these areas (@20.4411867, -97.3172739) is too high – 168 nodes!

nodes1

Also, 171 nodes for a small fence segment (@46.7487683, 23.559687)!

fence

the-node-density

Feel free to fork the GitHub repository and modify the code to suit your needs! Also, if you feel insipred, you can suggest a better and shorter name or acronym for SCOPE!

Facebooktwittergoogle_plus

OSM Mapping party – spring edition

 

On the 17th of April we had our first Mapping party event for this year. Our main focus was to improve the map of our hometown by reflecting the latest changes.                                                                                                   Cluj-Napoca is a dynamic city, many new buildings was constructed; POIs, turn restrictions, addresses have been changed and appeared since the last field mapping.

Around 30 map enthusiast show up in Sunday morning for the Mapping party. There were both experienced mappers and newbies present at the event. The event had started with a morning coffee and some instructions regarding data collections.

For data collections we used the following tools:
• Field papers: our colleague Florin Badita had took some time before the event and had created field papers for several city areas

field-papper1

• GPS tracker applications: OSMTracker, OSMAnd, Pushpin OSM and so an
OpenStreetView application 

We have divided the people into smaller groups of 2-3 persons. After each group had chosen an area to map we went out to collect the data.
On the afternoon we headed back to our meeting location to add the collected data into OpenStreetMap.

An outcome overview of our mapping effort is presented on the following images:

FinalEdits

FinalEditsOverview

Facebooktwittergoogle_plus

Improve OSM adds missing roads in Guatemala

In a new data release today, we added about 500 tiles worth of missing roads in and around Guatemala!

Missing roads near Coatepeque, Guatemala
Missing roads near Coatepeque, Guatemala in JOSM. Imagery from Bing.

We are excited to be adding more and more Missing Roads data to ImproveOSM using GPS data from our own users as well as from data partners, like we did in Brazil and in this case.

You will notice that the tiles look a little different from the ones you are used to if you have used ImproveOSM before: they don’t show the individual points. This is because this particular data was processed a little differently. If you use JOSM, you will also see an update to the ImproveOSM plugin to accommodate this change.

While you are looking at the new Missing Roads, perhaps you will also notice some other recent improvements to the ImproveOSM web site. We re-ran all tiles based on new map data from mid-April, and we improved our turn restriction detection so we won’t show a missing turn restriction when OSM already has a ‘only straight on’ restriction.

Happy Mapping!

Facebooktwittergoogle_plus