# feeds

key: derick_rethans
encoding: utf8
format: rss 2.0
title: Derrick Rethans
url: https://derickrethans.nl/
feed_url: http://derickrethans.nl/feed-openstreetmap.xml
summary: This feed shows the latest 15 items with the tag openstreetmap
generator: eZ Components Feed dev (http://ezcomponents.org/docs/tutorials/Feed)
updated: 2019-04-12 15:34:21
published: 2019-04-12 15:34:21
twitter: derickr
items_last_updated: 2015-08-05 08:25:00
fetched: 2019-07-20 20:30:22.386321
http_server: lighttpd/1.4.52
md5: dd92ed8b24c50c54d146e5ff23d85f8b
created_at: 2019-06-09 18:11:33.483000
updated_at: 2019-07-20 20:30:22.387470
body:
  <?xml version="1.0" encoding="utf-8"?>
  <rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd">
    <channel>
      <title>Derick Rethans - tag: openstreetmap</title>
      <link>https://derickrethans.nl/feed-openstreetmap.xml</link>
      <description>This feed shows the latest 15 items with the tag openstreetmap</description>
      <language>en-us</language>
      <copyright>All rights reserved - Derick Rethans</copyright>
      <managingEditor>phpinternalsnews@derickrethans.nl (Derick Rethans)</managingEditor>
      <pubDate>Fri, 12 Apr 2019 15:34:21 +0000</pubDate>
      <lastBuildDate>Fri, 12 Apr 2019 15:34:21 +0000</lastBuildDate>
      <generator>eZ Components Feed dev (http://ezcomponents.org/docs/tutorials/Feed)</generator>
      <docs>http://www.rssboard.org/rss-specification</docs>
      <ttl>60</ttl>
      <image>
        <url>https://derickrethans.nl/images/phpinternalsnews.jpg</url>
        <title>Derick Rethans</title>
        <link>https://derickrethans.nl/</link>
      </image>
      <itunes:author>Derick Rethans</itunes:author>
      <itunes:owner>
        <itunes:email>phpinternalsnews@derickrethans.nl</itunes:email>
        <itunes:name>Derick Rethans</itunes:name>
      </itunes:owner>
      <item>
        <title>Terracing Buildings</title>
        <link>https://derickrethans.nl/buildings.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="terracing_buildings"/&gt;Terracing Buildings&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Wednesday, August 5th 2015, 09:25 BST&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;Following up from the &lt;a href="https://derickrethans.nl/flats.html"&gt;previous&lt;/a&gt; article on addressing flats, in this article we deal with drawing buildings. The new area has lots of similarly looking buildings, and they're not exactly square. They do however all (broadly) share the same shape:&lt;/p&gt;
        &lt;img src="derickrethans.nl/rst/images/buildings.png"/&gt;
        &lt;p&gt;&lt;a href="http://josm.openstreetmap.de"&gt;JOSM&lt;/a&gt; has a &lt;a href="http://wiki.openstreetmap.org/wiki/JOSM/Plugins/Terracer"&gt;terracer&lt;/a&gt; plugin, which allows you to draw a rectangle, and then quickly convert it to a row of terraced houses and at the same time add house numbers to the generated houses.&lt;/p&gt;
        &lt;p&gt;You do the following steps with the plugin:&lt;/p&gt;
        &lt;table&gt;
          &lt;thead&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;p&gt;Image                            &lt;/p&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Action&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
          &lt;/thead&gt;
          &lt;tbody&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace01.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Find a spot that needs buildings added&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace02.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Draw the outline&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace03.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Add the &lt;code&gt;building=residential&lt;/code&gt; tag&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace04.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Select road and outline, and press &lt;code&gt;Shift-T&lt;/code&gt; to run the terracer plugin.&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace05.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Select every other middle node at the back&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace06.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Use JOSM's Way Accuracy Tool &lt;code&gt;w&lt;/code&gt; to add in the other nodes for the back&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace07.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Repeat for the other 5 buildings&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace08.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Select all buildings and use the Orthogonalise Shape tool &lt;code&gt;Q&lt;/code&gt; to "square" the buildings.&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
          &lt;/tbody&gt;
        &lt;/table&gt;
        &lt;p&gt;Having to do step 6 and 7 for every building, especially when there are a lot of them, is a very tedious, and inaccurate process. It is sometimes even faster to draw all the building outlines by hand, select them all, and orthogonalise them.&lt;/p&gt;
        &lt;p&gt;But the terracer does already save a lot of work, so what if I could make it better and make it support the type of buildings that are common in my new neighbourhood?&lt;/p&gt;
        &lt;p&gt;I had a look into the plugin, and it seems that I have to use Java to update it. I don't really use Java, and hardly ever written any code in it. It also sadly lives on some SVN repository, so pull requests against it are difficult too. Instead, I pushed my version of the repository to &lt;a href="https://github.com/derickr/josm-plugin-uberterrace"&gt;https://github.com/derickr/josm-plugin-uberterrace&lt;/a&gt; — and called my version Überterrace. Oh, and I learned some Java.&lt;/p&gt;
        &lt;p&gt;After a while of figuring out some ... syntax issues, and figuring out how to build this stuff, I managed to improve the plugin with another option. The new option is "fancy outline" and automatically draws in the "extensions at the back". The new work flow is now:&lt;/p&gt;
        &lt;table&gt;
          &lt;thead&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;p&gt;Image                            &lt;/p&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Action&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
          &lt;/thead&gt;
          &lt;tbody&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace01.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Find a spot that needs buildings added&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace10.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Draw the outline, including the backends&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace11.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Add the &lt;code&gt;building=residential&lt;/code&gt; tag&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace12.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Select road and outline, and press &lt;code&gt;Shift-T&lt;/code&gt; to run the terracer plugin with the "fancy outline" checkbox set.&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace13.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Drag the selected middle nodes to the right position.&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
            &lt;tr&gt;
              &lt;td&gt;
                &lt;img src="derickrethans.nl/rst/images/terrace14.png"/&gt;
              &lt;/td&gt;
              &lt;td&gt;
                &lt;p&gt;Press &lt;code&gt;Shift-E&lt;/code&gt; to select all buildings, and press &lt;code&gt;q&lt;/code&gt; to orthogonalise them.&lt;/p&gt;
              &lt;/td&gt;
            &lt;/tr&gt;
          &lt;/tbody&gt;
        &lt;/table&gt;
        &lt;p&gt;As you can see, this makes it much faster to draw such building outlines. When you look at it in 3D however, you can see there is still a problem though:&lt;/p&gt;
        &lt;img src="derickrethans.nl/rst/images/terrace-3d-now.png"/&gt;
        &lt;p&gt;The 3D image that is being drawn does not take into account that the "extension" at the back, isn't part of the gabled roof nor that it is only two instead of three levels. There are various possibilities of building layouts:&lt;/p&gt;
        &lt;img src="derickrethans.nl/rst/images/terrace-3d-alternatives.png"/&gt;
        &lt;p&gt;So the next versions of Überterrace are going to support (some of) these as well!&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/buildings.html</guid>
        <pubDate>Wed, 05 Aug 2015 08:25:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Adressing Flats</title>
        <link>https://derickrethans.nl/flats.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="adressing_flats"/&gt;Adressing Flats&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Tuesday, July 21st 2015, 09:11 BST&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;I recently moved into a new area in London, and that means, a new area to map. That means drawing in all the buildings, but also adding shops and addresses. Unlike my old area, the new neighbourhood has lots of buildings with flats. In &lt;a href="http://openstreetmap.org"&gt;OpenStreetMap&lt;/a&gt;, you can tag addresses on flats with the &lt;a href="http://wiki.openstreetmap.org/wiki/Key:addr:flats"&gt;addr:flats&lt;/a&gt; tag. In most cases, the addresses of the flats are formed by a flat number, building name (&lt;a href="http://wiki.openstreetmap.org/wiki/Key:addr:housename"&gt;addr:housename&lt;/a&gt;) and street name (&lt;a href="http://wiki.openstreetmap.org/wiki/Key:addr:street"&gt;addr:street&lt;/a&gt;). Only occasionally the address consists of flat number, house number, and street name.&lt;/p&gt;
        &lt;p&gt;When I started tagging a nearby collection of flat blocks, and the flat number/house name/street name on their entrance nodes, I noticed that it rendered as:&lt;/p&gt;
        &lt;img src="derickrethans.nl/rst/images/flatnr-before.png"/&gt;
        &lt;p&gt;As you can see, it ignored the &lt;a href="http://wiki.openstreetmap.org/wiki/Key:addr:flats"&gt;addr:flats&lt;/a&gt; tag, and instead just rendered the &lt;a href="http://wiki.openstreetmap.org/wiki/Key:addr:housename"&gt;addr:housename&lt;/a&gt; tag over and over again. Not particularly useful.&lt;/p&gt;
        &lt;p&gt;So I thought, why not "fix" the rendering. In the past, I already added a basic rendering for &lt;a href="http://wiki.openstreetmap.org/wiki/Tag:natural%3Dtree_row"&gt;natural=tree_row&lt;/a&gt; to the "default" rendering style. So I had another look on how to make &lt;a href="http://wiki.openstreetmap.org/wiki/Key:addr:flats"&gt;addr:flats&lt;/a&gt; render and show up instead of the house name 20 times.&lt;/p&gt;
        &lt;p&gt;OpenStreetMap's style sheet is developed through the Git repository at &lt;code&gt;git@github.com:gravitystorm/openstreetmap-carto.git&lt;/code&gt;. I cloned the repository and searched after the places where I needed to make modifications to add the flat numbers. The style sheets first require a "style" file to tell which data to import from the main database (or &lt;code&gt;.xml&lt;/code&gt; file). Only fields that are copied into the rendering database are available for use in the style sheets. OpenStreetMap Carto stores these definitions in the &lt;code&gt;project.mml&lt;/code&gt; file. But that's a nasty JSON file, so instead, you add the definition into the &lt;code&gt;project.yaml&lt;/code&gt; file. There is a script to convert the YAML file into the &lt;code&gt;project.mml&lt;/code&gt; file which the &lt;code&gt;carto&lt;/code&gt; tool can use to generate a &lt;a href="http://mapnik.org/"&gt;Mapnik&lt;/a&gt; stylesheet. Mapnik is then responsible for rendering the map.&lt;/p&gt;
        &lt;p&gt;In &lt;code&gt;project.yaml&lt;/code&gt;, just above the &lt;code&gt;- id: "housenumbers"&lt;/code&gt; line, I added the following section:&lt;/p&gt;
        &lt;pre&gt;- id: "flatnumbers"
    name: "flatnumbers"
    class: ""
    geometry: "point"
    &lt;&lt;: *extents
    Datasource:
      &lt;&lt;: *osm2pgsql
      table: |-
        (SELECT
            way,
            "addr:flats",
            way_area/NULLIF(!pixel_width!::real*!pixel_height!::real,0) AS way_pixels
          FROM planet_osm_polygon
          WHERE "addr:flats" IS NOT NULL
            AND building IS NOT NULL
        UNION ALL
        SELECT
            way,
            "addr:flats",
            NULL AS way_pixels
          FROM planet_osm_point
          WHERE "addr:flats" IS NOT NULL
          ORDER BY way_pixels DESC NULLS LAST
        ) AS flatnumbers
    properties:
      minzoom: 17
    advanced: {}

  &lt;/pre&gt;
        &lt;p&gt;It basically instructs the stylesheet to run an extra query to obtain information on &lt;code&gt;addr:flats&lt;/code&gt; and make that information available to the actual styles.&lt;/p&gt;
        &lt;p&gt;It requires an extra column in the database (&lt;code&gt;addr:flats&lt;/code&gt;), the import "style" needs to include this data while importing from the data source into the rendering database. The file containing the import definitions is &lt;code&gt;openstreetmap-carto.style&lt;/code&gt;. I added the following line in between the definitions for &lt;code&gt;access&lt;/code&gt; and &lt;code&gt;addr:housenumber&lt;/code&gt;:&lt;/p&gt;
        &lt;pre&gt;node,way   addr:flats   text         linear

  &lt;/pre&gt;
        &lt;p&gt;Adding an extra column into the "style" means that it is necessary to reimport the data from the main database (or XML file) into the rendering database. My original source is the &lt;code&gt;openstreetmap.osm&lt;/code&gt; file, which I re-imported with:&lt;/p&gt;
        &lt;pre&gt;/home/derick/install/osm2pgsql/osm2pgsql \
          -S openstreetmap-carto.style --slim -d gis -C 2400 openstreetmap.osm

  &lt;/pre&gt;
        &lt;p&gt;To render the flat numbers, and attach a style to them, I had to add a section to the &lt;code&gt;addressing.mss&lt;/code&gt; file which is a sub-stylesheet that deals with address information. This stylesheet is linked from the project file. I added the following lines in between the &lt;code&gt;#interpolation&lt;/code&gt; and &lt;code&gt;#housenumbers&lt;/code&gt; sections:&lt;/p&gt;
        &lt;pre&gt;#flatnumbers {
    [zoom &gt;= 17] {
          text-name: "[addr:flats]";
          text-placement: interior;
          text-min-distance: 1;
          text-wrap-width: 0;
          text-face-name: @book-fonts;
          text-fill: #666;
          text-size: 8;
    }
  }

  &lt;/pre&gt;
        &lt;p&gt;I picked the same style as for standard house numbers, but one size smaller.&lt;/p&gt;
        &lt;p&gt;With all the sections added to the right files I had to regenerate the &lt;code&gt;project.mml&lt;/code&gt; file from &lt;code&gt;project.yaml&lt;/code&gt; with:&lt;/p&gt;
        &lt;pre&gt;./scripts/yaml2mml.py &lt; project.yaml &gt; project.mml

  &lt;/pre&gt;
        &lt;p&gt;And generate a &lt;a href="http://mapnik.org/"&gt;Mapnik&lt;/a&gt; stylesheet from the &lt;code&gt;project.mml&lt;/code&gt; file with:&lt;/p&gt;
        &lt;pre&gt;carto project.mml &gt; mapnik.osm

  &lt;/pre&gt;
        &lt;p&gt;With all that in place, I re-rendered the area. The flat numbers are now visible on the map:&lt;/p&gt;
        &lt;img src="derickrethans.nl/rst/images/flatnr-after.png"/&gt;
        &lt;p&gt;I have packaged up all the changes, and prepared a &lt;a href="http://github.com/gravitystorm/openstreetmap-carto/pull/1665"&gt;pull request&lt;/a&gt; for &lt;code&gt;openstreetmap-carto&lt;/code&gt;. With some luck, the changes will make it to the OpenStreetMap website at some point in the future.&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/flats.html</guid>
        <pubDate>Tue, 21 Jul 2015 08:11:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Mapping Lubumbashi</title>
        <link>https://derickrethans.nl/mapping-lubumbashi.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="mapping_lubumbashi"/&gt;Mapping Lubumbashi&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Tuesday, May 6th 2014, 19:26 BST&lt;/div&gt;
        &lt;/div&gt;
        &lt;img class="right" align="right" src="derickrethans.nl/images/content/hot-logo.png"/&gt;
        &lt;p&gt;I spent last Saturday helping out HOT, the &lt;a href="http://wiki.openstreetmap.org/wiki/Humanitarian_OpenStreetMap_Team"&gt;Humanitarian OpenStreetMap Team&lt;/a&gt;, to improve the map of Lubumbashi, Congo. This session focussed on digitizing the data that was acquired on the ground by volunteers (with pen and paper).&lt;/p&gt;
        &lt;p&gt;But this was not where the effort started. It started a few weeks ago when &lt;a href="https://twitter.com/Zjwarie"&gt;Jorieke&lt;/a&gt; set off to Lubumbashi to organise the mapping party. You can read more about this on the &lt;a href="http://hot.openstreetmap.org/updates/2014-04-01_a_week_in_lubumbashi_drc"&gt;HOT blog&lt;/a&gt;.&lt;/p&gt;
        &lt;p&gt;In cooperation with &lt;a href="http://www.msf.org.uk/"&gt;Doctors without Borders&lt;/a&gt;/Médecins Sans Frontières UK (MSF UK), HOT organised this mapping effort, with the help of students from the University of Lubumbashi. The goal was to map the whole or at least as much as possible of the city of Lubumbashi.&lt;/p&gt;
        &lt;p&gt;At the start of the &lt;a href="https://wiki.openstreetmap.org/wiki/2014_London_HOT_Congo_Mapathon"&gt;Mapathon&lt;/a&gt;, &lt;a href="https://twitter.com/ivangayton"&gt;Harry Wood&lt;/a&gt; went through his usual spiel of introducing OpenStreetMap to the crowd. I dropped in the moment he was done, just in time to hear &lt;a href="https://twitter.com/ivangayton"&gt;Ivan Gayton&lt;/a&gt;'s talk on the work of MSF and their Congo mapping work. Mapping in Congo is not nearly as simple as it is here in London, where streets have street names that are neatly printed on street signs. This is not the case in Congo. There no street signs and often, people use names that are either not official, or have no official name anyway. The only way to find out how streets are called, is by asking the people living there!&lt;/p&gt;
        &lt;p&gt;MSF is interested in street name information, or more precisely, the neighbourhood names so that they have an identifier for statistical analysis for epidemiology. The accurate location names allows them to pinpoint for example cholera outbreaks, and hopefully in the future they can take preventative action against disease outbreaks.&lt;/p&gt;
        &lt;p&gt;The first step was creating a basic road layout, which was handled by a &lt;a href="http://wiki.openstreetmap.org/wiki/MSF_Congo_Mapping_Party,_Berlin"&gt;Mapathon in Berlin&lt;/a&gt;. Coordination of such large mapping efforts happen through the &lt;a href="http://tasks.hotosm.org/"&gt;OSM Tasking Manager&lt;/a&gt;. You can see the job for Lubumbashi at &lt;a href="http://tasks.hotosm.org/job/456"&gt;http://tasks.hotosm.org/job/456&lt;/a&gt; They did not stop at Lubumbashi though, and traced a whole of of other towns as well. And the mapping was not only remotely done from Berlin, but OpenStreetMap communities from &lt;a href="https://twitter.com/OSM_HU/statuses/447367461310971905"&gt;Hungary&lt;/a&gt;, &lt;a href="https://twitter.com/ndongamadou/statuses/447435101035847680"&gt;Togo&lt;/a&gt;, &lt;a href="https://twitter.com/FoFB7/statuses/447794067540176896"&gt;Burkina&lt;/a&gt; helped as well—a true global community.&lt;/p&gt;
        &lt;img class="left" align="left" src="derickrethans.nl/images/content/hot-lubumbashi.jpg"/&gt;
        &lt;p&gt;Besides this initial road tracing job, there are a few other things still to trace too. If you have time, give them a hand for &lt;a href="http://tasks.hotosm.org/job/459"&gt;http://tasks.hotosm.org/job/459&lt;/a&gt;&lt;/p&gt;
        &lt;p&gt;With a basic road layout traced from areal images you have a rough idea on how a city looks like, but nothing beats a survey on the ground to see how things actually are. It is not always easy to spot things on areal imagery.&lt;/p&gt;
        &lt;p&gt;While remote mapping parties were nearing their completion of the road network, a group of volunteers from Department of Geography of the University of Lubumbashi went out into the neighbourhoods with &lt;a href="http://fieldpapers.org/"&gt;FieldPapers&lt;/a&gt; to do the survey. Through hints from Ivan I understood that it took MSF some time to convince their professor to help out for free. He could hardly believe that some organisation was collecting all this data and then making it available for free. Not just for "no money", but also under an open license. Perhaps he was worried about what Google &lt;a href="http://wiki.openstreetmap.org/wiki/Google_Map_Maker"&gt;MapMaker&lt;/a&gt; is trying to do in underdeveloped countries—getting volunteers to collect data for them and then locking it up in GoogleMaps. Having open access to the data is of course much more valuable.&lt;/p&gt;
        &lt;p&gt;FieldPapers allows you to create &lt;a href="http://fieldpapers.org/atlas.php?id=vsm3w2nd"&gt;atlases&lt;/a&gt; of areas. It divides this up into bite size chunks so that you can print them out and mark your survey information on the paper. For the Lubumbashi mapthon, we had an immense stack of those FieldPapers with notes on them. And our task was to go through this stack and digitize the notes on the map through an OpenStreetMap editor, such as JOSM. I think I managed to go through about 100 to 150 pages of them during the mapathon. One of the &lt;a href="http://www.openstreetmap.org/changeset/22108730"&gt;areas&lt;/a&gt; that I did was about 30 sheets of paper!&lt;/p&gt;
        &lt;p&gt;In the end, we made it through all of the FieldPapers. Task completed! But that does not mean we're finished. This effort was only a small section of Lubumbashi and the local volunteers are going to continue their effort. But they are also learning how to edit the data themselves. And that is the only sustainable way, but I will try to help out whenever I can!&lt;/p&gt;
        &lt;div class="credits"&gt;
          &lt;a href="https://www.flickr.com/photos/harrywood/13909444627/"&gt;
            &lt;p&gt; Derick going through FieldPapers for Lubumbashi — CC by-sa Harry Wood&lt;/p&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/mapping-lubumbashi.html</guid>
        <pubDate>Tue, 06 May 2014 18:26:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Hunting for Postboxes (part 1)</title>
        <link>https://derickrethans.nl/postbox-hunting.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="hunting_for_postboxes_part_1"/&gt;Hunting for Postboxes (part 1)&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Tuesday, January 28th 2014, 09:11 GMT&lt;/div&gt;
        &lt;/div&gt;
        &lt;img class="left" align="left" src="derickrethans.nl/images/content/postbox-pebble.png"/&gt;
        &lt;p&gt;The new year brings new hobby projects! In this year, I am going to try to photograph as many UK postboxes as I can. For this I am going to build some apps and overview maps, but I will write more about those later. A teaser for a future post is here to the left ;-).&lt;/p&gt;
        &lt;p&gt;The first thing that I want my application(s) to do is to give a description of where a postbox is located and this post is the tale of how I went about just doing that.&lt;/p&gt;
        &lt;p&gt;As my data source, I am using an &lt;a href="http://openstreetmap.org"&gt;OpenStreetMap&lt;/a&gt; extract for the &lt;a href="http://metro.teczno.com/#london"&gt;London region&lt;/a&gt;. I import that through my "Import OSM data into MongoDB" script that I wrote about in an &lt;a href="https://derickrethans.nl/importing-osm-into-mongodb.html"&gt;earlier article&lt;/a&gt;. The script is available on GitHub as part of the 3angle repository. The latest version is at &lt;a href="https://raw.github.com/derickr/3angle/master/import-data.php"&gt;https://raw.github.com/derickr/3angle/master/import-data.php&lt;/a&gt; and it also requires &lt;a href="https://raw.github.com/derickr/3angle/master/classes.php"&gt;https://raw.github.com/derickr/3angle/master/classes.php&lt;/a&gt; for some GeoJSON helper classes and &lt;a href="https://raw.github.com/derickr/3angle/master/config.php"&gt;https://raw.github.com/derickr/3angle/master/config.php&lt;/a&gt; where you can set the database name and collection name (in my case, &lt;code&gt;demo&lt;/code&gt; and &lt;code&gt;poiConcat&lt;/code&gt;).&lt;/p&gt;
        &lt;p&gt;My goal is to get from a longitude/latitude pair to the closest postbox's reference number (&lt;code&gt;EC2 201&lt;/code&gt;), a description like (&lt;code&gt;On Paul Street, on the
  corner with Epworth Street&lt;/code&gt;), and a direction and distance (&lt;code&gt;East, 86m&lt;/code&gt;).&lt;/p&gt;
        &lt;p&gt;The first thing I do is to look-up the closest postbox. I can very easily do that with the following aggrgation query in MongoDB:&lt;/p&gt;
        &lt;pre&gt;&lt;?php
  $m = new MongoClient( 'mongodb://localhost' );
  $d = $m-&gt;selectDb( 'demo' );
  $c = $d-&gt;selectCollection( 'poiConcat' );

  $center = new GeoJSONPoint( (float) $_GET['lon'], (float) $_GET['lat'] );

  $res = $c-&gt;aggregate( array(
          '$geoNear' =&gt; array(
                  'near' =&gt; $center-&gt;getGeoJson(),
                  'distanceField' =&gt; 'distance',
                  'distanceMultiplier' =&gt; 1,
                  'maxDistance' =&gt; 5000,
                  'spherical' =&gt; true,
                  'query' =&gt; array( TAGS =&gt; 'amenity=post_box' ),
                  'limit' =&gt; 1,
          )
  ) );

  &lt;/pre&gt;
        &lt;p&gt;I am using the aggregation framework instead of a normal query because the aggregation framework also can return the distance to the found object. The above returns &lt;code&gt;$res&lt;/code&gt;, and our first result is located in &lt;code&gt;$res['result'][0]&lt;/code&gt;, which looks like:&lt;/p&gt;
        &lt;pre&gt;array(
          '_id' =&gt; 'n905143645',
          'l' =&gt; array(
                  'type' =&gt; 'Point',
                  'coordinates' =&gt; array( -0.1960, 51.5376 ),
          ),
          'm' =&gt; array(
                  'v' =&gt; 5,
                  'cs' =&gt; 14576087,
                  'uid' =&gt; 37137,
                  'ts' =&gt; 1357659884,
          ),
          'ts' =&gt; array(
                  0 =&gt; 'amenity=post_box',
                  1 =&gt; 'operator=Royal Mail',
                  2 =&gt; 'ref=NW6 14',
          ),
          'ty' =&gt; 1,
          'distance' =&gt; 162.15002059040299,
  ),

  &lt;/pre&gt;
        &lt;p&gt;So postbox &lt;code&gt;NW6 14&lt;/code&gt; is &lt;code&gt;162.15&lt;/code&gt; meters away from &lt;code&gt;-0.1979, 51.5385&lt;/code&gt; at &lt;code&gt;-0.1960, 51.5376&lt;/code&gt;, as you can see in this image:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/postbox1.png"/&gt;
        &lt;p&gt;To find a description of where the postbox is, we first find the closest road:&lt;/p&gt;
        &lt;pre&gt;$r = $res['result'][0];
  $query = [
          LOC =&gt; [
                  '$near' =&gt; $r['l']
          ],
          TAGS =&gt; new     MongoRegex(
                  '/^highway=(trunk|pedestrian|service|primary|secondary|tertiary|residential|unclassified)/'
          )
  ];
  $road = $c-&gt;findOne( $query );

  &lt;/pre&gt;
        &lt;p&gt;&lt;code&gt;highway&lt;/code&gt; is an &lt;a href="http://openstreetmap.org"&gt;OpenStreetMap&lt;/a&gt; tag that is also used for footpaths, service roads and alleys. These generally don't have names, so with the regular expression we restrict the query to only return "normal" roads. After executing this query, &lt;code&gt;$road&lt;/code&gt; now contains:&lt;/p&gt;
        &lt;pre&gt;array (
          '_id' =&gt; 'w4442243',
          'ty' =&gt; 2,
          'l' =&gt; array (
                  'type' =&gt; 'LineString',
                  'coordinates' =&gt; array (
                          array (
                                  -0.2046823,
                                  51.5346008,
                          ),
                          …
                          array (
                                  -0.1940129,
                                  51.5384693,
                          ),
                  ),
          ),
          'ts' =&gt; array (
                  'hgv=destination',
                  'highway=secondary',
                  'lit=yes',
                  'name=Brondesbury Road',
                  'note=Signed as maxweight 7.5T for goods vehicles except for access, so have tagged as hgv=destination',
                  'ref=B451',
                  'sidewalk=both',
                  'source:ref=OS OpenData StreetView',
          ),
          'm' =&gt; array (
                  'v' =&gt; 15,
                  'cs' =&gt; 18802367,
                  'uid' =&gt; 37137,
                  'ts' =&gt; 1384017096,
          ),
  )

  &lt;/pre&gt;
        &lt;p&gt;As an image this looks like:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/postbox2.png"/&gt;
        &lt;p&gt;We are interested only in the name (&lt;code&gt;name=Brondesbury Road&lt;/code&gt;) and the geometry (&lt;code&gt;l&lt;/code&gt;). Right now, we can already assemble the description &lt;code&gt;NW6
  14, on Brondesbury Road&lt;/code&gt;. But we also want to know the closest cross road, which we can find by finding all roads that intersect with our geometry (in &lt;code&gt;l&lt;/code&gt;) by running the following query:&lt;/p&gt;
        &lt;pre&gt;$q = $c-&gt;find( [
          'l' =&gt; [
                  '$geoIntersects' =&gt; [ '$geometry' =&gt; $road['l'] ]
          ],
          'ts' =&gt; new MongoRegex(
                  '/^highway=(trunk|pedestrian|service|primary|secondary|tertiary|residential|unclassified)/'
          ),
          '_id' =&gt; [ '$ne' =&gt; $road['_id'] ],
  ] );

  &lt;/pre&gt;
        &lt;p&gt;This returns nineteen roads. An extract looks like:&lt;/p&gt;
        &lt;pre&gt;array(19) {
          'w4211713' =&gt; array(5) {
                  '_id' =&gt; string(8) "w4211713"
                  'ty' =&gt; int(2)
                  'l' =&gt; array(2) {
                          'type' =&gt; string(10) "LineString"
                          'coordinates' =&gt; array(22) { ... }
                  }
                  'ts' =&gt;
                  array(4) {
                          [0] =&gt; string(19) "highway=residential"
                          [1] =&gt; string(15) "maxspeed=20 mph"
                          [2] =&gt; string(23) "name=Brondesbury Villas"
                          [3] =&gt; string(13) "sidewalk=both"
                  }
                  …
          }
          'w245650577' =&gt;
          array(5) {
                  '_id' =&gt; string(10) "w245650577"
                  'ty' =&gt; int(2)
                  'l' =&gt; array(2) {
                          'type' =&gt; string(10) "LineString"
                          'coordinates' =&gt; array(5) { ... }
                  }
                  'ts' =&gt;
                  array(6) {
                          [0] =&gt; string(15) "highway=primary"
                          [1] =&gt; string(7) "lit=yes"
                          [2] =&gt; string(15) "maxspeed=30 mph"
                          [3] =&gt; string(22) "name=Kilburn High Road"
                          [4] =&gt; string(6) "ref=A5"
                          [5] =&gt; string(13) "sidewalk=both"
                  }
                  …
          }
  }

  &lt;/pre&gt;
        &lt;p&gt;As an image this looks like:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/postbox3.png"/&gt;
        &lt;p&gt;We are only interested in the roads that have a name and have a &lt;strong&gt;different&lt;/strong&gt; name than the road we have run the intersection query for. In some cases, &lt;a href="http://openstreetmap.org"&gt;OpenStreetMap&lt;/a&gt; splits up a road in more than one segment carrying the same name. We discard both those in a loop and are then left with an array of intersecting road IDs in the &lt;code&gt;$intersectingWays&lt;/code&gt; variable:&lt;/p&gt;
        &lt;pre&gt;$intersectingWays = array();
  foreach ( $q as $crossRoad )
  {
          $crossTags = Functions::split_tags( $crossRoad[TAGS] );
          if ( !in_array( "name={$roadName}", $crossRoad ) &amp;&amp; array_key_exists( 'name', $crossTags ) )
          {
                  $intersectingWays[] = $crossRoad['_id'];
          }
  }

  &lt;/pre&gt;
        &lt;p&gt;With these IDs, we then search for the closest road(s) to the initially found postbox location:&lt;/p&gt;
        &lt;pre&gt;$res = $c-&gt;aggregate( array(
          '$geoNear' =&gt; array(
                  'near' =&gt; $r['l'],
                  'distanceField' =&gt; 'distance',
                  'distanceMultiplier' =&gt; 1,
                  'maxDistance' =&gt; 5000,
                  'spherical' =&gt; true,
                  'query' =&gt; [
                          '_id' =&gt; [ '$in' =&gt; $intersectingWays ],
                          'ts' =&gt; [ '$ne' =&gt; "name={$roadName}" ]
                  ],
                  'limit' =&gt; 1,
          )
  ) );

  &lt;/pre&gt;
        &lt;p&gt;Again, the result in &lt;code&gt;$res&lt;/code&gt; is in a similar format as before, so I won't repeat that. We use the aggrgation framework again so that we also get the distance of this intersecting road to the originally found postbox location. Depending on the distance to the intersecting road, we either use &lt;code&gt;on the
  corner of &lt;roadname&gt;&lt;/code&gt; (less thatn 25m) or &lt;code&gt;near &lt;roadname&gt;&lt;/code&gt; if it's further away than 25m. For our example postbox, that makes &lt;code&gt;NW6 14, on Brondesbury
  Road, near Algernon Road&lt;/code&gt; which is illustrated by this image:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/postbox4.png"/&gt;
        &lt;p&gt;The full code for this example can be found at &lt;a href="https://github.com/derickr/3angle/tree/master/maps-postbox"&gt;https://github.com/derickr/3angle/tree/master/maps-postbox&lt;/a&gt; and you see it in action (for London) at: &lt;a href="http://maps.derickrethans.nl/?l=postbox,lat=51.5&amp;lon=-0.128&amp;zoom=17"&gt;http://maps.derickrethans.nl/?l=postbox,lat=51.5&amp;lon=-0.128&amp;zoom=17&lt;/a&gt;&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/postbox-hunting.html</guid>
        <pubDate>Tue, 28 Jan 2014 09:11:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Smoothing lines with splines</title>
        <link>https://derickrethans.nl/splines.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="smoothing_lines_with_splines"/&gt;Smoothing lines with splines&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; La Clusaz, France&lt;/div&gt;
          &lt;div class="date"&gt;Tuesday, January 21st 2014, 09:44 CET&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;For my &lt;a href="https://derickrethans.nl/year-of-edits-2013.html"&gt;OpenStreetMap year of edits&lt;/a&gt; videos I use &lt;a href="http://povray.org"&gt;PovRay&lt;/a&gt; to animate edits of &lt;a href="http://openstreetmap.org"&gt;OpenStreetMap&lt;/a&gt;. Over the course of a year I show all the edits that have been made. In previous years, I used a simple sine function to rotate the Earth alone it's North/South and East/West axises. This year I wanted to highlight specific events so I needed a different method to rotate the Earth.&lt;/p&gt;
        &lt;p&gt;I started out with finding events and created two paths for them:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/path2.png"/&gt;
        &lt;p&gt;This one starts in Greenland, moves towards Brazil and then heads over Russia, India, the Philippines and ends in Indonesia.&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/path1.png"/&gt;
        &lt;p&gt;The second one starts in Korea, moves through Africa, heads towards Antarctica and then heads back to France over Brazil.&lt;/p&gt;
        &lt;p&gt;Each of the two paths have points highlighted which are (about) 15 days apart. The paths I converted into data files, which look like&lt;/p&gt;
        &lt;pre&gt;0000  -24   70   8500
  0180  -64   58   9500
  0240  -81   29  10000
  0480  -43  -16  15000
  0720   -8   36  20000
  0840   27   44  15000
  1020   76   20  15000
  1140  115   25  15000
  1320  127   -2   9000
  1460  157  -24  10000

  &lt;/pre&gt;
        &lt;p&gt;The first column is the frame number (four frames per day), then the longitude and the latitude and finally the height of the camera over the point. As you can see, I have not included a line for each of the points in the path images.&lt;/p&gt;
        &lt;p&gt;From the points in the data files, I then needed to generate all the intermediate points for each of the three columns. And my first (naive) attempt was to simply do a linear interpolation. Point &lt;em&gt;1&lt;/em&gt;'s longitude can be calculated by: &lt;code&gt;-24 + (1-0) * ((-64 - -24) / (180 - 0))&lt;/code&gt; and point &lt;em&gt;856&lt;/em&gt;'s longitude with: &lt;code&gt;27 + (855 - 840) * ((76 - 115) / (1020 - 840))&lt;/code&gt;. Doing that for all of the three axis (for the first 300 frames) results in the following video:&lt;/p&gt;
        &lt;div class="video"&gt;
          &lt;div class="videoWrapper"&gt;
            &lt;iframe src="https://player.vimeo.com/video/83049723?title=0&amp;byline=0&amp;portrait=0" frameborder="0"/&gt;
          &lt;/div&gt;
          &lt;p&gt;&lt;a href="https://vimeo.com/83049723"/&gt;Linear interpolation of a line between a set of points&lt;/p&gt;
        &lt;/div&gt;
        &lt;p&gt;As you can see, the changes in direction (and height) are really abrupt, and doesn't make for a nice smooth rotating body. So I had to come up with something else: a smoothed line.&lt;/p&gt;
        &lt;p&gt;There are various different ways of smoothing line, but one of the easier ones that I found was the &lt;a href="http://en.wikipedia.org/wiki/Spline_%28mathematics%29"&gt;Spline&lt;/a&gt;. A spline is a smoothing function that connects all the data points with sufficiently smooth curves. I used the implementation at &lt;a href="http://www.script-tutorials.com/smooth-curve-graphs-with-php-and-gd/"&gt;http://www.script-tutorials.com/smooth-curve-graphs-with-php-and-gd/&lt;/a&gt; with the other script available through &lt;a href="https://github.com/derickr/osm-year-in-edits/blob/master/changes/create-camera-from-file.php"&gt;https://github.com/derickr/osm-year-in-edits/blob/master/changes/create-camera-from-file.php&lt;/a&gt;&lt;/p&gt;
        &lt;p&gt;If we now look at the same section of the video, we see that the turns and zoom level are much smoother:&lt;/p&gt;
        &lt;div class="video"&gt;
          &lt;div class="videoWrapper"&gt;
            &lt;iframe src="https://player.vimeo.com/video/83049724?title=0&amp;byline=0&amp;portrait=0" frameborder="0"/&gt;
          &lt;/div&gt;
          &lt;p&gt;&lt;a href="https://vimeo.com/83049724"/&gt;Interpolation between a set of points with a spline&lt;/p&gt;
        &lt;/div&gt;
        &lt;p&gt;Just focussing on the zoom aspect, I am also reproducing a line graph of the interpolated points here:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/spline.png"/&gt;
        &lt;p&gt;As you can see, between 1172 and 1328 it makes a very big dip—and that wrecked with my video as it zoomed in too much. I've fixed that manually for the video, but I would like to find an algorithm that did produce smooth lines without such a big "dip". Any ideas are most welcome.&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/splines.html</guid>
        <pubDate>Tue, 21 Jan 2014 08:44:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>OpenStreetMap: A Year of Edits 2013</title>
        <link>https://derickrethans.nl/year-of-edits-2013.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="openstreetmap_a_year_of_edits_2013"/&gt;OpenStreetMap: A Year of Edits 2013&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Wednesday, January 1st 2014, 14:02 GMT&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;Just like the &lt;a href="https://derickrethans.nl/year-of-edits-2013.html"&gt;last few years&lt;/a&gt; I've analysed all the edits to OpenStreetMap and produced a visualisation. This year I wanted to highlight different projects that OpenStreetMap contributors have done and so instead of having an uniform rotational spinning globe it moved between interesting events.&lt;/p&gt;
        &lt;p&gt;The result is: &lt;strong&gt;OpenStreetMap: A Year of Edits (2013)&lt;/strong&gt;:&lt;/p&gt;
        &lt;div class="video"&gt;
          &lt;div class="videoWrapper"&gt;
            &lt;iframe src="https://player.vimeo.com/video/83164362?title=0&amp;byline=0&amp;portrait=0" frameborder="0"/&gt;
          &lt;/div&gt;
          &lt;p&gt;&lt;a href="https://vimeo.com/83164362"/&gt;OpenStreetMap: A Year of Edits (2013)&lt;/p&gt;
        &lt;/div&gt;
        &lt;p&gt;This animation shows all additions and modifications of nodes (flashes) and cumulative edits (fading to purple) in 2013.&lt;/p&gt;
        &lt;p&gt;Some of the highlighted events in no particular order are:&lt;/p&gt;
        &lt;ul&gt;
          &lt;li&gt;
            &lt;p&gt;
              &lt;a href="http://wiki.openstreetmap.org/wiki/Antarctica/Import_2013"&gt;Antarctica coastline imports&lt;/a&gt;
            &lt;/p&gt;
          &lt;/li&gt;
          &lt;li&gt;
            &lt;p&gt;&lt;a href="http://www.openstreetmap.fr/36680-communes"&gt;Completion of the boundary data in France&lt;/a&gt; (In French)&lt;/p&gt;
          &lt;/li&gt;
          &lt;li&gt;
            &lt;p&gt;
              &lt;a href="http://hot.openstreetmap.org/updates/2013-11-10_remote_hot_activation_in_the_philippines_for_typhoon_yolandahaiyan"&gt;Relief mapping in the Philippines&lt;/a&gt;
            &lt;/p&gt;
          &lt;/li&gt;
        &lt;/ul&gt;
        &lt;p&gt;You can see the video in HD on &lt;a href="http://vimeo.com/derickr/osm-2013"&gt;Vimeo&lt;/a&gt;.&lt;/p&gt;
        &lt;p&gt;I have started to make the code for this available through GitHub, but the &lt;a href="https://github.com/derickr/osm-year-in-edits"&gt;code&lt;/a&gt; is very much not modularized and it is likely it will only work on my set-up. Perhaps I manage to fix this better during the upcoming year!&lt;/p&gt;
        &lt;p&gt;
          &lt;strong&gt;Mappy New Year!&lt;/strong&gt;
        &lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/year-of-edits-2013.html</guid>
        <pubDate>Wed, 01 Jan 2014 14:02:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Contributing Advent 24: Wrapping up!</title>
        <link>https://derickrethans.nl/advent24.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="contributing_advent_24_wrapping_up"/&gt;Contributing Advent 24: Wrapping up!&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; Dieren, Netherlands&lt;/div&gt;
          &lt;div class="date"&gt;Tuesday, December 24th 2013, 10:33 CET&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;Yay, the last one! Writing 24 blog posts in 24 days is a lot more work than actually contributing 24 times. Let's go over the different things that I managed to accomplish, and what you can do to continue some of that work yourself.&lt;/p&gt;
        &lt;p&gt;
          &lt;strong&gt;PHP Date/Time&lt;/strong&gt;
        &lt;/p&gt;
        &lt;p&gt;A few of my contributions are related to PHP's DateTime support. On the &lt;a href="https://derickrethans.nl/advent06.html"&gt;6th&lt;/a&gt; and the &lt;a href="https://derickrethans.nl/advent09.html"&gt;9th&lt;/a&gt; I wrote about a bug fixes in "Cookie formats" and "Day of the week" calculations. The bug reports (&lt;a href="https://bugs.php.net/bug.php?id=61599"&gt;#61599&lt;/a&gt;, &lt;a href="https://bugs.php.net/bug.php?id=63391"&gt;#63391&lt;/a&gt;, &lt;a href="https://bugs.php.net/bug.php?id=63776"&gt;#63776&lt;/a&gt; and &lt;a href="https://bugs.php.net/bug.php?id=64294"&gt;#64294&lt;/a&gt;) mentioned issues all came with a good reproducible script. Please continue filing bug reports, and make sure you follow the guidelines (if possible/necessary) from the &lt;a href="https://derickrethans.nl/advent23.html"&gt;23rd&lt;/a&gt;.&lt;/p&gt;
        &lt;p&gt;The other DateTime issue that I touched on the &lt;a href="https://derickrethans.nl/advent21.html"&gt;21st&lt;/a&gt; is probably something that you can't really contribute too—unless you want to start petitions and lobby to get rid of Daylight Saving Time altogether…&lt;/p&gt;
        &lt;p&gt;
          &lt;strong&gt;Xdebug&lt;/strong&gt;
        &lt;/p&gt;
        &lt;p&gt;I have been working on and off on &lt;a href="http://xdebug.org"&gt;Xdebug&lt;/a&gt; for about 13 years now, and it is still a project I am very interested in. So it could not have come as a surprise that there were four posts about it. Some of them, such as on the &lt;a href="https://derickrethans.nl/advent01.html"&gt;1st&lt;/a&gt; ("Xdebug and hidden properties)" were helped by a good &lt;a href="http://bugs.xdebug.org/view.php?id=987"&gt;bug report&lt;/a&gt;. But there are plenty of more &lt;a href="http://bugs.xdebug.org/view_all_bug_page.php"&gt;bugs&lt;/a&gt; in Xdebug. Some of them I can easily reproduce, others only with difficulty and some bugs are just weird. Going over the bug list of your favourite project and trying to reproduce and clarify them is of great help to project maintainers that do not always have a lot of time themselves. In some cases, they might not even have access to the same platforms (ugh, Windows) as what the bug report was filed for.&lt;/p&gt;
        &lt;p&gt;Other ways to help an Open Source project is by answering questions on the project's mailing list, or on IRC as I did in the post on the &lt;a href="https://derickrethans.nl/advent08.html"&gt;8th&lt;/a&gt;. Project maintainers are not awake 24 hours, and especially if you are in a different timezone, your knowledge can help other people. I often lurk in channels on IRC to help out on Xdebug, PHP, OpenStreetMap and MongoDB.&lt;/p&gt;
        &lt;p&gt;And of course you can contribute to Open Source projects by just submitting pull requests like I did for the &lt;a href="https://derickrethans.nl/advent15.html"&gt;15th&lt;/a&gt; and the &lt;a href="https://derickrethans.nl/advent20.html"&gt;20th&lt;/a&gt;. Most projects will have guidelines on how to contribute. For example Xdebug has a &lt;a href="http://xdebug.org/contributing.php"&gt;contributing&lt;/a&gt; page, and the MongoDB driver for PHP has a &lt;a href="https://github.com/mongodb/mongo-php-driver/blob/master/CONTRIBUTING.md"&gt;page&lt;/a&gt; on GitHub.&lt;/p&gt;
        &lt;p&gt;
          &lt;strong&gt;OpenStreetMap&lt;/strong&gt;
        &lt;/p&gt;
        &lt;p&gt;Another large group of contributions (7!) are related to &lt;a href="http://openstreetmap.org"&gt;OpenStreetMap&lt;/a&gt;. There are many things you can do yourself to help fighting commercial monopolies on geographic data. Two of the posts on OpenStreetMap (on the &lt;a href="https://derickrethans.nl/advent02.html"&gt;2nd&lt;/a&gt; and &lt;a href="https://derickrethans.nl/advent10.html"&gt;10th&lt;/a&gt;) dealt with ITO World's &lt;a href="http://www.itoworld.com/static/openstreetmap_tools/osm_analysis.html"&gt;OSM Analysis&lt;/a&gt;. This tool highlights differences between Ordnance Survey's OS Locator product and OpenStreetMap. If you are in the UK, why not have a look if there are any errors near you? For example, in &lt;a href="http://www.itoworld.com/product/data/osm_analysis/area?name=Manchester"&gt;Manchester&lt;/a&gt; there are 189 discrepancies right now. If you don't know how to get the right data into OpenStreetMap, take a (geolocated) photo of the street sign and add a link to it in an &lt;em&gt;OpenStreetMap note&lt;/em&gt;. Please do not just believe OS Locator's variant of the name, or copy any data of any other map. The only thing that will suffice is proof (a photo) that the name is different. In some cases, you might of course need more photos as I showed in the post for the &lt;a href="https://derickrethans.nl/advent16.html"&gt;16th&lt;/a&gt;.&lt;/p&gt;
        &lt;p&gt;I posted an article on OpenStreetMap notes on the &lt;a href="https://derickrethans.nl/advent05.html"&gt;5th&lt;/a&gt;. Notes &lt;strong&gt;can&lt;/strong&gt; be posted anonymously but then we see the text "This note includes comments from anonymous users which should be independently verified." You can &lt;a href="https://www.openstreetmap.org/user/new"&gt;sign up&lt;/a&gt; as an OpenStreetMap user, where it will be explained that you &lt;strong&gt;can never copy data from other maps&lt;/strong&gt;, but a nice photo of a street sign is of course good enough as well. You can also add comments to notes as a user, and if you sign up as a user, then you can also add and modify the rest of the map like I demonstrated on the &lt;a href="https://derickrethans.nl/advent19.html"&gt;19th&lt;/a&gt;.&lt;/p&gt;
        &lt;p&gt;
          &lt;strong&gt;Hacking time!&lt;/strong&gt;
        &lt;/p&gt;
        &lt;p&gt;The last group of contributions that I want to talk about are for your &lt;em&gt;own&lt;/em&gt; project. I do lots of little things here and there, for example with the &lt;a href="https://github.com/php-geospatial/geospatial"&gt;geospatial&lt;/a&gt; extension that I wrote about on the &lt;a href="https://derickrethans.nl/advent12.html"&gt;12th&lt;/a&gt;. In my opinion there is nothing more fun that to hack on your pet project. However, sometimes that results in a project that you work on for 12 years (Xdebug!), so be warned!&lt;/p&gt;
        &lt;p&gt;Merry Christmas, and I hope this series of posts inspired you to make your own contributions. Perhaps make it a new year's resolution!&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/advent24.html</guid>
        <pubDate>Tue, 24 Dec 2013 09:33:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Contributing Advent 19: Lunch time mapping</title>
        <link>https://derickrethans.nl/advent19.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="contributing_advent_19_lunch_time_mapping"/&gt;Contributing Advent 19: Lunch time mapping&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Thursday, December 19th 2013, 09:10 GMT&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;I like going out for walks at lunch time. At least during winter times, it's the only amount of fresh air during daylight I am actually getting. During my walks, I also like to do something useful. Then, my laptop's background image shows me an &lt;a href="http://www.openstreetmap.org/about"&gt;OpenStreetMap&lt;/a&gt; map of the area the laptop is currently at. (This now works a lot better again after Mozilla started doing their &lt;a href="https://derickrethans.nl/advent11.html"&gt;location project&lt;/a&gt;). When I am at home, this background however has this (empty) spot on it:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-19a.png"/&gt;
        &lt;p&gt;And empty spots are annoying! So, during a lunch break in the last few days, I went out and walked over &lt;a href="http://www.openstreetmap.org/way/4072795"&gt;The Avenue&lt;/a&gt; to do some mapping. I find it way easier to do this with pen and paper, as compared to an application like &lt;a href="http://wiki.openstreetmap.org/wiki/Vespucci"&gt;Vespucci&lt;/a&gt;. Pen and paper is both faster, and I don't have to fiddle with a tiny phone keyboard. I do however use Vespucci for things like changing a shop's name or type.&lt;/p&gt;
        &lt;p&gt;Notes on pen and paper I make plenty, and they look like this:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-19b.png"/&gt;
        &lt;p&gt;Over the past few years I have developed a short hand for all kinds of things. You can see house numbers (a), how many floors, the colour of the building, the shape and colour of the roof (b), crossings (c) and fences (d) on the extract above.&lt;/p&gt;
        &lt;p&gt;With the notes on paper, I then use &lt;a href="http://josm.openstreetmap.de"&gt;JOSM&lt;/a&gt; to add the information to the map. JOSM's interface looks like:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-19c.png"/&gt;
        &lt;p&gt;JOSM is a little bit like the &lt;a href="http://vim.org"&gt;VIM&lt;/a&gt;, but then for editting OpenStreetMap. It is not the easiest editor, but so much more powerfull than the online editor &lt;a href="http://wiki.openstreetmap.org/wiki/ID"&gt;iD&lt;/a&gt;. With some practice, and learning key combinations, it allows you to pretty quickly add the information from your notes to the map. The following video (at 8 times speed) shows how I added the information from the notes from above to the map:&lt;/p&gt;
        &lt;div class="video"&gt;
          &lt;div class="videoWrapper"&gt;
            &lt;iframe src="https://player.vimeo.com/video/81856483?title=0&amp;byline=0&amp;portrait=0" frameborder="0"/&gt;
          &lt;/div&gt;
          &lt;p&gt;&lt;a href="https://vimeo.com/81856483"/&gt;Contributing Advent 19: Lunch time mapping&lt;/p&gt;
        &lt;/div&gt;
        &lt;p&gt;In the end, when everything has been added and uploaded, and the &lt;a href="http://www.openstreetmap.org/about"&gt;OpenStreetMap&lt;/a&gt; website rendered my updates, the map now looks like:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-19d.png"/&gt;
        &lt;p&gt;Now it is just a matter of adding the information from the other eight pages of notes I made on this trip!&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/advent19.html</guid>
        <pubDate>Thu, 19 Dec 2013 09:10:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Contributing Advent 18: Mapping without access</title>
        <link>https://derickrethans.nl/advent18.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="contributing_advent_18_mapping_without_access"/&gt;Contributing Advent 18: Mapping without access&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Wednesday, December 18th 2013, 09:10 GMT&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;So, this is another one about chasing down issues on ITO World's analysis, and probably not the last one. However, in this case it needed a little bit more than photographing street name signs. This is the tale of &lt;a href="http://www.openstreetmap.org/way/32954706#map=18/51.49746/-0.19013"&gt;St. Mary's Place&lt;/a&gt;.&lt;/p&gt;
        &lt;p&gt;OpenStreetMap had this street as St. Mary's Gate, a road in the same &lt;a href="http://www.openstreetmap.org/#map=18/51.49741/-0.19055"&gt;area&lt;/a&gt;. ITO World's analysis had flagged this up as being incorrect, so before the last &lt;a href="http://bit.ly/londonosm10"&gt;OpenStreetMap winter meet-up&lt;/a&gt; I went to have a look. After checking out an apparently &lt;a href="http://www.openstreetmap.org/way/26741199"&gt;missing street&lt;/a&gt;, and adding it to OpenStreetMap I walked towards the supposed location of St. Mary's Place. However, I couldn't really get near as there was a &lt;a href="http://www.openstreetmap.org/node/376497546"&gt;gate&lt;/a&gt; blocking my way.&lt;/p&gt;
        &lt;p&gt;Not really a problem I thought, so walked to the other side of the estate and encountered &lt;a href="http://www.openstreetmap.org/node/371156450"&gt;another set of gates&lt;/a&gt;. Now I got annoyed as I had walked quite a bit, and I still had to check out &lt;a href="http://www.openstreetmap.org/way/157471066"&gt;Palace Green&lt;/a&gt; before the meet-up as well. In any case, I was looking around to see whether there was a public entrance after all, when a pedestrian left through a gate. Instead of trying to sneak in, I just asked him about the name of the street… and that was it. He could confirm that indeed St. Mary's Gate and St. Mary's Place split in the middle.&lt;/p&gt;
        &lt;p&gt;Getting to know information about Palace Green is tricky too, and that conclusion will have to wait until I've spoken to &lt;a href="http://www.openstreetmap.org/user/Paul%20The%20Archivist"&gt;Paul&lt;/a&gt; who mapped it in the first place.&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/advent18.html</guid>
        <pubDate>Wed, 18 Dec 2013 09:10:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Contributing Advent 16: St. Johns vs. St John's</title>
        <link>https://derickrethans.nl/advent16.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="contributing_advent_16_st_johns_vs_st_john_s"/&gt;Contributing Advent 16: St. Johns vs. St John's&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Monday, December 16th 2013, 09:10 GMT&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;I already wrote about comparing &lt;a href="http://data.gov.uk/dataset/os-locator1"&gt;OS Locator&lt;/a&gt; data with OpenStreetMap for &lt;a href="https://derickrethans.nl/advent02.html"&gt;day 2&lt;/a&gt; and &lt;a href="https://derickrethans.nl/advent08.html"&gt;day 8&lt;/a&gt;, but there are plenty of other highlights in London that I can easily have a look at. So a certain lunch time walk brought me to the borough of &lt;a href="http://en.wikipedia.org/wiki/Kensington_and_Chelsea_London_Borough_Council"&gt;Kensington and Chelsea&lt;/a&gt;. I first looked at &lt;a href="http://www.openstreetmap.org/way/34150447"&gt;Camborne Mews&lt;/a&gt; (which OS Locator had as Cambourne Mews) and &lt;a href="http://www.openstreetmap.org/way/34227497"&gt;Cowling Close&lt;/a&gt; (which OpenStreetMap had as Cowling Cross).&lt;/p&gt;
        &lt;p&gt;But the more interesting thing to look at was the last one of the three. Apostrophes are always a tricky part, both for mappers as well as councils. I had already run into &lt;a href="http://www.openstreetmap.org/way/4981900/history"&gt;St. Marks Road&lt;/a&gt; &lt;a href="http://www.openstreetmap.org/changeset/5449937"&gt;back in 2010&lt;/a&gt; but ITO World now also flagged up St. Johns Gardens. Both segments &lt;em&gt;were&lt;/em&gt; mapped as St. John's Gardens in OpenStreetMap. OS Locator has the street as two &lt;strong&gt;different&lt;/strong&gt; names. For the  &lt;a href="http://ris.dev.openstreetmap.org/oslmusicalchairs/map?osl_id=695310"&gt;western segment&lt;/a&gt; it has &lt;em&gt;St John's Gardens&lt;/em&gt;:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-16a.png"/&gt;
        &lt;p&gt;For the &lt;a href="http://ris.dev.openstreetmap.org/oslmusicalchairs/map?osl_id=695945"&gt;eastern segment&lt;/a&gt; it has &lt;em&gt;St Johns Gardens&lt;/em&gt;:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-16b.png"/&gt;
        &lt;p&gt;These images are extracts from another comparison tool between OS Locator and OpenStreetMap, called &lt;a href="http://ris.dev.openstreetmap.org/oslmusicalchairs/map?osl_id=695310&amp;zoom=18&amp;lat=51.51014&amp;lon=-0.20641&amp;layers=B0TT&amp;view_mode=pseudorandom"&gt;OS Locator musical chairs&lt;/a&gt;, made by &lt;a href="http://wiki.openstreetmap.org/wiki/User:Robert"&gt;Robert Scott&lt;/a&gt;.&lt;/p&gt;
        &lt;p&gt;In any case, after visiting Cowling Close slightly to the west, I came through Clarendon Road and there on the corner with St John's Gardens, I spotted the following sign:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-16c.png"/&gt;
        &lt;p&gt;Ah! So I concluded, the name of the street is St John's Gardens, with apostrophe.&lt;/p&gt;
        &lt;p&gt;I continued along the road towards St Johns Church, and on the corner with Lansdowne Road I found the following two signs:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-16d.png"/&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-16e.png"/&gt;
        &lt;p&gt;Argh, both without apostrophe! On the north end around the church, on the corner with Lansdowne Crescent, there two more signs without apostrophe (sorry, one was too blurry to add here):&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-16f.png"/&gt;
        &lt;p&gt;I took another photo on the corner with Ladbroke Grove but didn't pay much attention to it and assumed it was without apostrophe as well. So with five signs saying &lt;em&gt;St Johns Gardens&lt;/em&gt; (without apostrophe) and sign with &lt;em&gt;St John's Gardens&lt;/em&gt; (with apostrophe), the street is surely just called &lt;em&gt;St Johns Gardens&lt;/em&gt;. Back home, I then &lt;a href="http://www.openstreetmap.org/changeset/19256893"&gt;edited&lt;/a&gt; OpenStreetMap to say that the name of the streets are &lt;em&gt;St Johns Gardens&lt;/em&gt; and &lt;strong&gt;not&lt;/strong&gt; &lt;em&gt;St John's Gardens&lt;/em&gt;.&lt;/p&gt;
        &lt;p&gt;Just now, while writing the article I had a better look at the sign on the corner with Ladbroke Grove:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-16g.png"/&gt;
        &lt;p&gt;And yes, there is indeed an apostrophe… What surprises me most however is that it's really up to the council to decide the spelling of street names, and convey that information to Ordnance Survey. Apparently, they can't make their minds up on what to print on the signs, and what to tell Ordnance Survey (considering they had two parts of the same street with two different names). In any case, OpenStreetMap &lt;em&gt;maps what is on the ground&lt;/em&gt;, and right now, with three against two, &lt;em&gt;St Johns Gardens&lt;/em&gt; (without apostrophe) wins.&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/advent16.html</guid>
        <pubDate>Mon, 16 Dec 2013 09:10:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Contributing Advent 14: Brown Hart Gardens</title>
        <link>https://derickrethans.nl/advent14.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="contributing_advent_14_brown_hart_gardens"/&gt;Contributing Advent 14: Brown Hart Gardens&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Saturday, December 14th 2013, 09:10 GMT&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;Besides fixing bugs from &lt;a href="https://derickrethans.nl/advent02.html"&gt;ITO World&lt;/a&gt; or &lt;a href="https://derickrethans.nl/advent05.html"&gt;OpenStreetMap notes&lt;/a&gt; I also tend to do some general mapping when I am out and about. Coinciding with my "buying the whisky of the month" event (if you want to contribute to this, see the &lt;a href="http://www.amazon.co.uk/registry/wishlist/SLCB276UZU8B"&gt;wishlist&lt;/a&gt; ;-) ), we ended up walking past a rather curious building with a park on top in London: &lt;a href="http://en.wikipedia.org/wiki/Brown_Hart_Gardens"&gt;Brown Hart Gardens&lt;/a&gt;.&lt;/p&gt;
        &lt;p&gt;The gardens, new café, and the "Duke Street Electricity Substation" that lays underneath were not mapped at all, and of course I could not let that be. From photos and memory, I re-created the &lt;a href="http://www.openstreetmap.org/relation/3354838#map=19/51.51315/-0.15145"&gt;gardens&lt;/a&gt; in &lt;a href="http://josm.openstreetmap.de/"&gt;JOSM&lt;/a&gt;, my preferred OpenStreetMap editor and now it is available on the &lt;a href="http://www.openstreetmap.org/#map=19/51.51310/-0.15151"&gt;map&lt;/a&gt; — including the &lt;a href="http://www.openstreetmap.org/way/249455143"&gt;steps&lt;/a&gt; and the &lt;a href="http://www.openstreetmap.org/node/2560520283"&gt;elevator&lt;/a&gt;:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-14a.png"/&gt;
        &lt;p&gt;There is also a 3D rendering &lt;a href="http://map.f4-group.com/#lat=51.5129156&amp;lon=-0.1514470&amp;zoom=19&amp;ui.discoveryOpen=false&amp;camera.theta=71.087&amp;camera.phi=141.853"&gt;available&lt;/a&gt;:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-14b.png"/&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/advent14.html</guid>
        <pubDate>Sat, 14 Dec 2013 09:10:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Contributing Advent 13: Haiyan Progress Video</title>
        <link>https://derickrethans.nl/advent13.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="contributing_advent_13_haiyan_progress_video"/&gt;Contributing Advent 13: Haiyan Progress Video&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Friday, December 13th 2013, 09:10 GMT&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;Once in a while, the &lt;a href="http://wiki.openstreetmap.org/wiki/London/Winter_2013-2014_events"&gt;London OpenStreetMap group&lt;/a&gt; organises a &lt;a href="http://wiki.openstreetmap.org/wiki/London/London_Hack_Weekend_Nov_2013"&gt;hack weekend&lt;/a&gt; where a bunch of us hack on various different projects, such as the &lt;a href="https://www.mapbox.com/blog/redesigning-openstreetmap-org/"&gt;new design&lt;/a&gt;, &lt;a href="http://project-osrm.org"&gt;project OSRM&lt;/a&gt; or &lt;a href="http://sk53-osm.blogspot.co.uk/2013/12/harry-we-missed-you-london-osm-hack.html"&gt;using Food Hygiene Open Data&lt;/a&gt; to improve OpenStreetMap.&lt;/p&gt;
        &lt;p&gt;The last few that I have attended were always close to the end of the year, so I tend to use them to make my "year of edits" videos:&lt;/p&gt;
        &lt;ul&gt;
          &lt;li&gt;
            &lt;p&gt;2010: &lt;a href="https://vimeo.com/34406096"&gt;https://vimeo.com/34406096&lt;/a&gt;&lt;/p&gt;
          &lt;/li&gt;
          &lt;li&gt;
            &lt;p&gt;2011: &lt;a href="https://vimeo.com/34404102"&gt;https://vimeo.com/34404102&lt;/a&gt;&lt;/p&gt;
          &lt;/li&gt;
          &lt;li&gt;
            &lt;p&gt;2012: &lt;a href="https://vimeo.com/56374742"&gt;https://vimeo.com/56374742&lt;/a&gt;&lt;/p&gt;
          &lt;/li&gt;
        &lt;/ul&gt;
        &lt;p&gt;This year instead I focussed on creating a mapping party progress video for the great work that the Humanitarian OpenStreetMap Team (HOT) has been doing for organising and mapping the destruction of Typhoon &lt;a href="http://en.wikipedia.org/wiki/Typhoon_Haiyan"&gt;Haiyan&lt;/a&gt; in the Philippines. There have been lots of &lt;a href="http://wiki.osm.org/wiki/Typhoon_Haiyan#Media_coverage"&gt;media coverage&lt;/a&gt; but I wanted to show how much they have added in the two weeks since the thyphoon hit—and what better to do this then with a progress video.&lt;/p&gt;
        &lt;p&gt;I have made many videos before, mostly for mapping parties in London, but also for other areas, such as &lt;a href="https://vimeo.com/channels/osm/54113868"&gt;Burgos&lt;/a&gt; in Spain, and for the &lt;a href="https://vimeo.com/channels/osm/62772466"&gt;Carse of Gowrie&lt;/a&gt;. At first I wanted to make them for the large area around Tacloban but in the end it made more sense to just do a slightly smaller area:&lt;/p&gt;
        &lt;div class="video"&gt;
          &lt;div class="videoWrapper"&gt;
            &lt;iframe src="https://player.vimeo.com/video/80922315?title=0&amp;byline=0&amp;portrait=0" frameborder="0"/&gt;
          &lt;/div&gt;
          &lt;p&gt;&lt;a href="https://vimeo.com/80922315"/&gt;HOT: Tacloban (Typhoon Haiyan mapping)&lt;/p&gt;
        &lt;/div&gt;
        &lt;p&gt;I have used a slightly tweaked variant of my OSM progress software that you can find on GitHub: &lt;a href="https://github.com/derickr/osm-progress"&gt;https://github.com/derickr/osm-progress&lt;/a&gt; — I'll soon clean up my changes and push them to the repository as well.&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/advent13.html</guid>
        <pubDate>Fri, 13 Dec 2013 09:10:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Contributing Advent 12: Geospatial algorithms</title>
        <link>https://derickrethans.nl/advent12.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="contributing_advent_12_geospatial_algorithms"/&gt;Contributing Advent 12: Geospatial algorithms&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Thursday, December 12th 2013, 09:10 GMT&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;This is technically not a new contribution, but one I've been working on and off at for the past months. The &lt;a href="https://github.com/php-geospatial/geospatial"&gt;geospatial&lt;/a&gt; PHP extension contains a whole bunch of algorithms that deal with geospatial problems. The &lt;a href="http://en.wikipedia.org/wiki/Haversine_formula"&gt;Haversine&lt;/a&gt; and &lt;a href="http://en.wikipedia.org/wiki/Vincenty%27s_formulae"&gt;Vincenty&lt;/a&gt; functions calculate distances on a globe, an implementation of the &lt;a href="http://en.wikipedia.org/wiki/Helmert_transformation"&gt;Helmert transformation&lt;/a&gt; algorithm converts between different &lt;a href="http://en.wikipedia.org/wiki/Datum_%28geodesy%29"&gt;datums&lt;/a&gt;, and some general functions to convert from f.e. 16°45'12" N to 16.7533° N.&lt;/p&gt;
        &lt;img class="left" align="left" src="derickrethans.nl/images/content/advent2013-12a.png"/&gt;
        &lt;p&gt;These are all useful functions, but I'd like to talk about an algorithm that I've added some time ago now. It is an implementation of &lt;a href="http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm"&gt;Ramer-Douglas-Peucker&lt;/a&gt; that can be used to simplify a polyline. I have used that for the display of the timezone boundaries that I show in one of my maps examples: &lt;a href="http://maps.derickrethans.nl/?l=timezone&amp;lat=51.5&amp;lon=0&amp;zoom=8"&gt;http://maps.derickrethans.nl/?l=timezone&amp;lat=51.5&amp;lon=0&amp;zoom=8&lt;/a&gt;. The boundary data that I have for the UK for example, is so complex that it takes quite a bit of time to render them on the map. On higher zoom levels where that amount of detail simply does not show up, we can of course do with a simpler line, and that is what RDP helps us to achieve.&lt;/p&gt;
        &lt;p&gt;The algorithm has as input an array of coordinate pairs:&lt;/p&gt;
        &lt;pre&gt;$points = [
          [  0,  10 ],
          [  1,   6 ],
          [  3,   0 ],
          [  5,   6 ],
          [  7,   8 ],
          [  8,   7 ],
          [  9,   4 ],
          [  13, 10 ],
  ];

  &lt;/pre&gt;
        &lt;p&gt;As well as a parameter (called ε) that determines the maximum distance from a line that a point is allowed to be so that it is included in the returned array. The algorithm is recursive and will for each segment—starting with the whole array:&lt;/p&gt;
        &lt;ul&gt;
          &lt;li&gt;
            &lt;p&gt;Find the further point from the line as marked by the first and last point&lt;/p&gt;
          &lt;/li&gt;
          &lt;li&gt;
            &lt;p&gt;If the point is closer than ε to the line segment then any points not currently marked to keep can be discarded without the simplified curve being worse than ε.&lt;/p&gt;
          &lt;/li&gt;
          &lt;li&gt;
            &lt;p&gt;If the point furthest from the line segment is greater than ε from the approximation then that point must be kept. The algorithm recursively calls itself with the first point and the worst point and then with the worst point and the last point (which includes marking the worst point being marked as kept).&lt;/p&gt;
          &lt;/li&gt;
        &lt;/ul&gt;
        &lt;p&gt;With the above &lt;code&gt;$points&lt;/code&gt; array, we can run the function like:&lt;/p&gt;
        &lt;pre&gt;$points = rdp_simplify( $points, 2.75 );

  &lt;/pre&gt;
        &lt;p&gt;We are left with:&lt;/p&gt;
        &lt;pre&gt;$points = [
          [  0,  10 ],
          [  3,   0 ],
          [  5,   6 ],
          [  9,   4 ],
          [  13, 10 ],
  ];

  &lt;/pre&gt;
        &lt;p&gt;The image below shows the effect of &lt;code&gt;ε = 2.75&lt;/code&gt; and other values:&lt;/p&gt;
        &lt;img src="derickrethans.nl/images/content/advent2013-12b.png"/&gt;
        &lt;p&gt;In case you do not want to install a PHP extension to get that functionality, I also have an example implementation in PHP at &lt;a href="https://github.com/derickr/3angle/blob/master/rdp.php"&gt;https://github.com/derickr/3angle/blob/master/rdp.php&lt;/a&gt;, however, the time it takes to run the algorithm in PHP is about the same as the browser needs to draw the additional points of the lines.&lt;/p&gt;
        &lt;p&gt;Expect to see an updated version in the &lt;a href="https://github.com/php-geospatial/geospatial"&gt;geospatial&lt;/a&gt; extension that also handles multi-polygons in the future. This should speed up running this over the large amount of UK polygons, as there are plenty of islands that also need to be done.&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/advent12.html</guid>
        <pubDate>Thu, 12 Dec 2013 09:10:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Contributing Advent 11: Mozilla's geolocation services</title>
        <link>https://derickrethans.nl/advent11.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="contributing_advent_11_mozilla_s_geolocation_services"/&gt;Contributing Advent 11: Mozilla's geolocation services&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Wednesday, December 11th 2013, 09:10 GMT&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;About a months and a half ago Mozilla &lt;a href="https://blog.mozilla.org/services/2013/10/28/introducing-the-mozilla-location-service/"&gt;announced&lt;/a&gt; its &lt;a href="https://wiki.mozilla.org/CloudServices/Location"&gt;Location Services&lt;/a&gt;. It is "an experimental pilot project to provide geolocation lookups based on publicly observable cell tower and WiFi access point information". In other words, it is going to provide a similar service as Google's location service that Firefox currently uses for its &lt;a href="http://www.mozilla.org/en-GB/firefox/geolocation/"&gt;location aware browsing&lt;/a&gt;.&lt;/p&gt;
        &lt;img class="left" align="left" src="derickrethans.nl/images/content/advent2013-11a.png"/&gt;
        &lt;p&gt;In order to provide this service, they need geolocated data so that they know where WiFi access points and cell towers are. They have released a tool, &lt;a href="https://github.com/dougt/MozStumbler/releases"&gt;MozStumbler&lt;/a&gt;, that allows everybody to collect this data and provide it to Mozilla. I believe that there should be open alternatives for important services like this, and I think that Mozilla seems like a good caretaker of this data, respecting privacy. So I have installed &lt;a href="https://github.com/dougt/MozStumbler/releases"&gt;MozStumbler&lt;/a&gt; and turn it on whenever I go on a new route through town.&lt;/p&gt;
        &lt;img class="right" align="right" src="derickrethans.nl/images/content/advent2013-11b.png"/&gt;
        &lt;p&gt;The application isn't the prettiest as you can see, but it does work well. There is also a "Test Mozilla Location Services" which uses their API to lookup the location—without using the GPS. &lt;a href="https://mozilla-ichnaea.readthedocs.org/en/latest/"&gt;Ichnaea&lt;/a&gt; is the public API that you can use to do lookups yourself. The &lt;a href="https://mozilla-ichnaea.readthedocs.org/en/latest/api/geolocate.html"&gt;Geolocate&lt;/a&gt; API implements the same interface as the &lt;a href="https://developers.google.com/maps/documentation/business/geolocation/"&gt;Google Maps Geolocation API endpoint&lt;/a&gt;. But there is a more extensive &lt;a href="https://mozilla-ichnaea.readthedocs.org/en/latest/api/search.html"&gt;Search&lt;/a&gt; API as well. I will soon be upgrading &lt;a href="http://derickrethans.nl/projects.html#haunt"&gt;Haunt&lt;/a&gt; to use this new API as the one it currently uses is discontinued.&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/advent11.html</guid>
        <pubDate>Wed, 11 Dec 2013 09:10:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
      <item>
        <title>Contributing Advent 10: Dragons in Dollis Hill</title>
        <link>https://derickrethans.nl/advent10.html</link>
        <description>&lt;div class="article"&gt;
    &lt;div class="body"&gt;
      &lt;div class="articleListItem"&gt;
        &lt;h1&gt;&lt;a name="contributing_advent_10_dragons_in_dollis_hill"/&gt;Contributing Advent 10: Dragons in Dollis Hill&lt;/h1&gt;
        &lt;dl class="head"/&gt;
        &lt;div class="articleMetaData"&gt;
          &lt;div class="location"&gt; London, UK&lt;/div&gt;
          &lt;div class="date"&gt;Tuesday, December 10th 2013, 09:10 GMT&lt;/div&gt;
        &lt;/div&gt;
        &lt;p&gt;I had previous managed to clear out all the discrepancies on ITO World's Analysis for &lt;a href="http://www.itoworld.com/product/data/osm_analysis/area?name=Camden"&gt;Camden&lt;/a&gt;, &lt;a href="http://www.itoworld.com/product/data/osm_analysis/area?name=Kensington+And+Chelsea"&gt;Kensington and Chelsea&lt;/a&gt; and the &lt;a href="http://www.itoworld.com/product/data/osm_analysis/area?name=City+Of+Westminster"&gt;City of Westminster&lt;/a&gt;. But I have so far not yet managed to address all of the issues in my home borough &lt;a href="http://www.itoworld.com/product/data/osm_analysis/area?name=Brent"&gt;Brent&lt;/a&gt;. I guess this is mostly because the "issues" are not really in the nicest parts of the borough—perhaps there are even dragons there. But I am trying to make a point to get to them anyway, so I set out by bus to Dollis Hill to tackle &lt;a href="http://www.openstreetmap.org/browse/way/248940442"&gt;Grunwick Close&lt;/a&gt;, &lt;a href="http://www.openstreetmap.org/browse/way/67752051"&gt;Flowers Close&lt;/a&gt; and &lt;a href="http://www.openstreetmap.org/browse/way/4081584"&gt;St Andrews Close&lt;/a&gt;.&lt;/p&gt;
        &lt;p&gt;The first one, Grunwick Close was a new road and I couldn't find any street sign at all. I had to ask a local resident which indeed confirmed that I had the correct street name. The latter two actually had a "FIXME" tag on them, and no name tag. The first one, Flowers Close, was &lt;a href="http://www.openstreetmap.org/browse/way/67752051/history"&gt;previously&lt;/a&gt; incorrectly mapped as Flower Road (and OS Locator had it right), but the second one was wrong on OS Locator as it had thought there was an apostrophe in "Andrews"—the street sign disagreed.&lt;/p&gt;
        &lt;img class="right" align="right" src="derickrethans.nl/images/content/advent2013-10a.png"/&gt;
        &lt;p&gt;The &lt;a href="http://www.openstreetmap.org/browse/way/67254155/history"&gt;platform at Dollis Hill&lt;/a&gt; had this annoyingly nagging "FIXME" tag on it and I couldn't let that go unfixed of course. (Did I mention OpenStreetMap is addictive and that I have some form of OCD?) So I spend some time mapping all the aspects of the tube station: entrances, tunnels, stairs, platforms, tracks and stop positions—all following the "new" &lt;a href="http://wiki.openstreetmap.org/wiki/Proposed_features/Public_Transport"&gt;public transport scheme&lt;/a&gt;. The image here shows how the new station looks like on the map.&lt;/p&gt;
        &lt;p&gt;You can see it on the website at &lt;a href="http://www.openstreetmap.org/browse/relation/3350099"&gt;http://www.openstreetmap.org/browse/relation/3350099&lt;/a&gt;&lt;/p&gt;
        &lt;div class="patreon"&gt;
          &lt;a href="https://www.patreon.com/bePatron?u=7864328"&gt;
            &lt;img src="https://derickrethans.nl/images/become_a_patron_button.png" alt="Become a Patron!"/&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        
        
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  </description>
        <author>phpinternalsnews@derickrethans.nl</author>
        <guid isPermaLink="true">https://derickrethans.nl/advent10.html</guid>
        <pubDate>Tue, 10 Dec 2013 09:10:00 +0000</pubDate>
        <itunes:explicit>no</itunes:explicit>
      </item>
    </channel>
  </rss>


key: shtosm
encoding: utf8
format: rss 2.0
title: ШТОСМ
url: http://shtosm.ru/
feed_url: http://shtosm.ru/rss/
generator: E2 (v3254; Aegea)
twitter: shtosm
items_last_updated: 2019-10-26 14:32:01
fetched: 2019-10-26 18:30:23.006271
http_etag: f07c62f2c2bfb327be462e181ac1e4cd
http_last_modified: Sat, 26 Oct 2019 18:17:41 +0000
http_server: Apache/2.4.10 (Fedora) OpenSSL/1.0.1e-fips PHP/5.5.26 mod_wsgi/3.5 Python/2.7.5
md5: d954b9f3255100c4213cd3b15251bb15
created_at: 2019-06-09 18:11:33.488998
updated_at: 2019-10-26 18:30:23.008346
body:
  <?xml version="1.0" encoding="utf-8"?> 
  <rss version="2.0">

  <channel>

  <title>ШТОСМ</title>
  <link>http://shtosm.ru/</link>
  <description></description>
  <generator>E2 (v3254; Aegea)</generator>

  <item>
  <title>Главный вопрос OpenStreetMap</title>
  <guid isPermaLink="false">1507</guid>
  <link>http://shtosm.ru/all/glavny-vopros-openstreetmap/</link>
  <comments>http://shtosm.ru/all/glavny-vopros-openstreetmap/</comments>
  <description>&lt;p&gt;&lt;i&gt;Сейчас в Призрене, Косово, идёт конференция State of the Map SEE (юго-восточной Европы). Видеозаписей не будет: она по-настоящему локальная, для той пары сотен человек, кто понимает английский. После моего доклада появились несколько идей, которые стоило записать. Если хотите контекст — следите за телеграм-каналами &lt;a href="https://t.me/foss4g_ru"&gt;@foss4g_ru&lt;/a&gt; и &lt;a href="https://t.me/shtosm"&gt;@shtosm&lt;/a&gt;.&lt;/i&gt;&lt;/p&gt;
  &lt;p&gt;Рассказал про OpenStreetMaps во множественном числе. Втопил за многоязычность и разнообразие среди участников — и понимание, что у нас не одна OpenStreetMap с единым всемирным сообществом, а тысяча отдельных OSM с тысячью маленьких и больших сообществ, у каждого из которых свои практики тегирования и мнения о полноте карты. Я не понимаю, как здесь, в Косово, отметят свои кебабные рестораны на карте их владельцы, когда они не могут даже узнать про OSM (сайт этой конференции — только на английском) и узнать его правила (перевода вики на албанский нет).&lt;/p&gt;
  &lt;p&gt;Сейчас с Феликсом классно обсудили эти наблюдения. Спросил у него про нападки на Missing Maps. Европейцы замечали, что от всей этой инициативы веет колониализмом: белые мужчины решают, как выглядеть карте Африки и Азии. С другой стороны, карта колониальна по своей природе: ты идёшь в соседний двор и решаешь, как ему выглядеть на карте, что должно там быть и что нет. Карта — это власть; нельзя делать карту и не иметь власти. Цель подобных конференций — поделиться этой властью, дать её как можно большему количеству людей.&lt;/p&gt;
  &lt;p&gt;Но, опять же, среди слушателей нет владельцев кебабных. Они не поймут доклады, хотя среди них есть очень доходчивые ликбезы, потому что они на английском. Нужно ли им это? Закрадывается странная мысль, что может, пусть они жарят свои кебабы, карта им ни к чему. Поэтому их нет в OSM. Это опасно походит на поверье десятилетней давности, что женщин в технических профессиях мало, потому что это не женское дело, это им не интересно. Стыдная мысль, даже писать её неловко.&lt;/p&gt;
  &lt;p&gt;Другое, о чём мы задумались, — OpenStreetMap как средство получения и передачи знаний. Ты не рисуешь домики, а узнаёшь, как устроена карта, как проекция влияет на прямоту углов, как спутниковые снимки соотносятся с картой. Ты получаешь важные знания и даже не обращаешь на это внимание. Потенциальные картографы в деревнях, охваченных Missing Maps, лишаются этого: всё нарисовано за них. Для компаний и кампаний OpenStreetMap — это данные. Нужно больше данных, нужны свежие данные. Но людям нужны не данные, а знания. Люди — особенно люди из далёких стран — предпочтут научиться, предпочтут освоить удочку, а не получить рыбу. Закрашивая карту из лучших побуждений, мы лишаем этих людей знания.&lt;/p&gt;
  &lt;p&gt;Важная часть обучения картографии — ошибки. Придя в проект, ты рисуешь кривые домики, выбираешь глупые теги (я помню про свои &lt;i&gt;juridical_faces&lt;/i&gt;) и забываешь соединить улицы. Потом понимаешь, в чём ошибка. Придумываешь свои теги, обращаешь внимание на мелочи, которые не интересуют других: на полосы обгона, кусты вдоль дороги, люки. Быть в OpenStreetMap на заре его становления было классно: ты двигался наощупь и узнавал многое в процессе.&lt;/p&gt;
  &lt;p&gt;Сейчас если ты сделаешь ошибку, тебе в личку быстро придёт кто-то из сотни валидаторов и сообщит о ней. Может, даже отругает. Всё стандартизировано, везде валидаторы. Карту используют в Facebook, Mapbox, Microsoft, Apple. Она должна быть однообразна и корректна. У современных новичков в OSM нет права на ошибку. Они приходят и становятся частью даже не сообщества, а армии картографов, где всё должно быть ровно, параллельно и покрашено в уставные цвета. Из проекта уходит игра и приходят задачи. А игра — это лучший способ получить знания. Корпорации и опытные картографы лишают участников знаний в обмен на большие и качественные данные.&lt;/p&gt;
  &lt;p&gt;Непонятно, что с этим делать. Возможно, это лишь моментное чувство, а OpenStreetMap, как мы знаем, не оперирует моментом: на него нужно смотреть через десять, пятьдесят лет. ML и AI, которые никогда не приносят новых знаний, как пришли, так и уйдут. Может, мы поймём, что OSM идёт не туда, и сделаем альтернативу. Может, мы запретим массовые правки. Или запретим препятствовать им. В ближайшие годы нам придётся ответить, &lt;i&gt;что такое OpenStreetMap&lt;/i&gt;. Данные или карта? Игровая площадка или галеры? Средство общения или проект? Польза или сообщество? Наш совместный ответ определит будущее проекта — и, конечно, оно не понравится никому в любом случае.&lt;/p&gt;
  </description>
  <pubDate>Sat, 26 Oct 2019 18:32:01 +0400</pubDate>
  </item>

  <item>
  <title>Не согласны делиться</title>
  <guid isPermaLink="false">1506</guid>
  <link>http://shtosm.ru/all/ne-soglasny-delitsya/</link>
  <comments>http://shtosm.ru/all/ne-soglasny-delitsya/</comments>
  <description>&lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/osm_reg_header.png" width="700" height="200" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;С первого октября в рассылке osmf-talk люди обсуждают public domain. Сначала Северин &lt;a href="https://lists.openstreetmap.org/pipermail/osmf-talk/2019-October/006208.html"&gt;спросил&lt;/a&gt;, можно ли как-то снять галочку, поставленную напротив согласия с PD при регистрации. Ему быстро напомнили, что эта галочка — не юридически значимое согласие, а, скорее, опрос мнений. Не нужно делать второй аккаунт со снятой галочкой или писать пул-реквест в код сайта: твоё мнение и так ничего не решает.&lt;/p&gt;
  &lt;p&gt;Чтобы получить значимость, галочка должна сопровождаться длинным списком условий. Например, что пользователь обязуется не использовать источники не в общественном достоянии (такие как спутниковые снимки или открытые данные в РФ). И что он не основывает свои правки — по части геометрии или тегов — на данных других пользователей OSM, не отдающих свои правки в public domain.&lt;/p&gt;
  &lt;p&gt;Нельзя отрицать, что права на свой вклад готова отдавать значительная часть сообщества: по &lt;a href="https://blog.openstreetmap.org/2013/10/15/the-pd-checkbox/"&gt;данным 2013 года&lt;/a&gt;, 44% редакторов с хотя бы одной правкой поставили эту галочку. 580 участников с личной страницей в нашей вики поставили на неё &lt;a href="https://wiki.openstreetmap.org/wiki/Category:Users_whose_contributions_are_in_the_public_domain"&gt; шаблон PD-user&lt;/a&gt;, который в более строгих терминах разрешает использовать их правки. Правда, людей в вики формально нельзя связать с редакторами в API. А эта галочка… Многие признают, что она выглядит как стандартная «правила не читал, но согласен», свойственная многим формам регистрации, поэтому люди нажимают на неё непроизвольно и не понимая, чем это может быть чревато.&lt;/p&gt;
  &lt;p&gt;Так что участники пришли к консенсусу: галочку нужно убрать из формы регистрации. Это не должно быть проблемой, потому что список поставивших её недоступен никому, кроме OSMF, и потому никто его не использовал. Фредерик послал официальный запрос в LWG. Участники уже начали заглядывать в код сайта, чтобы понять, насколько сложно будет вычистить оттуда public domain. И нашли там смешное.&lt;/p&gt;
  &lt;p&gt;Оказалось, что где-то между 2010 и 2015 годами значение галочки про public domain перестало сохраняться в базу данных. Мы не знаем, кто с тех пор был согласен, а кто нет. Команду сохранения &lt;a href="https://github.com/openstreetmap/openstreetmap-website/commit/898c47e647fbffcd0543b78033945b04f1b9aeae#diff-99cb709286f806f8c8b24ea88bc311afL212"&gt;случайно удалили&lt;/a&gt; во время добавления на сайт условий участия. Поскольку сайтом у нас заведует единоличный Том, никакого code review его коммита не было, строчка просто исчезла.&lt;/p&gt;
  &lt;p&gt;Вслед за анонимными комментариями, ещё один привычный элемент сайта, до которого никому не было дела, скоро уберут. Данных OpenStreetMap в общественном достоянии не будет — но, может быть, пора подумать над вариантами.&lt;/p&gt;
  </description>
  <pubDate>Wed, 09 Oct 2019 14:41:44 +0400</pubDate>
  </item>

  <item>
  <title>Рисование карты по Канту</title>
  <guid isPermaLink="false">1504</guid>
  <link>http://shtosm.ru/all/kant-and-openstreetmap/</link>
  <comments>http://shtosm.ru/all/kant-and-openstreetmap/</comments>
  <description>&lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/summit2019.jpg" width="1148" height="492" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Прочитал на «репаблике» краткий &lt;a href="https://republic.ru/posts/94582"&gt;пересказ&lt;/a&gt; тезисов Канта в отношении морали и этики и понял, чем мне не нравится инициатива Missing Maps. Они подменяют категорический императив гипотетическим, этим вредя проекту, перенося его в статус необязательного.&lt;/p&gt;
  &lt;p&gt;Многие из нас рисуют карту просто потому что могут. «Картировать должен каждый» — эта фраза, в которой обычное «может» подменяется императивом «должен», руководит нашим выбором. Особенно если подменить слово «картировать» на «улучшать OpenStreetMap», где первое слово обозначает всё: и программирование, и написание текстов, и организацию мероприятий. Это категорический императив: он ничем не обусловлен и мир стал бы лучше, если бы им руководствовался каждый.&lt;/p&gt;
  &lt;p&gt;Тут приходят активисты Missing Maps и говорят: «рисуйте карту — она спасёт жизни!» На призыв откликаются тысячи людей, которые очень быстро наполняют данными белые пятна в OpenStreetMap. Проекты гуманитарной команды один полезнее другого: где помогают оценить ущерб от наводнения, где — спрятать девочек от обрезания, в Филиппинах превентивно закартировали всю страну не только для экстренного реагирования, но и для госпроектов. Всех людей в гуманитарной команде объединяет наличие цели: они не рисуют карту, а помогают людям. Для них карта утилитарна, картировать — это гипотетический императив.&lt;/p&gt;
  &lt;p&gt;По Канту, следование гипотетическому императиву легально, то есть, полезно и правильно, но не морально. Этически развитому человеку не нужны причины, он следует долгу, и если он знает про открытую карту, то улучшает её, не придумывая, зачем он (или она) это делает. При этом, не имеет значения, OSM рисует человек или «Народную карту»: то и то считается моральным, потому что позволяет реализовывать свой долг его описания. Способ реализации не важен, покуда действие является добром для мира. Поэтому я когда-то вполне мог оказаться в НЯК и рассказывать по всему миру о нём, а не об OpenStreetMap.&lt;/p&gt;
  &lt;p&gt;Если люди рисуют карту одинаково продуктивно, руководствуясь и категорическим, и гипотетическим императивами, то какая разница? По Канту, поступок, сделанный не из долга, несовершенен. С точки зрения опытных участников проекта OSM разница налицо: имея целью не выполнение долга по улучшению карты, а стороннюю цель («спасаем жизни!»), люди на самом деле не улучшают карту, а вредят ей. Клацают кривые домики «на отвяжись», засоряют города линиями без тегов и дорогами невпопад. Ещё не сформировавшимся местным сообществам придётся не рисовать карту с нуля, как мы любим, а разгребать чужие ошибки. И это мы ещё не говорим про автоматическое рисование.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/missingmaps19.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Проект Missing Maps невероятно эффективен в привлечении редакторов. Но его существование — проблема для проекта, равно как и наоборот. В 2016 году мы спорили насчёт термина craft mappers, которым Майкл Мигурски &lt;a href="http://mike.teczno.com/notes/openstreetmap-at-a-crossroads.html"&gt;окрестил&lt;/a&gt; людей, препятствующих развитию OSM. Кажется, философия Канта объясняют, в чём именно он был неправ: craft mappers — это люди, для которых улучшение карты является категорическим императивом. Этот императив требует от них контроля и абстрагирования от реальных задач для карты. Их сложно понять тем, у кого карта &lt;a href="https://www.facebook.com/darafei/posts/10217965825364698"&gt;не входит&lt;/a&gt; в число абсолютных ценностей.&lt;/p&gt;
  &lt;p&gt;Это не значит, что Missing Maps нужно закрывать, или что их мероприятий нужно избегать. Напротив. У лекторов проекта можно научиться заражать своим видением мира других, а затем помочь участникам не сильно портить карту. Они дают шанс подумать, что станет с OSM, когда о нём действительно узнает каждый. В субботу 12 октября в Москве рядом с метро «Белорусская» состоится первый марафон Missing Maps в России. &lt;a href="https://vrachi-bez-granits-m-deci.timepad.ru/event/1070614/"&gt;Сходите на него&lt;/a&gt; и побудьте ответственным картографом, записавшись в валидаторы и проверяя правки новичков.&lt;/p&gt;
  </description>
  <pubDate>Fri, 04 Oct 2019 20:01:59 +0400</pubDate>
  </item>

  <item>
  <title>Желаем того, что имеем</title>
  <guid isPermaLink="false">1503</guid>
  <link>http://shtosm.ru/all/zhelaem-togo-chto-imeem/</link>
  <comments>http://shtosm.ru/all/zhelaem-togo-chto-imeem/</comments>
  <description>&lt;p&gt;Пока готовился к докладу, прослушал много попыток представить API 0.7 другими людьми, особенно &lt;a href="https://2018.stateofthemap.org/2018/T107-Modding_the_OSM_Data_Model/"&gt;Йохеном Топфом&lt;/a&gt; и &lt;a href="https://2015.stateofthemap.us/some-osm-futures"&gt;Энди Алланом&lt;/a&gt;. И прочитал, конечно, &lt;a href="https://wiki.openstreetmap.org/wiki/API_v0.7"&gt;страницу в вики&lt;/a&gt;. Предложений много. Самые радикальные, но с которыми сложно не согласиться, проходят по разряду «наша модель данных сложная, давайте приблизим её к Simple Features». Специальный тип для полигонов, например. Но правильно ли мы хотим?&lt;/p&gt;
  &lt;p&gt;У нас же всё есть. Мы уже умеем отличать закрытые линии от периметров, умеем разбирать мультиполигоны. Зачем упрощать, когда незачем упрощать? То же самое в предложении Йохена убрать точки без тегов и добавить линиям координаты — по сути, собственную геометрию. Да, сейчас кэширование координат ест память, собирать объекты неудобно. Но кто сказал, что должно быть удобно потребителям данных? Удобно должно быть редакторам, потому что OpenStreetMap — проект для редакторов. С их позиции всё отлично: тайлы рендерятся, данные в базу загружаются, osmium вообще быстрый стал.&lt;/p&gt;
  &lt;p&gt;Технарям, которые пишут софт, право голоса ни к чему. У них есть рычаг, коего лишены большинство участников проекта: они умеют писать софт. А остальным с моделью данных, на которую они пытаются повлиять, придётся жить. И ни разу не очевидно, что жизнь станет проще: мало ли, разрезать окружности напополам станет сложнее, или потребуется указывать, насколько важна каждая точка. Делать лишний клик на перекрёстках. Вся нужная информация уже есть в базе, а память и хранилища дешевеют быстрее, чем мы рисуем новые объекты.&lt;/p&gt;
  &lt;p&gt;Разумеется, API всё равно устарел и его нужно менять. Только смотреть не туда, куда показывают натруженные пальцы программистов. А туда, где болит конкретно у вас, картографов. Вандалы замучали? Давайте думать, что нам упростит поиск проблемных правок и их откат. Лень тратить время на исправление тупых ошибок вроде линий из одной точки? Посмотрим, какую валидацию можно встроить в обработчики загрузки данных. Не выбрать между landuse=forest и natural=wood? Давайте запретим один из них на уровне API.&lt;/p&gt;
  &lt;p&gt;И ещё пора перестать делать вид, что Overpass API — сторонний проект, никак не связанный с OSM API. На минуточку, в вики более сорока страниц про первый и всего одна про второй. OSM API не находится в вакууме, это интерфейс доступа к данным не лучше и не хуже других. Поскольку OSM децентрализован, то ничего удивительного, что API разбросан не только по нескольким странам, но и по нескольким способам доступа. Overpass API — один из них, и он решает очень много пожеланий к API 0.7. У него одна проблема: в отличие от Taginfo, Nominatim или форума он не установлен на серверах OSMF, и поэтому его сложно воспринимать всерьёз участникам проекта. Даже несмотря на то, что сервисов, написанных на Overpass, больше, чем использующих API. Ваш API 0.7 — это Overpass API, смиритесь с этим и думайте, какие особенности OSM API ограничивают все способы работы с данными.&lt;/p&gt;
  &lt;p&gt;Я поставлю на &lt;a href="https://www.youtube.com/watch?v=MKLwLI8fyn0"&gt;отказ от пакетов правок&lt;/a&gt;: как метаданные, они лучше расстановки одинаковых тегов на всех объектах, но не оправдывают никаких ожиданий. Запрос для скачивания содержимого пакета, а не только тегов, должен быть выкорчеван из API, чтобы не подавать странных идей. И поставлю на лучшие связность и версионирование: не могу не согласиться с Йохеном, что когда внутри одной версии у линии может быть несколько разных геометрий, это дезавуирует всю систему версионирования. Нельзя откатить линию или отношение к прошлой версии, потому что это вызывает вопросы, к каким версиям откатывать их члены. Так мы возвращаемся к API 0.5, когда история адресовалась по меткам времени. Поэтому специфику OSM, «всё связано со всем», нужно подчеркнуть указанием ссылок на родительские элементы, и версии делать либо составными, либо увеличивать на каждый чих.&lt;/p&gt;
  &lt;p&gt;В докладе на State of the Map, помимо &lt;a href="https://t.me/shtosm/292"&gt;проблем комментариев к правкам&lt;/a&gt;, я упомянул, что неплохо бы добавить способ выкачивания полной истории для любого объекта, включая историю его членов. И получение удалённых объектов для заданной области. Которое у нас уже есть для Potlatch 1, но не для остальных редакторов, несмотря на &lt;a href="https://github.com/openstreetmap/openstreetmap-website/pull/1448"&gt;попытки Фредерика&lt;/a&gt;. Да, это частично умеет Overpass API, но работа с исходной базой данных поможет получить результат быстрее и помочь в выкапывании истории другим инструментам, которые не приняли Overpass как нашего отца и спасителя. Наконец, задумавшись об истории в API, мы сможем сделать ещё один маленький шажок и добавить в него метод /revert. А затем и соответствующие кнопочки на сайт.&lt;/p&gt;
  &lt;p&gt;Нам не нужно упрощать данные. Нынешние инструменты отлично справляются с преобразованием их в удобные другим людям форматы. Нам нужно подумать, чего мы лишены. И доделать это: изменить инструменты мониторинга или поправить модель данных, чтобы понять историю нашей карты.&lt;/p&gt;
  </description>
  <pubDate>Fri, 27 Sep 2019 12:29:16 +0400</pubDate>
  </item>

  <item>
  <title>Гейдельберг 2019: понедельник</title>
  <guid isPermaLink="false">1502</guid>
  <link>http://shtosm.ru/all/geydelberg-2019-ponedelnik/</link>
  <comments>http://shtosm.ru/all/geydelberg-2019-ponedelnik/</comments>
  <description>&lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/morning-rain-heid.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;С утра дождь, ровно по расписанию, которое я посмотрел неделю назад. Но это же Гейдельберг, тут такое безобразие долго продолжаться не может: к обеду уже снова солнце. Погода тут простая: если солнце, то жара. Если нет, то подмерзаем. На ужине Кристина посетовала, что я пишу сплошь плохие вещи про Германию. Возможно: моя планка ожиданий была очень высока, потому что мне в четырнадцатом году понравился Карлсруэ вокруг SotM EU, и я надеялся, что тут ещё приятнее. А тут — обычно. Красивый старый город, которому повезло пережить войну, красивые горы всегда на фоне. Тропа философов вообще лучшая: будто за полчаса переносишься в совсем другой мир с другим климатом. Везде царит немного южный беспорядок, но он контейнирован, его держат в рамках немецкого приличия. Ездить на автобусах и трамваях с комби-билетом, который подарили каждому участнику конференций, — сплошное удовольствие: доехать можно всюду, пересадки не вызывают такого раздражения, как в России. За пять дней этот билет, правда, не контролировали ни разу.&lt;/p&gt;
  &lt;p&gt;Вечером скатался в район новостроек, и там всё для жизни: дома от четырёх до семи этажей, у каждого свой дворик, через район провели искусственный канал, рядом магистральная велодорожка. Не страшно выпускать детей: движения почти нет, зато есть где поиграть. Когда достроят центральную площадь, вокруг неё явно будет жизнь, и она будет приятной. Все здания практичны, нет моментов разочарования, как во многих других странах: «ну блин, как можно было об этом не подумать». Входы на уровне земли, никакой глупой фигни вроде заборчиков вокруг газонов или стихийной парковки у входов. Проекты так близки к идеалу, что даже тошно. Никаких приключений: вот тебе спокойная жизнь, вот тебе трамвай до работы, вот тебе гора, в которую каждый день нужно бежать.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/heid-neue-night.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Поехал с утра послушать первые доклады, но как всегда, залип с кем-то поговорить в холле. Даже не помню, с кем. Понимают ли мои собеседники, что если бы не бейджики, я бы на всей конференции опознал бы всего человек десять, и не понимал бы, за какую работу благодарить остальных? Завидую тем, кто запоминает лица. «Кажется, ты меня не узнал» — с улыбкой сказал парень, которого я вечером встретил в ресторане у кинотеатра. Это был Давид из Чехии, с которым мы делали редактор маршрутов транспорта в рамках Google Summer of Code, созваниваясь по видео каждые выходные. Он приехал на SotM впервые и стеснялся вписываться в разные компании, прямо как я когда-то. Потому и сидел в ресторане вместо того, чтобы бегать по городу с Квинси, Йостом, Джанет, Мингом и остальной шайкой.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/roland-proposals.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;На сегодня Роланд (автор Overpass API) запланировал большое обсуждение проблем предложений новых тегов. Пропозалов, то есть. Он подготовил табличку 4×2: что желательно иметь и чего избегать, что реально изменить и что нет. Дошли мы едва до третьей ячейки: у каждого из сорока человек в зале были свои мысли. Многие топили за диктатуру: какую-нибудь рабочую группу по тегированию, которая будет руководить обсуждениями. Чтобы искала людей с нужным опытом, проверяла адекватность и имела решающий голос. Вот только кого в неё выбрать, чтобы никто не возражал, и как избежать обычного для рабочих групп выгорания через год-два? Сейчас эту группу как бы составляют авторы популярных редакторов: именно они решают, что попадает в заготовки, которыми пользуются почти все картографы.&lt;/p&gt;
  &lt;p&gt;Думали про технические решения, стараясь не скатываться в совсем дебри типа модулей для вики. Например, явно стоит посмотреть в базе, кто уже пользуется предложенной схемой. Принятым схемам не хватает указания потребителей: люди же обычно рисуют карту не просто чтобы наполнить базу данных, а чтобы их объекты было видно на сайте или в приложении. &lt;a href="https://taginfo.openstreetmap.org/projects"&gt;Проекты Taginfo&lt;/a&gt; — неплохое начало, но на практике пользоваться ими невозможно: что из того, что OsmAnd поддерживает теги &lt;i&gt;highway=*&lt;/i&gt;? Вики-страницы про теги плохи тем, что каждый их меняет как хочет, неожиданно и не всегда логично. Какая-нибудь система с RFC, запросами на изменение, могла бы помочь. А то и вообще вынести все схемы тегирования в отдельный специализированный проект.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/roland-audience.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Потеряв счёт времени, едва успел на короткие доклады. Гильем Ришар, сидевший рядом, тоже чуть не опоздал делать первый из них. А там, как водится, технические проблемы, люди куда-то пропадают, прошло уже десять минут, а люди до сих пор жонглируют флэшками и ноутбуками.&lt;/p&gt;
  &lt;ul&gt;
  &lt;li&gt;Джерри про почтовые ящики. Как всегда, он выдал порцию офигительных историй про атаки суфражисток, взорванные ящики из-за неправильных гербов и разноцветные ящики просто потому что. Для тегирования ящиков применяют полтора десятка тегов, и это нормально: у них действительно столько атрибутов.&lt;/li&gt;
  &lt;li&gt;Тьерри из OSM Brazil. Из презентации я узнал, что у правительства Франции под-домен &lt;i&gt;gouv.fr&lt;/i&gt;, а не как мы привыкли. Автор узнал про инициативу сбора адресов во Франции (проект Bano), и захотел то же самое в Бразилии. Стучится теперь в свои муниципалитеты.&lt;/li&gt;
  &lt;li&gt;Майкл Райхарт показал слои в OSM Inspector: дублированные линии, «острова» роутинга и неприсоединённые линии, которые теоретически должны быть соединены.&lt;/li&gt;
  &lt;li&gt;Гильем прокладывает маршрут в OSRM и делает из него отношение маршрута автобуса. Но предпочёл бы, чтобы этой фигнёй не приходилось заниматься, и для отношения маршрута хватало бы списка остановок.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/jerry-postboxes.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;В большом зале после обеда устроили серию докладов про проекты по заказу SNCF, французских железных дорог. Сначала Антуан и ещё один парень показали, как непроста пешеходная маршрутизация по вокзалам. В первом приближении всё просто: рисуешь &lt;i&gt;footway&lt;/i&gt; и заводишь роутер. Но платформы отрисованы полигонами: нужно делать роутинг по площадям. Повезло, что масштабы невелики, можно тупо соединить рёбрами все углы и точки входов-выходов. Указатели и запреты делают известными отношениями &lt;i&gt;destination&lt;/i&gt; и &lt;i&gt;restriction&lt;/i&gt;. Наконец, в помещениях не избежать тегов &lt;i&gt;indoor=*&lt;/i&gt;. Таким образом оформили уже 83 станции, маршруты всех устраивают. Всё тегирование они &lt;a href="https://wiki.openstreetmap.org/wiki/Guidelines_for_pedestrian_navigation"&gt;описали в вики&lt;/a&gt;.&lt;/p&gt;
  &lt;p&gt;Но пешеходу непросто следовать указаниям вида «через 300 метров поверните направо». Фредерик Родриго далее повторил свой старый рассказ про использование опорных POI типа кафе и магазинов для аннотирования пешеходных маршрутов. «За макдональдсом направо» куда проще запомнить и найти.&lt;/p&gt;
  &lt;p&gt;Сотрудники SNCF Грант и Денис втопили про маршрутизацию по железным дорогам. То есть, то, что в 2012 году &lt;a href="http://shtosm.ru/2012/11/11/1/"&gt;делал Дмитрий&lt;/a&gt;, но чему мы тогда не придумали применения. А в SNCF придумали: рисовать красивую линию в приложении и матчить GPS-треки с поездов (точность 10 см!) к рельсам для аналитики. Попросили Geofabrik написать модуль к движку GraphHopper, почистили данные от рельсов с &lt;i&gt;service=*&lt;/i&gt; и &lt;i&gt;usage=branch&lt;/i&gt;, подлатали дорожную сеть — и ура, работает по всей Франции. Поскольку опорные точки часто стоят на здании вокзалов, алгоритм не всегда понимает, какой из путей выбрать, поэтому во избежание странных маршрутов с разворотами приходится понижать точность. Конечной целью работы авторы заявляют роутинг от двери до двери: пешком до остановки, на метро до вокзала и на поезде между городов.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/sncf-rail-routing.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;В другом зале Виктор с Женей хвастались маршрутизацией по общественному транспорту в OsmAnd. Задумали её год назад на прошлом сотме, где я топил в трёх разных местах про новую схему общественного транспорта. Тем смешнее было видеть «NEW public transport schema PTv2». Ей в мае десять лет исполнилось, она старее API 0.6, вы чего. Главное, конечно, что у авторов всего за полгода получилось класть нормальные маршруты по вермишели отношений, которую никто никогда не валидировал. Это куда сложнее сделать, чем переписать потом на более адекватную и &lt;a href="https://wiki.openstreetmap.org/wiki/Proposed_features/Refined_Public_Transport"&gt;современную схему&lt;/a&gt;. Принять эту схему — вот это действительно сложная задача. На фоне понедельничных докладов я решил, что хватит это терпеть: сделаем с Гильемом, Джозефом и ещё парой ребят рабочую группу и сообща пропихнём.&lt;/p&gt;
  &lt;p&gt;Грегори подытожил конференцию фотографиями с этих трёх дней и вызвал на сцену всех добровольцев, помогавших с организацией. Их было человек тридцать. Примерно понимаешь сложность и масштаб организации: в этом году зарегистрировались 570 человек, и ещё человек сто были бы рады приехать дополнительно. Мы смеялись над странными обедами, но пойди прокорми такую толпу. В Милане и Сиэттле обошлись пакетами с салатами и бутербродами, тут же хоть горячий суп обеспечили. И на этом фоне рабочая группа SotM предлагает думать над проведением конференции в вашем городе через два-три года. Сложновато. Аплодирую смелости и инициативности ребят из Кейптауна, которые заявили и выиграли право провести &lt;a href="https://wiki.openstreetmap.org/wiki/State_of_the_Map_2020"&gt;State of the Map 2020&lt;/a&gt;. Жду не дождусь своей первой поездки в Африку.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/sotm19-is-over.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  </description>
  <pubDate>Tue, 24 Sep 2019 14:24:28 +0400</pubDate>
  </item>

  <item>
  <title>Гейдельберг 2019: воскресенье</title>
  <guid isPermaLink="false">1501</guid>
  <link>http://shtosm.ru/all/geydelberg-2019-voskresenye/</link>
  <comments>http://shtosm.ru/all/geydelberg-2019-voskresenye/</comments>
  <description>&lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/mapping-trees.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;У нас есть друзья, которые переехали в Германию и теперь грустят — от другой культуры, языка, от непонятных процессов и будто враждебной среды. Их можно понять. Германия предоставляет какой-то базовый уровень комфорта, но это не значит, что можно расслабиться. Нужно выдавать результаты на более высоких уровнях. Все бегуны, кого я тут видел, бежали в гору. Фредерик развернул огромную карту университета, и я с ужасом обнаружил, что он уточняет размеры и виды каждого дерева на территории. А тут их целый лес! Всегда нужно быть начеку: от тебя ожидают, что ты знаешь все расписания и все возможные проблемы. Немцы постоянно в курсе, что нужно хотя бы раз в месяц заглядывать в захламлённый подвал невзрачного административного здания, в неприметную дверь с плакатом «осторожно, леопард!»: там могут быть важные документы. Здесь нет неожиданностей, но есть неосведомлённость. Непонятно, что хуже.&lt;/p&gt;
  &lt;p&gt;Воскресенье — чудовищный день, потому что у меня сразу два доклада, и для основного ещё нужно допилить слайды. Поэтому решил выйти попозже, сначала потренировавшись пару раз. Смотрю в расписание — а у меня рассказ про штосм в 10:30.  Делать нечего — пришлось садиться на трамвай. Короткие доклады были сплошь демонстрациями: Саид показал валидатор распознанных знаков в Mapillary, Беата — модуль ImproveOSM для JOSM, Алина — аннотирование снимков в OpenStreetCam. Я же призвал людей перестать считать за достижения только код (как везде на State of the Map) или только картирование (как везде на HOT Summit). Было бы здорово почитать мысли участников, а для этого нужно, чтобы они остановились, подумали и написали в дневнички или бложики.&lt;/p&gt;
  &lt;p&gt;Чуть позже Томас рассказал про приватность в сети, про то, что ваши координаты могут рассказать про вас. И, конечно, первое приложение, которое появляется в слайдах, — maps.me. Он покликал в точки, подвигал карту. Смотрим логи: что ушло на сервер? В фейсбук ушли название телефона, ширина экрана и уровень громкости, в mopub — идентификатор мобильного оператора. При клике на гостиницу букинг узнаёт, куда вы кликнули. В приложении можно отключить статистику и за деньги отключить рекламу. Последнее заметно сократило список серверов, куда отсылали данные, — до трёх. Хорошо бы убрать всё, кроме, разве что, crashlytics. Османд тоже что-то спрашивает у своего сервера, но это терпимо. По мне, любой продвинутой карте рано или поздно приходится заводить серверную часть.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/concept-of-junction.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Наконец, Мартин из CycleStreets в весёлых скриншотах нашей карты показал, как офигевают от нашей модели рисования и тегирования велосипедных полос и дорожек люди, которым это всё использовать. Сколько светофоров на перекрёстке? Как его проезжать, если не отрисован переход? В каком месте развязки предупреждать, что пора поворачивать направо? Кажется, починить рисование велосипедной и пешеходной инфраструктуры можно только введя виртуальные полигоны, ограничивающие дороги и все прилегающие элементы типа парковок и тротуаров.&lt;/p&gt;
  &lt;p&gt;Все остальные доклады я пропустил: готовился к своему выступлению. Ни одной картинки на слайде, только ключевые слова из размышления, что мы можем сделать для облегчения слежения за правками и их отката. Кажется, получилось вдохновляюще: Кристина передала, что некоторые участники называли мой доклад самым важным на конференции. Осталось только найти людей, которые готовы придумать, как нам всё это сотворить. На шинковку пакетов правок нужны не только ресурсы, но и понимающие люди, способные составить план. Может, рассказ зажжёт сердце какого-нибудь сотрудника фейсбука или граба, и тот продавит руководство на новый полезный проект. На неделе опубликую основные мысли доклада, а запись уже доступна, как и &lt;a href="https://media.ccc.de/c/sotm2019"&gt;все остальные&lt;/a&gt; с конференции.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/mathematicon-poster.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;После докладов — Poster Session: всех пригласили в соседнее здание, где развешены плакаты про OpenStreetMap. Карт среди них печально мало: три или четыре из примерно пятнадцати. Остальное — научные исследования или просто наблюдения. Как плакат с шестью видами адресации в Великобритании. Или пара плакатов от YouthMappers: они много слышали про то, что если картографов-новичков вдохновить, то мероприятие получится эффективнее. Ну как в Missing Maps всегда вначале рассказывают, что картой вы спасаете жизни, вот это всё. YM проверили это утверждение, поделив новичков на две группы. Оказалось, вдохновление поднимает мораль, но не делает из плохих картографов хороших. Всё это описано в &lt;a href="https://zenodo.org/record/3405431"&gt;материалах&lt;/a&gt; академической секции, на предпоследней странице. Постер-сессия удобнее обычного разглядывания плакатов: рядом стоят люди и объясняют всё, что сложновато.&lt;/p&gt;
  &lt;p&gt;За пивом с рыбным канапе в углу разговорил мужика из какой-то страны в средней Азии. «Что я могу рассказать на конференции, вон пусть люди, которые делают картовстречи, говорят». Разумеется, за следующие десять минут я услышал от него потрясающие рассказы про картирование близ военных частей и как он однажды учил работе с картами местную армию. То же с Джерри Кло: стоит его поймать, как байки из его профессорского опыта обеспечены. Хорошо, что последнего не нужно заставлять выступать, он сам меня приглашает на пятиминутный ликбез по почтовым ящикам завтра. Тут и там кто-то роняет на пол бутылки пива Ruprecht, замечательного тем, что на этикетке у него карта и правильная атрибуция в углу.&lt;/p&gt;
  &lt;p&gt;Вечером нашли в старом городе бар-ресторан, оформленный с отсылками к популярным фильмам. Заказал коктейль «Холодное сердце» и пиццу в честь какого-то актёра. Пицца оказалась острой и коктейль тоже оказался острым. Ну что за дела. Рядом Доротея страдала, что её коктейль оказался сладким. Здесь я обычно вспоминаю, о чём мы говорили, но после доклада я был выжат и медленно молча приходил в себя. Даже в плед завернулся, потому что энергии не хватало на самообогрев. Осмеры были в восторге: русскому &lt;i&gt;холодно!&lt;/i&gt;&lt;/p&gt;
  </description>
  <pubDate>Mon, 23 Sep 2019 21:34:11 +0400</pubDate>
  </item>

  <item>
  <title>Гейдельберг 2019: суббота</title>
  <guid isPermaLink="false">1500</guid>
  <link>http://shtosm.ru/all/geydelberg-2019-subbota/</link>
  <comments>http://shtosm.ru/all/geydelberg-2019-subbota/</comments>
  <description>&lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/big-hall-sotm19.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Пришёл на автобусную остановку за полчаса до первого доклада. На табло: подожди трамвая 23 минуты, лол. Вместе с другими осмерами запрыгнул в автобус примерно того же направления: на полпути он сворачивает в сторону. Как только он свернул, все осмеры вышли и пошли вдоль трамвайных рельсов. Но я же изучил карту: со следующей остановки можно будет срезать угол и прийти быстрее всех этих. Учитесь картам, салаги! За окном пролетает следующая остановка и я понимаю, что впервые за три дня нужно было нажать кнопку «стоп». Пришёл я, конечно, быстрее других, но к зажигательному выступлению Грегори опоздал.&lt;/p&gt;
  &lt;p&gt;Открывающие доклады были захватывающими для новичков, но я успел половину &lt;a href="https://twitter.com/digitalmobilBW/status/1174222885969387520"&gt;посмотреть в интернете&lt;/a&gt;, другую — услышать на FOSDEM. В этот раз я предлагал себя в качестве открывающего спикера, но организаторы предпочли людей на заметных постах. Директорка Software Freedom Conservancy — это, конечно, круто, но сама система, когда конференция приглашает известных людей открывать не из-за того, что те имеют сказать, а просто потому что это как-то зависит от статуса, — странная. Вон, рижская ГИС-конференция на март пригласила Стива Коста, зачем-то. Хотя есть сотни людей, которые могут вдохновить слушателей более актуальными речами.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/human-editors.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Фредерик и Томас начали в западном зале развлекательную сессию на полтора часа про основы OpenStreetMap. Наряжая авторов редакторов в ленты с названиями, опытных жонглёров данными в лабораторные халаты и используя ящик прочего реквизита, они доходчиво описали всё, что нужно знать новичку. Ну, наверное: &lt;a href="https://media.ccc.de/c/sotm2019"&gt;записи опубликованы&lt;/a&gt;, шоу должно быть блистательным. Я появился буквально на пару минут: когда люди-редакторы закончили отвечать на вопросы, я нацепил табличку Level0 и заставил их сделать селфи. Потому что это же самый молодой и актуальный редактор среди них!&lt;/p&gt;
  &lt;p&gt;В докладе про развитие юго-восточной Азии через OSM Джинал и Эдгар из Grab (местного такси-сервиса) коротко пересказали, как они работают с сообществом картографов, и едва уложились в двадцать минут. Картина на слайдах и видео разворачивается тоталитарненькая: тысячи, десятки, сотни тысяч людей в футболках и жилетках Grab вовсю любят OpenStreetMap, рисуют карту, собираются в комнатах и на стадионах. Как будто одна компания захватила всю инициативу OpenStreetMap в регионе. Их доброе намерение понятно: силами компании возможно донести OSM до каждого, и хочется воспользоваться этим на всю катушку. Но где местное сообщество, не аффилированное с Grab? Что произойдёт, когда у них возникнет спор? Как долго Grab сможет не доставать аргумент, что они — крупнейший пользователь OpenStreetMap в регионе?&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/prikly-thing.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Короткие доклады, как всегда, странные, местами непонятные из-за акцента, но слишком короткие, чтобы успеть надоесть:&lt;/p&gt;
  &lt;ul&gt;
  &lt;li&gt;Бангладеш в местном языке — когда у жителей города и соседнего района разные языки, диванные картографы подписывают всё неправильно, и местным на карте ничего не понятно. Приходится отдельно заводить проекты по переводу карты.&lt;/li&gt;
  &lt;li&gt;&lt;a href="https://solidary.city/"&gt;SolidaryCityMap&lt;/a&gt; — Просто описание карты бесплатных вещей. Туда импортируют полмиллиона точек из нескольких сервисов — про фрукты, душ, одежду и т. п. Все данные под CC0/PDDL и есть выгрузки. Заметил на сайте, как ужасна прокрутка дробного масштаба в OpenLayers.&lt;/li&gt;
  &lt;li&gt;OpenLitterMap: давайте идти по улице и фотографировать разный мусор! А потом сравним страны по количеству мусора. Уникальный проект: прошёл, за тобой проехала мусорка, и вся карта неактуальна. Жалобы в управу, видимо, не работают, остаётся публиковать данные под открытой лицензией. Вечером устроили картопати вокруг университета, пофоткали бычки.&lt;/li&gt;
  &lt;li&gt;Джанет снова рассказала про женское обрезание и как карты спасают девочек. Я снова подумал (и обсудил с ней), что карты никого не спасают, они тихо лежат в шкафу или на экране компьютера. Спасают люди. То есть, нужно общаться, учить девочек пошаговому реагированию, когда их пытаются забрать, готовить маршруты с местными властями и домами-укрытиями, искать активистов, контролировать всех. А карта — никогда не самоцель.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/group-reversed-sotm19.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;ul&gt;
  &lt;li&gt;«Is OSM up to date?» — задаёт риторический вопрос слайд французов. Представляю себе сайт с большим «НЕТ» и больше ничем. Марко и Франческо &lt;a href="https://wiki.openstreetmap.org/wiki/Is_OSM_up-to-date"&gt;сделали карту&lt;/a&gt;, на которой все объекты раскрашены по дате. Полагаю, они скачиваются по overpass, и это всё жутко медленно. Есть раскрашивание по последней правке, версиям, количеству картографов и по частоте обновления. Находишь красные линии и идёшь обновлять.&lt;/li&gt;
  &lt;li&gt;Крис рассказал про редактор Deriviste, который когда-то написал Ричард. Там видишь фоточку из Mapillary и отмечаешь объекты на ней, типа фонарных столбов, прямо в OpenStreetMap. Основные патчи в код пришли от Криса и Рори. Времени доклада хватило только на то, чтобы показать основные кнопки, но всё равно красиво. В будущем Крис предполагает поступательное развитие: поддержку oauth, добавление заметок, рисование линий, использование классифицированных Mapillary снимков.&lt;/li&gt;
  &lt;li&gt;Минг быстро и чётко описал, как американцы двенадцать лет чинили TIGER и импортировали всё остальное.&lt;/li&gt;
  &lt;li&gt;Анди и Арнали напомнили, что во многих сообщества перспектива женщин отличается от мужской, и важно привлекать их к рисованию карт. А не как обычно.&lt;/li&gt;
  &lt;li&gt;Я быстро пересказал свой &lt;a href="https://www.youtube.com/watch?v=KwUZkUkpITk"&gt;доклад с byGIS&lt;/a&gt; про обратный геокодер в Juno.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;p&gt;Ещё я очень хотел узнать что-то новое из презентации &lt;a href="https://wiki.openstreetmap.org/wiki/OsmInEdit"&gt;OsmInEdit&lt;/a&gt;, нового веб-редактора от автора OpenLevelUp и форка iD для редактирования поэтажных планов. Редактор опубликовали только два дня назад, и я не вынес ни единой мысли из слайдов. Какая-то ретроспектива схемы Simple Indoor Tagging, упоминание французской компании SNCF, которая оплатила поэтажные планы ж/д станций в OpenStreetMap, скриншоты с обычными переключателями этажей и линиями, которые фиг распутаешь. Решил, что вместо попыток понять акцент Адриена лучше как-нибудь потыкаю сам редактор.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/hebelhalle-yard.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Удивительно, как все здания в современной части Гейдельберга (а то и всей Германии) выглядят одинаково. Серые коробки с серыми ставнями, будь это библиотека или шоурум Mercedes. Помещение для пышных церемоний снаружи не отличишь от товарного склада. Неудивительно, что Simple 3D придумали в Германии: эта схема описывает почти все местные здания, даже теги для цвета не нужны. Так мы и прибыли на вечеринку в HebelHalle: отличили его только по обилию людей в синих футболках. И ещё полчаса были уверены, что вся площадка состоит из двора со столиками и узкой барной стойки у туалета — пока из ниоткуда не начали появляться осмеры с тарелками еды.&lt;/p&gt;
  &lt;p&gt;В банкетном зале вдоль стен были разложены старые печатные карты немецких районов, а на одной стене наклеены пронумерованные распечатки дорог из разных столиц. Угадать, где какая, почти невозможно — узнал только Москву. Отличная идея, но хотелось бы какого-то продолжения: викторины или просто ключа. Иан Вагнер подметил признаки, по которым можно отличить американские города от южноамериканских и европейских: слишком много магистралей. Я всю дорогу был уверен, что Иан работает в Северной Корее и обсуждал её, но оказалось, что нет, он в Южной. Парень из Северной не смог приехать.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/hebelh-maps.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Премия OpenStreetMap оказалась главным и самым громким событием этого вечера. С помощью Труди из Замбии мы нашли людей, которые будут вручать награды, подготовили стопку из книг, дипломов и конвертов и загнали всех в церемониальный зал. Немного нервно за минуту до выступления пытаться придумать первые слова. Но включается центральный прожектор, ты встаёшь в круг света перед тремя сотнями зрителей, держишь нужные паузы, и внезапно слова складываются, шутки появляются из ниоткуда, аудитория смеётся и аплодирует, и уже зовёшь Саджада для объявления премии за вклад в инфраструктуру. Поздравляю Виктора Щерба с &lt;a href="http://awards.osmz.ru/"&gt;победой&lt;/a&gt; в номинации имени Ульфа Мёллера, а также Виктора, Алмаза и literan с номинациями.&lt;/p&gt;
  &lt;p&gt;Название Social Event подразумевает, что главная цель — социализация. Познакомиться и поговорить с максимальным количеством человек. Странным образом, тут не страшно подходить к новым людям и вливаться в чужие беседы. Это в русскоязычных конференциях все деловые и неприступные, а тут улыбнутся, «hi man» и продолжаем. Пять минут — и ты уже знаешь, кто есть кто, сам тоже уже рассказал всякое.&lt;/p&gt;
  &lt;p&gt;Увидел, наконец, Энди Аллана вживую. Тот сейчас работает над обвязкой для внедрения &lt;a href="https://github.com/openstreetmap/openstreetmap-website/pull/2353"&gt;нового API&lt;/a&gt; и опрашивает разработчиков всех редакторов, что тех волнует. Рассел Дефнер из HOT с помощью технологий фейсбука и десятков добровольцев почти закончил рисовать дорожную сеть Индонезии. Типа, в официальных картах четыре тысячи километров, а в OSM — миллион. Одна нейросеточка заменяет тысячу новичков в Missing Maps. Так, глядишь, и всех остальных заменит. Предложил ему переключиться на Россию. Фредерик же поделился секретом обёртки книг для премии: она сделана из тайлов OpenWhateverMap без трёх тёмных слоёв. Ни один узор не повторяется: он специально запустил скрипт семь раз. Обожаю, когда люди заморачиваются дополнительно ради каких-то мелочей, которые никто не заметит: очень по-японски, всё равно почувствуешь, пусть и не поймёшь.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/hebelhalle-guitarist.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  </description>
  <pubDate>Mon, 23 Sep 2019 02:43:25 +0400</pubDate>
  </item>

  <item>
  <title>Гейдельберг 2019: пятница</title>
  <guid isPermaLink="false">1499</guid>
  <link>http://shtosm.ru/all/geydelberg-2019-pyatnica/</link>
  <comments>http://shtosm.ru/all/geydelberg-2019-pyatnica/</comments>
  <description>&lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/campus-heigit.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Германия очень контрастирует со Штатами. Там ты, даже не будучи местным, получаешь заряд свободы и активности: «мы рады тебе и твоей работе, всё вокруг открыто, делай что-нибудь!». Здесь всё квадратно и практично. Библиотека неотличима от парковки, парковка неотличима от музея. Иди по линиям, соблюдай расписание, приготовь документы. Уменьшай свой экологический след, посторонись, не мешай серьёзным людям. «Мы заняты, но ты тоже давай, приноси пользу, будь эффективным». Я люблю порядок, но Германия — на каком-то другом уровне, тут так любят порядок, что стремятся выпрямить всё вокруг, блистательно проваливаются в этом, но делают вид, что всё в порядке и это только ты виноват. В отличие от Миннесоты, я бы не отдал детей в местный университет. Он зелёный и унылый. Пусть тут и каждый третий студент говорит на русском.&lt;/p&gt;
  &lt;p&gt;На всём HOT Summit около сотни участников — но они настолько разные, что кажется, будто полмира съехалось поделиться радостью от пользы карт. За практическими примерами теряется сам OpenStreetMap, который на каждом шагу сопротивляется извлечению пользы. Примерно об этом говорил Микел в своём докладе. Мол, правкам гуманитарной команды не все рады, и в целом как-то нужно регистрировать проблемы потребителей, картографов и организации. Объясняет, чем занимается Chief Data Officer и как проекты проходят валидацию в команде. Рассказ, впрочем, быстро сдувается в единственную мысль: давайте подумаем про сообщество.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/horsaal-wall.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Кажется, что команда невелика, но числа от Ральфа показывают другую картину. В слайдах про валидацию задач в Tasking Manager он огорошил всех сотнями тысяч новичков, многие из которых кликают один квадратик в менеджере и уходят. А за ними нужно всё подчищать. Для этого нужны люди, валидирующие все эти квадратики, и как ни старается команда, более двух третей проекта отвалидировать не удаётся. Нужно больше опытных картографов. Если вспомнить, что это HOT, планка опытности довольно низка: достаточно понимать, чем дорога отличается от реки на снимке и в OSM.&lt;/p&gt;
  &lt;p&gt;Роб Бейкер рассказал про этичные данные. Всё как обычно: за «зачем» те, кто публикует и работает с данными, теряют более важный вопрос «кому». Смотрите, мы опубликовали сто миллионов точек. А анонимизировать вы не забыли, случайно? Публикация данных очень сложна с точки зрения настоящей анонимизации, потому что часто не проблема сопоставить несколько наборов и определить, где конкретный человек находился или что делал. Активисты в Нью-Йорке великолепно обработали «чистые» данные от телефонной компании, построив маршрут мэра и его охраны за один из дней. То же с использованием данных: за примерами даже далеко ходить не надо, у нас многие технари любят классифицировать районы города или людей по очевидным признакам.&lt;/p&gt;
  &lt;p&gt;Единственное, о чём Роб не говорил, — это OpenStreetMap. С этикой у нас швах по всем фронтам. Какая анонимизация, когда люди ставят точки на карту с названиями «мой дом», а дачи картографов — самые детально прорисованные в дачных посёлках? Вся гуманитарная команда картирует ради настоящего и пары сотен волонтёров, вообще не считаясь с будущим картографии. Если не вспоминать местные инициативы в паре стран, типа Индонезии и Кении. Отсутствие правил в тегировании и выборе объектов для отрисовки порождает религиозные споры. Отсутствие документации и злые местные картографы — постоянную фрустрацию у новичков. Каждый в OpenStreetMap, независимо от опыта, постоянно унижен и дезориентирован. Давайте поговорим об этике.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/sand-mountain-heid.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Генри из Уганды и Алексей с Софией из Украины используют Mapillary для одного и того же: они снимают все улицы и дороги своего города и затем используют распознавание нейросеточками и маркетплейс Mapillary для поиска на этих сотнях тысяч картинок мусорных контейнеров и несанкционированных свалок. Удивительно, как кажущаяся нишевым развлечением технология преображается от масштаба. Когда на нашей карте была пара дорог, ею интересовались только энтузиасты. Когда мы нарисовали все дороги — внезапно она понадобилась всем. То же и с Mapillary: пока что это просто хранилище фоточек, но с ростом сообщества, объёма данных и качества алгоритмов распознавания оно принесёт какую-нибудь неожиданную пользу каждому.&lt;/p&gt;
  &lt;p&gt;Mhairi, Garauv и Emmanuel из проекта METEOR классифицируют структуру и прочие атрибуты зданий, чтобы в случае землетрясения, наводнения или годзиллы можно было быстро оценить, куда в первую очередь высылать пожарных и куда отвозить раненых. Интересный рассказ с фоточками отогнал сонливость: ребята в Дар эс Салааме и Катманду обошли по две с половиной тысячи зданий и для каждого указали около &lt;a href="https://wiki.openstreetmap.org/wiki/GED4ALL"&gt;пятнадцати атрибутов&lt;/a&gt;. Местами выглядит подозрительно, потому что вместимость, возраст, или количество подземных этажей не всегда просто определить. Поэтому одинаковым числам для каждого тега напротив результатов в Танзании поверить сложно.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/the-bubble.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Все с разными акцентами: сложно переводить на лету, приходится распознавать, думать, подстраиваться. Мне несложно понимать, о чём говорят по радио или поют в англоязычных песнях, но тут у всех такие дикие акценты, что приходится часто переспрашивать, и это фрустрирует и меня, и собеседника. Английский язык универсален, но он всё равно явный барьер для участия в подобных по-настоящему международных конференциях. Мы же смотрим сериалы с субтитрами — пора применить технологию для конференций. Американцы успешно делают субтитры в прямом эфире, автоматические переводчики пробивают потолок (см. &lt;a href="https://www.deepl.com/translator"&gt;deepl&lt;/a&gt;). Давайте добавим технологий в зал презентаций. Пусть люди говорят как привыкли. Странно HOT Summit ограничивать английским языком.&lt;/p&gt;
  &lt;p&gt;HOT Summit — странный зверь, как и сама гуманитарная команда. Часто сидел и не понимал, о чём говорят люди, зачем это нужно и почему никто не думает о карте, об OpenStreetMap. Как поделился Фредерик, он рад, что приехал поздно, потому что иначе смотрел бы доклады и не мог удержаться от язвительных комментариев. Мне было нормально: я понимаю, что карта у каждого своя. Но было ощущение, что я совсем не на картографической конференции, а, скорее, на встрече волонтёров. Которые любят совместную работу, помогают спасать людей и делают много полезных вещей. Но интересы которых совсем не на карте, и потому не особо пересекаются с моими.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/nekkar-dam.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  </description>
  <pubDate>Sat, 21 Sep 2019 12:34:52 +0400</pubDate>
  </item>

  <item>
  <title>Гейдельберг 2019: четверг</title>
  <guid isPermaLink="false">1498</guid>
  <link>http://shtosm.ru/all/geydelberg-2019-chetverg/</link>
  <comments>http://shtosm.ru/all/geydelberg-2019-chetverg/</comments>
  <description>&lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/heid-market.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Как максимально непродуктивно провести день в новом городе? Пойти смотреть достопримечательности. От их созерцания получаешь секунду удовольствия, шаблонную фоточку, коих тысячи в интернете, стёртую обувь и потерянные полчаса. И, если удаётся приглушить внутренний голос, чувство удовлетворённости выполненной программой. Так что я приглушил, захватил флэшку с дипломами номинантам Премии OSM и поехал в центр. Пройдёмся по списку достопримечательностей.&lt;/p&gt;
  &lt;ul&gt;
  &lt;li&gt;Общественный транспорт. Его много, ходит как часы, насколько это возможно в Германии. Ну, то есть, от вокзала в центр я ехал на трамвае номер «сорри, ауфидерзейн», или как-то так. Организаторы обеих конференций сотворили для участников билеты на весь общественный транспорт на пять дней в сумме, поэтому ездил я всюду и помногу. Ни одного контролёра не встретил, поэтому не знаю, правильные ли это билеты.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;ul&gt;
  &lt;li&gt;Центр. Встречает традиционным ТЦ «Галерея». Во всех городах всех стран есть такой, ровно в центре. От него идёт пешеходная улица с магазинами одежды и плюшек. Всё как обычно. Книжные — только на немецком, одно расстройство.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/heidelberg-central-church.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;ul&gt;
  &lt;li&gt;Собор Петра и Церковь Святого Духа. Ну соборы. Сфоткал и пошёл. Внутри церкви был органный концерт, но с выступлениями в белорусской филармонии не сравнится.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;ul&gt;
  &lt;li&gt;Старый мост. Это мост.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;ul&gt;
  &lt;li&gt;Тропа философов. Перейдя через мост, зашёл в закуток между зданиями и начал подниматься по лестнице. Поднимаюсь. Слева стена и справа стена. Поднимаюсь. Становится жарковато. Я вспоминаю Айзувакамацу, где я тоже поднимался в гору в ужасную жару и весь промок. Туристический сайт пишет, что на тропе, к которой я наконец выхожу, климат более южный, поэтому из флоры здесь пальмы и лимоны, а из фауны — ящерицы и змеи. Климат и высота располагают к тому, чтобы посидеть пару часов, подумать, никогда больше никуда не ходить. Вид с тропы отличный, нельзя не сфотографировать старый мост и замок на том берегу.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/heid-old-bridge.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;ul&gt;
  &lt;li&gt;Водопад. С южной стороны реки Неккар две горы: логично, что вода с них стекает в долину и затем сливается в сторону города. Этот ручей под городом идёт в трубе, но на окраинах течёт по поверхности, с водопадом совсем недалеко от Собора Петра. Единственный и главный водопад Гейдельберга, о котором не рассказывает ни один путеводитель, отмечен на OpenStreetMap. Посмотрев, что времени остаётся либо на фуникулёр, либо на водопад, выбрал второй и пошёл искать. Водопад действительно высокий, метров двадцать, очень красиво оформленный. Одна беда: воды не дали. Сезон сухой, камни едва влажные, и жалкому ручейку не хватает сил даже дотечь до решётки водостока.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;ul&gt;
  &lt;li&gt;Гора Кёнигштуль. От водопада оказалось недалеко до фуникулёра, но по пути мне повезло: поймал тридцатый автобус, который едет туда же. Ну, почти: вместо горы я оказался в обсерватории. Выглядит космически. Но до темноты полчаса, поэтому так и не зашёл, не посмотрел на фотографии космоса и модели аппаратов. Вместо этого пошёл по maps.me искать верхушку горы. Она оказалась на закрытой территории близ ретрансляционной вышки. Сплошные разочарования с этими горами. Нормальная обзорная площадка нашлась рядом со станцией фуникулёра, но вечернее солнце не дало сделать приличную фотографию. Из плюсов — ларёк с мороженым. Главный горный план я, всё-таки, выполнил: посмотрел на город с двух высоких точек. Нет, чтобы купить квадрокоптер и не мучаться.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/mountain-railway.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;ul&gt;
  &lt;li&gt;Еда. То ли мне не везёт, то ли вся еда в Гейдельберге отвратительна. Завтрак на вокзале — либо жуёшь хлеб, либо хрустишь листьями. На обеде я внезапно осознал, что запиваю очень средние спагетти невкусным пивом, и за всё это я отдал семнадцать евро. Вечером не выдержал, купил в магазине бананов и нормальной выпечки, потому что невозможно же.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;ul&gt;
  &lt;li&gt;Канцелярский магазин Кноблаух Шрайбварен. Найти хорошую цветную бумагу для дипломов в Гейдельберге непросто. Во всех магазинах одни и те же оттенки от одного и того же производителя. Прямо СовБумПромСнаб. Зелёный слишком зелёный, жёлтый недостаточно жёлтый. Потребовалось нагуглить «лучший канцелярский в Гейдельберге», чтобы найти второй набор цветов. Дальше проще: в городе со старейшим университетом в Германии распечатать странички А4 предсказуемо можно на каждом углу.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;ul&gt;
  &lt;li&gt;Русская речь. Тоже можно услышать на каждом углу. Иногда кажется, что даже чаще, чем немецкую. Студенты в автобусе обсуждают защиту в университете. Девушка за завтраком рассказывает приехавшей из России подруге, как что устроено. Мужик на улице по телефону руководит доставкой. Даже сейчас, за соседним столиком на HOT Summit, русскоязычные ребята готовятся к докладу про Mapillary в Украине.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;ul&gt;
  &lt;li&gt;Замок. Нафиг эти руины.&lt;/li&gt;
  &lt;/ul&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/heid-castle.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Почувствовав недостаток социализации, зашёл на вечеринку HOT на окраине города. Входишь в старинный дом, а там сотня человек перекрикивают друг друга, половина знакомых, со всеми нужно поздороваться, обсудить перелёт и «о, мы ж две недели назад в Штатах виделись». Встретил Сатоши из Японии, с которым виделся в Айзу два года назад. Мэгги, Райана, Дмитрия и Дженнингса из Штатов. Дорофея, конечно же. Кучу девушек из местного университета, организовавших вот это всё. О чём говорили? О погоде и о времени, конечно же: обстановка не располагает ни к чему более серьёзному.&lt;/p&gt;
  &lt;p&gt;В попытке скрыться от постоянного шума спустился в подвал, где внезапно организовали картособытие Missing Maps. Девушка полчаса рассказывала, как важны карты для спасения людей, показывала Tasking Manager и объясняла, как выбрать проект и что они все значат. Мероприятие казалось немного странным: любой из слушателей наверняка мог бы выдать такую же речь. Но иногда комфортно просто посидеть и послушать знакомые слова. «А теперь давайте порисуем карту» — и два человека, принёсших ноутбуки, открывают их.&lt;/p&gt;
  &lt;p&gt;На обратном пути догнал парней из Mapillary под предводительством Криса. По пути подумали, почему даже на лучших конференциях детей приходится занимать чем-то посторонним в детских комнатах. На мероприятиях Esri иногда устраивают семейный день, когда конференция работает в режиме выставки и дети могут поиграть промо-бумажками. А как сделать такую конференцию, чтобы детям было интересно, но в то же время, взрослым было познавательно? Непонятно, но первым шагом должно быть уйти от схемы «все сидят полчаса и смотрят на одного человека».&lt;/p&gt;
  &lt;p&gt;Завтра пойду весь день сидеть и смотреть на одних людей.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/heid-observatory.jpg" width="1400" height="700" alt="" /&gt;
  &lt;/div&gt;
  </description>
  <pubDate>Fri, 20 Sep 2019 13:58:35 +0400</pubDate>
  </item>

  <item>
  <title>Гейдельберг 2019: среда</title>
  <guid isPermaLink="false">1497</guid>
  <link>http://shtosm.ru/all/heidelberg-2019-sreda/</link>
  <comments>http://shtosm.ru/all/heidelberg-2019-sreda/</comments>
  <description>&lt;p&gt;Как в прошлый раз, в самолёте набросал заметку на абстрактную тему, связанную с OpenStreetMap. Обнаружил, что в ней пересказал половину своего воскресного доклада. Так что подожду публиковать, чтобы пропустить окно WeeklyOSM и удивить слушателей логичными, но совершенно неочевидными инсайтами. Типа, что тип area для API 0.7 — это интересное мысленное упражнение, но дурацкая затея.&lt;/p&gt;
  &lt;p&gt;Дорога к немцам — верх предсказуемости: билеты от начала до конца получены и оплачены, на всех проставлено время, выходишь на полчаса раньше обычного и в срок приходишь в терминал. Самолёт «Люфтганзы» задерживают на полчаса. Ничего, с Дорофеем весело проводить время. На паспортном контроле во Франкфурте каждому чуть ли не сумки потрошат, теряем целый час. Ничего, зато накликал полтысячи картинок в &lt;a href="https://www.mapillary.com/app/marketplace"&gt;тренере Mapillary&lt;/a&gt;.&lt;/p&gt;
  &lt;p&gt;Приходим на вокзал аэропорта, я краем глаза замечаю, что наш поезд в Гейдельберг задерживается на десять минут. Ну мелочи же. Через пять минут подходит состав, мы залезаем и находим свободное место. Я запускаю maps.me, посмотреть на синюю стрелочку на карте Германии. И удивляюсь, как далеко она от наших предполагаемых координат. Смотрю на табло в вагоне.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/ne-tuda-seli.jpg" width="1400" height="500" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Поднимаю взгляд на Дорофея и вижу за его спиной контролёра в тамбуре. Немедленно вспоминаю истории про страшные штрафы, встаём с мест и идём в тамбур, чтобы перехватить инициативу. И чудом проходим мимо контролёра в дверях. Через пять минут выходим на окраине Лимбурга. Поля с одной стороны, два бизнес-центра и шоссе с другой. Тишина, только иногда скоростные поезда проносятся на 250 км/ч по рельсам на бетонной подложке. Что делать — непонятно.&lt;/p&gt;
  &lt;p&gt;Для сравнения — это как если бы вы прилетели в Шереметьево, сели на поезд в сторону Тулы, а оказались в Клину. Первым делом, нужно вернуться в предсказуемое состояние: в аэропорт. Следующий поезд через двадцать минут — а нет, это же Deutsche Bahn, он опаздывает ещё на полчаса. Тогда быстрее на скоростном, который придёт через час двадцать. Есть время пообедать. Напротив KFC у трассы, который мы выбрали, стоит Макдональдс — он крытым переходом соединён с фитнес-клубом. Очень эффективно. Но логично ли?&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/net-fuel-station.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  &lt;p&gt;Полтора часа и шестьдесят евро спустя, мы снова в аэропорту, ждём поезда на Гейдельберг. Я постоянно палю табло. Наш поезд через десять минут на Basel SBB через Мангейм. Через две минуты. И тут его строчка ИСЧЕЗАЕТ. Ничего никуда не приходит. Да ёлки-палки, DB не устаёт удивлять. Следующий поезд через пятнадцать минут на Штуттгарт, тоже с остановкой в Мангейме, нам подходит. Очень внимательно слежу за экраном, когда приходит состав — сравниваем номер с тем, что написан в расписании и на табло. Ночь, улица, море велосипедов — мы, наконец, в Гейдельберге. Пока я наслаждаюсь жареной уткой и комплиментарным бокалом пива, Дорофей обнаруживает в своём номере &lt;a href="https://www.facebook.com/darafei/posts/10217879504686735"&gt;спящих людей&lt;/a&gt;.&lt;/p&gt;
  &lt;p&gt;Сегодня первый день HOT Summit. Я регистрировался только на второй, а сегодня буду готовить Премию OSM и гулять по городу. Поэтому на канале &lt;a href="https://t.me/foss4g_ru"&gt;@foss4g_ru&lt;/a&gt; вещание целиком в руках Дорофея. Там неожиданно интересно, подпишитесь.&lt;/p&gt;
  &lt;div class="e2-text-picture"&gt;
  &lt;img src="http://shtosm.ru/pictures/post-office-trash.jpg" width="1400" height="600" alt="" /&gt;
  &lt;/div&gt;
  </description>
  <pubDate>Thu, 19 Sep 2019 14:40:05 +0400</pubDate>
  </item>


  </channel>
  </rss>


key: steve_singer
encoding: utf8
format: rss 2.0
title: Steve Singer
url: https://scanningpages.wordpress.com/category/openstreetmap/
feed_url: https://scanningpages.wordpress.com/category/openstreetmap/feed/
summary: Steve Singer's blog on databases, GIS and other software like things
generator: http://wordpress.com/
updated: 2013-06-13 03:22:16
items_last_updated: 2013-06-13 03:19:05
fetched: 2019-10-26 23:00:23.545561
http_etag: W/"66f7d8786cd97409aee59ac76000c0a9"
http_last_modified: Sat, 26 Oct 2019 23:00:23 GMT
http_server: nginx
md5: a25d55daa19540b68e545cb1f1a26596
created_at: 2019-06-09 18:11:33.495402
updated_at: 2019-10-26 23:00:23.546787
body:
  <?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
  	xmlns:content="http://purl.org/rss/1.0/modules/content/"
  	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
  	xmlns:dc="http://purl.org/dc/elements/1.1/"
  	xmlns:atom="http://www.w3.org/2005/Atom"
  	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
  	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
  	xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
  	>

  <channel>
  	<title>openstreetmap &#8211; Scanning Pages</title>
  	<atom:link href="https://scanningpages.wordpress.com/category/openstreetmap/feed/" rel="self" type="application/rss+xml" />
  	<link>https://scanningpages.wordpress.com</link>
  	<description>Steve Singer&#039;s blog on databases, GIS and other software like things</description>
  	<lastBuildDate>Thu, 13 Jun 2013 03:22:16 +0000</lastBuildDate>
  	<language>en</language>
  	<sy:updatePeriod>
  	hourly	</sy:updatePeriod>
  	<sy:updateFrequency>
  	1	</sy:updateFrequency>
  	<generator>http://wordpress.com/</generator>
  <cloud domain='scanningpages.wordpress.com' port='80' path='/?rsscloud=notify' registerProcedure='' protocol='http-post' />
  <image>
  		<url>https://s0.wp.com/i/buttonw-com.png</url>
  		<title>openstreetmap &#8211; Scanning Pages</title>
  		<link>https://scanningpages.wordpress.com</link>
  	</image>
  	<atom:link rel="search" type="application/opensearchdescription+xml" href="https://scanningpages.wordpress.com/osd.xml" title="Scanning Pages" />
  	<atom:link rel='hub' href='https://scanningpages.wordpress.com/?pushpress=hub'/>
  	<item>
  		<title>State Of The Map US 2013</title>
  		<link>https://scanningpages.wordpress.com/2013/06/13/state-of-the-map-us-2013/</link>
  				<comments>https://scanningpages.wordpress.com/2013/06/13/state-of-the-map-us-2013/#respond</comments>
  				<pubDate>Thu, 13 Jun 2013 03:19:05 +0000</pubDate>
  		<dc:creator><![CDATA[scanningpages]]></dc:creator>
  				<category><![CDATA[openstreetmap]]></category>
  		<category><![CDATA[community]]></category>
  		<category><![CDATA[maps]]></category>
  		<category><![CDATA[open source]]></category>
  		<category><![CDATA[opendata]]></category>
  		<category><![CDATA[osm]]></category>
  		<category><![CDATA[SFO]]></category>
  		<category><![CDATA[sotm]]></category>
  		<category><![CDATA[sotmus]]></category>

  		<guid isPermaLink="false">http://scanningpages.wordpress.com/?p=622</guid>
  				<description><![CDATA[I am sitting on a plane flying eastbound at 37,000 feet over middle of America. A few hours ago we left San Francisco the host city for State Of The Map US 2013. State of the Map US is the American regional conference for OpenStreetMap participants and enthusiasts. A common theme for the weekends events [&#8230;]]]></description>
  								<content:encoded><![CDATA[<div data-shortcode="caption" id="attachment_628" style="width: 310px" class="wp-caption aligncenter"><a href="https://scanningpages.files.wordpress.com/2013/06/code_sprint1.jpg"><img aria-describedby="caption-attachment-628" data-attachment-id="628" data-permalink="https://scanningpages.wordpress.com/2013/06/13/state-of-the-map-us-2013/code_sprint/" data-orig-file="https://scanningpages.files.wordpress.com/2013/06/code_sprint1.jpg" data-orig-size="1500,1004" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;9&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;NIKON D3000&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;1370886418&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;18&quot;,&quot;iso&quot;:&quot;400&quot;,&quot;shutter_speed&quot;:&quot;0.05&quot;,&quot;title&quot;:&quot;&quot;}" data-image-title="code_sprint" data-image-description="&lt;p&gt;Developers working hard at the SOTMUS 2013 code sprint&lt;/p&gt;
  " data-medium-file="https://scanningpages.files.wordpress.com/2013/06/code_sprint1.jpg?w=300" data-large-file="https://scanningpages.files.wordpress.com/2013/06/code_sprint1.jpg?w=614" src="https://scanningpages.files.wordpress.com/2013/06/code_sprint1.jpg?w=300&#038;h=200" alt="Code Sprint Photo" width="300" height="200" class="size-medium wp-image-628" srcset="https://scanningpages.files.wordpress.com/2013/06/code_sprint1.jpg?w=300&amp;h=200 300w, https://scanningpages.files.wordpress.com/2013/06/code_sprint1.jpg?w=598&amp;h=400 598w, https://scanningpages.files.wordpress.com/2013/06/code_sprint1.jpg?w=150&amp;h=100 150w" sizes="(max-width: 300px) 100vw, 300px" /></a><p id="caption-attachment-628" class="wp-caption-text">OSM hard at work</p></div>
  <p>I am sitting on a plane flying eastbound at 37,000 feet over middle of America.  A few hours ago we left San Francisco the host city for <a href="http://www.stateofthemap.us" target="_blank">State Of The Map US 2013</a>.  State of the Map US is the American regional conference for OpenStreetMap participants and enthusiasts.  A common theme for the weekends events was building community.</p>
  <p><span id="more-622"></span></p>
  <p>I go to open-source conferences so that I can spend time informally talking with other members of my open source community.  In San Francisco this week I was able to meet OpenStreetMap celebrities such as Richard Fairhurst (author of Potlach) and Frederick Ramm (former JOSM maintainer, Geofabrick founder and OSMF board member).  I also spent time informally chatting with dozens of people who attended the conference to learn more about OpenStreetMap.   Many people I spoke with told me they were interested in OpenStreetMap and were looking into using the data but had only done a small amount of mapping.</p>
  <p>Conferences typically have three sets of activities the most popular of these are the sessions.  My favorite sessions of this conference was the session Ryan Closner gave on on using Voxel.js to render OpenStreetMap data and a set of sessions on who is doing mapping.  Voxel.js is a open-source clone of Minecraft written in Javascript.  The presenter has built software that allows your character to &#8216;walk through&#8217; a world consisting of real streets and building locations through OpenStreetMap data.  My other favorite session was the track on community analysis.  Richard Fairhurst, Alan McConchie, and Martjin van Exel presented a set of very related talks on who is doing the mapping. I hope to use some of their techniques and code to perform an analysis of activities in the Canadian community.</p>
  <p>The second conference track is the hall track where everyone talks informally between sessions in the hall or conference while waiting for a session to start.  I find that I get the most out of a conference by making a point of saying hi to the person sitting next to me while waiting for a session to start.  I might ask them where they are from, why they are at the conference and what types of things they have mapped.  I learned that there are many people new to the OpenStreetMap community.  I have been adding to data to OpenStreetMap for almost 5 years and have spoken on OpenStreetMap at at least 3 conferences and numerous mapping parties and hack days.  This was the first OpenStreetMap conference for many of the people I spoke with. Many of them are coming either from a traditional geography background or web background where they are starting to use OpenStreetMap for stuff.  As a community we,  need to keep these people engaged as regular casual mappers to make sure that they help us keep the map strong. I spoke with many people that we were able to convince too spend their weekend at a OpenStreetMap conference but we haven&#8217;t yet been able to convince them to spend their weekend adding to the map. How do we convince these people to occasionally take half a day and spend it on improving the map?</p>
  <p>We also need to make sure that they can easily use our data. We need to reach out and build bridges with communities that can use our map. A group of us met in a BOF to discuss improving the diversity of the people involved in OpenStreetMap. We agreed to create a <a href="http://lists.openstreetmap.org/listinfo/diversity-talk" target="_blank">diversity-talk</a> mailing list to discuss ways we can expand the OpenStreetMap community.   I hope to spend some time thinking of ways I can grow our community by making inroads into other established communities.</p>
  <p>The third conference track I attended was the party track.  I attended three parties in San Francisco. There was a party Saturday night at The Raven a popular bar on Mission in San Francisco.  On Sunday, Stamen design welcomed the OpenStreetMap world into their office for a party. The Stamen office is filled with cool maps. Monday night Mapbox also hosted a small party at a pub following the code-sprint.  I missed the Code for America party on Friday but I don&#8217;t doubt that was fun.  Eating and drinking with other mappers allows you to get to know them in a more relaxed atmosphere.  I find that it is much easier to ask someone for help or their honest opinion when you&#8217;ve spent time socializing with them.</p>
  <p>I want to thank OpenStreetMap.US and the sponsors for organizing this conference and all of the folks from Europe and Asia (I met people who flew in from Japan and New Zealand). </p>
  ]]></content:encoded>
  							<wfw:commentRss>https://scanningpages.wordpress.com/2013/06/13/state-of-the-map-us-2013/feed/</wfw:commentRss>
  		<slash:comments>0</slash:comments>
  						
  		<media:content url="https://1.gravatar.com/avatar/42e7367ea86bb9c5fc44b02c814d9e54?s=96&#38;d=identicon&#38;r=G" medium="image">
  			<media:title type="html">scanningpages</media:title>
  		</media:content>

  		<media:content url="https://scanningpages.files.wordpress.com/2013/06/code_sprint1.jpg?w=300" medium="image">
  			<media:title type="html">Code Sprint Photo</media:title>
  		</media:content>
  	</item>
  		<item>
  		<title>Maproulette Hack Weekend</title>
  		<link>https://scanningpages.wordpress.com/2013/03/13/maproulette-hack-weekend/</link>
  				<comments>https://scanningpages.wordpress.com/2013/03/13/maproulette-hack-weekend/#respond</comments>
  				<pubDate>Wed, 13 Mar 2013 02:11:29 +0000</pubDate>
  		<dc:creator><![CDATA[scanningpages]]></dc:creator>
  				<category><![CDATA[openstreetmap]]></category>
  		<category><![CDATA[community]]></category>
  		<category><![CDATA[hack weekend]]></category>
  		<category><![CDATA[open source]]></category>
  		<category><![CDATA[osm]]></category>
  		<category><![CDATA[ryerson]]></category>
  		<category><![CDATA[software]]></category>
  		<category><![CDATA[software design]]></category>
  		<category><![CDATA[technology]]></category>
  		<category><![CDATA[toronto]]></category>

  		<guid isPermaLink="false">http://scanningpages.wordpress.com/?p=589</guid>
  				<description><![CDATA[This weekend we had the second annual Toronto OpenStreetMap developer weekend. The nice folks at the Ryerson Department of Geography hosted us. My focus this weekend was to work the Serge and Martijn on maproulette Maproulette is software that presents an easy to do mapping task to users which they can complete and then mark [&#8230;]]]></description>
  								<content:encoded><![CDATA[<p>This weekend we had the second annual <a href="http://wiki.openstreetmap.org/wiki/Toronto_Hack_Weekend_March_2013">Toronto OpenStreetMap developer weekend</a>.   The nice folks at the <a href="http://www.ryerson.ca/geography">Ryerson Department of Geography</a> hosted us.  My focus this weekend was to work the <a href="http://wiki.openstreetmap.org/wiki/User:Emacsen">Serge</a> and <a href="http://oegeo.wordpress.com/martijnvanexel/">Martijn</a> on <a href="http://www.maproulette.org">maproulette</a></p>
  <p>Maproulette is software that presents an easy to do mapping task to users which they can complete and then mark the task as completed.  Examples of past maproulette mapping challenges include fixing connectivity errors or fixing objects touched by the license change.<br />
  <span id="more-589"></span><br />
  Serge and Martijn wanted to redesign parts of maproulette to make it easier develop more challenge modules.  I am going to discuss some of the design decisions we debated during the weekend.</p>
  <p><strong>Challenges</strong><br />
  A Challenge is a type of mapping activity that map roulette can assign to users.   Examples of challenges might be &#8216;Add bridges to roads with where we suspect a bridge exists&#8217;  or &#8216;Add the street address tags to restaurants that don&#8217;t have addresses&#8217;. </p>
  <p><strong>Tasks</strong><br />
  A task is a specific work item within a challenge.  In the &#8216;Add bridges to roads&#8217; challenge a task might be to add a bridge to a specific intersection (or determine that bridge should be there)</p>
  <p>Maproulette needs to provide a web based interface that displays challenge tasks to users(task assignments).  This requires us to somehow store information about the tasks that make up a challenge.  Challenge specific modules will compute tasks (ie find missing bridges).  Figuring out how the web interface and the programs that find challenge tasks should be connected was one of the questions we discussed this weekend.</p>
  <p><strong>Proposal 1: Discrete Modules</strong></p>
  <p>Each challenge has a challenge module that supports operations such as </p>
  <ul>
  <li> Give me some available tasks near this lat/lon</li>
  <li> Mark this task as assigned to a user </li>
  <li>Mark this task as complete/skipped</li>
  </ul>
  <p>Each challenge module would be responsible for storing task data in it&#8217;s own<br />
  format (or using database tables specific to the module).  The user&#8217;s web-browser would talk to a common front-end component that would invoke the individual challenge modules.  The challenge modules would return HTML/Javascript/JSON suitable that the browser would use in displaying information about the task to the user. </p>
  <p><strong>Proposal 2: Shared Task Representation</strong></p>
  <p>Challenge modules compute a list of tasks by examining a mixture of OSM and external data sources.  The challenge modules then convert these tasks into a common representation made up of a task and associated sets of task_objects.  A task to find add a missing bridge might be made up of three objects. An object for the place we think the bridge belongs, an object for the road the bridge is part of and a third object for the road/stream that the bridge crosses over.  These tasks and associated objects get stored in a database.  Tasks and associated objects for different challenges get stored in the same pair of database tables.  The web application that users interact with can then display and manipulate information about tasks from all challenges.<br />
  <strong><br />
  Contrasting the options<br />
  </strong><br />
  When designing software systems you often have two or more different approaches to a given problem.  Deciding which option to proceed with is the challenge software designers have to face.  Some of the questions I asked myself this weekend included</p>
  <ul>
  <li><em>How can code between different challenges be shared or re-used in each of the options? </em>.  The second approach sounds like it has more places where code can be re-used because all challenges use the same data model for assigning tasks</li>
  <li><em>Is one approach more flexible than the other? Are there things different challenges might deal with differently that will be hard to accommodate with one of the designs? </em> Our concern with the second proposal was that we were generalizing all challenges to object + geometry and not much else.  What if some challenges needed to display extra things such as textual hints?</li>
  <li><em>How do the proposals make it easier or harder for someone to come along and develop a new challenge?</em>   In the first approach a challenge developer can has free reign to implement things in any fashion they like as long as they implement the task assignment API I discuss above.  In the second proposal challenge authors can still pick their programming language and methods for detecting challenges but they must use the database schema we provide for storing information about tasks.  With the second approach challenge developers only need to worry about writing programs to find challenge tasks and store them in the database. Task assignment/management is handled by the common code</li>
  </ul>
  <p><strong><br />
  A third proposal<br />
  </strong><br />
  We were still debating the merits of these different approaches on  late-morning on Sunday when we decided to take a break and go for lunch.  One would think that a group of mappers would know lots of places nearby for lunch in downtown Toronto but we seemed to be wondering aimlessly back and forth down the street.   While we were doing this we came up with the idea of a shared database schema approach like proposal 2 but letting each challenge store a blob of  JSON with the task.  Each challenge would also implement challenge specific rendering/display could that would know how to take the JSON describing the task and display information about the task to the user in a meaning full way.</p>
  <p>I want to thank <a href="http://weait.com">Richard Weait</a> for organizing this weekend and Serge and Martijn for letting me participate in the design process.  Next we need to finish implementing what we discussed.  This should help us figure out if our ideas were good ideas.</p>
  ]]></content:encoded>
  							<wfw:commentRss>https://scanningpages.wordpress.com/2013/03/13/maproulette-hack-weekend/feed/</wfw:commentRss>
  		<slash:comments>0</slash:comments>
  						
  		<media:content url="https://1.gravatar.com/avatar/42e7367ea86bb9c5fc44b02c814d9e54?s=96&#38;d=identicon&#38;r=G" medium="image">
  			<media:title type="html">scanningpages</media:title>
  		</media:content>
  	</item>
  		<item>
  		<title>Python urllib2, cookies and OpenStreetMap</title>
  		<link>https://scanningpages.wordpress.com/2012/12/29/python-urllib2-cookies-and-openstreetmap/</link>
  				<comments>https://scanningpages.wordpress.com/2012/12/29/python-urllib2-cookies-and-openstreetmap/#comments</comments>
  				<pubDate>Sat, 29 Dec 2012 06:00:36 +0000</pubDate>
  		<dc:creator><![CDATA[scanningpages]]></dc:creator>
  				<category><![CDATA[openstreetmap]]></category>
  		<category><![CDATA[cookies]]></category>
  		<category><![CDATA[osm]]></category>
  		<category><![CDATA[python]]></category>
  		<category><![CDATA[software]]></category>
  		<category><![CDATA[urllib2]]></category>

  		<guid isPermaLink="false">http://scanningpages.wordpress.com/?p=576</guid>
  				<description><![CDATA[I found the documentation on urllib2 a bit unclear about how to get cookie handling working properly. I was working on a python script that needed to contact the OpenStreetMap web server, login with my OSM credentials and interact with the website. The first step is to setup a urllib2 opener instance that is configured [&#8230;]]]></description>
  								<content:encoded><![CDATA[<p>I found the documentation on urllib2 a bit unclear about how to get cookie handling working properly.</p>
  <p>I was working on a python script that needed to contact the OpenStreetMap web server, login with my OSM credentials and interact with the website.</p>
  <p>The first step is to setup a urllib2 opener instance that is configured to store cookies.</p>
  <p><code></p>
  <pre>
  import cookielib,Cookie,urllib2,urllib
  import xml.etree.cElementTree as ElementTree

  cookies = cookielib.CookieJar()
  opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookies))
  </pre>
  <p></code></p>
  <p>This will create an opener that can be used to retrieve URL&#8217;s.  Any cookies set in the HTTP response will be stored in the cookie&#8217;s cookie jar.  If I needed to add additional openers (ie for special redirect handling) I would just add them as additional parameters to the build_opener call.   ie urllib2.build_opener(handler1, handler2,handler3)&#8230;</p>
  <p>Next we need to contact OpenStreetMap to get a blank login form.  The blank login screen has a hidden variable &#8216;authenticity_token&#8217; that needs to be passed back as part of the POST with my login credentials<br />
   <code></p>
  <pre>
  inptag = '{http://www.w3.org/1999/xhtml}input'
  formtag = '{http://www.w3.org/1999/xhtml}form'
  # fetch the blank login form
  response_tokenfetch = opener.open(request)
  html = response_tokenfetch.read()
  htmlfile=StringIO.StringIO(html)
  # parse the HTML elements in the form
  # extract any input fields for later resubmission
  # this will pick up the authenticity_token and anything else
  xml_tree = ElementTree.parse(htmlfile)
  for form in xml_tree.getiterator(formtag):                
      for field in form.getiterator(inptag):
          if 'name' in field.attrib and 'value' in field.attrib:
                  login_payload[field.attrib['name']] = field.attrib['value']
  login_payload['username'] = username
  login_payload['password'] = password
  login_payload['remember_me'] = 'yes'
  login_payload['cookie_test'] = 'true'
  </pre>
  <p></code></p>
  <p>Next we submit the LOGIN request as a POST.  Any session cookies returned as part of the blank form will be added to the second request.<br />
  <code></p>
  <pre>
  cookies.add_cookie_header(request)
  response = opener.open(request,urllib.urlencode(login_payload))
  </pre>
  <p></code></p>
  <p>If our login was successful then cookies contains an _osm_session and _osm_username that will be used in subsequent API calls.</p>
  <p><code></p>
  <pre>
  request2=urllib2.Request('http://api06.dev.openstreetmap.org/user/stevens/inbox)
  cookies.add_cookie_header(request2)
  response2 = opener.open(request2)
  html=response2.read()
  </pre>
  <p></code></p>
  <p>You could then parse the HTML to extract a list of messages.<br />
  If your using the formal OpenStreetMap API (ie calls under /api/0.6/&#8230;) then you should instead use oauth for authentication instead of logging in through the website.  Some OSM features such as messaging can only be accessed by pretending to be a web session and parsing/faking HTML.</p>
  ]]></content:encoded>
  							<wfw:commentRss>https://scanningpages.wordpress.com/2012/12/29/python-urllib2-cookies-and-openstreetmap/feed/</wfw:commentRss>
  		<slash:comments>2</slash:comments>
  						
  		<media:content url="https://1.gravatar.com/avatar/42e7367ea86bb9c5fc44b02c814d9e54?s=96&#38;d=identicon&#38;r=G" medium="image">
  			<media:title type="html">scanningpages</media:title>
  		</media:content>
  	</item>
  		<item>
  		<title>Learning from mistakes</title>
  		<link>https://scanningpages.wordpress.com/2012/08/16/learning-from-mistakes/</link>
  				<comments>https://scanningpages.wordpress.com/2012/08/16/learning-from-mistakes/#respond</comments>
  				<pubDate>Thu, 16 Aug 2012 23:38:02 +0000</pubDate>
  		<dc:creator><![CDATA[scanningpages]]></dc:creator>
  				<category><![CDATA[openstreetmap]]></category>
  		<category><![CDATA[failure]]></category>
  		<category><![CDATA[foss4g]]></category>
  		<category><![CDATA[gis]]></category>
  		<category><![CDATA[mistakes]]></category>
  		<category><![CDATA[open source]]></category>
  		<category><![CDATA[osgeo]]></category>
  		<category><![CDATA[post-mortem]]></category>
  		<category><![CDATA[project-management]]></category>
  		<category><![CDATA[technology]]></category>

  		<guid isPermaLink="false">http://scanningpages.wordpress.com/?p=549</guid>
  				<description><![CDATA[Last year I attended and wrote about FOSS4G 2011 in Denver. The 2012 FOSS4G event was supposed to take place in Beijing this September. Last month OSGEO announced that the conference was cancelled. Cameron Shorter has posted a detailed analysis on how the conference fell apart. His post does a great job of looking at [&#8230;]]]></description>
  								<content:encoded><![CDATA[<p>Last year I attended and wrote about <a href="https://scanningpages.wordpress.com/2011/09/17/big-tent-conferences/">FOSS4G 2011</a> in Denver.  The 2012 FOSS4G event was supposed to take place in Beijing this September.  Last month OSGEO announced that the conference was cancelled.<br />
  <a href="http://cameronshorter.blogspot.ca/">Cameron Shorter</a> has posted a <a href="http://lists.osgeo.org/pipermail/conference_dev/2012-August/001886.html">detailed analysis</a> on how the conference fell apart.  His post does a great job of looking at the sequence of events that lead up to the current situation.</p>
  <p><span id="more-549"></span></p>
  <p>In the corporate world when an event, company or product falls on its face the detailed reasons are hushed up and rarely get a public airing.  When the government screws up sometimes we get an expensive public inquiry where lawyers argue about how someone else did or didn&#8217;t screw up.  OSGEO has a tradition of building consensus and discussing non-technical issues on their mailing lists.  Good open-source projects not only be give us amazing software but they allow us to learn valuable lessons on public relations, finances and management by letting us watch their inner-workings.</p>
  <p>When I was learning too fly airplanes I made a point of regularly reading the investigation reports of plane crashes. A common theme was that a single mistake didn&#8217;t cause the accident but a chain of mistakes, malfunctions and bad luck all added up to the crash. Common thinking in the aviation community is that if you learn about the mistakes others have made then you will be more likely to avoid making the same mistakes.  The story Cameron tells is about limited options, ignoring red flags, communication break downs and bad luck.  These are all issues we see everyday across all aspects of life.</p>
  <p>The mailing list discussion about the lessons that can be learned for future FOSS4G events is still on-going but as outsiders we can follow their discussion and try to learn from their mistakes.  When our events, initiatives or products don&#8217;t succeed we should also conduct a carefully look at how we got to that state and what things, with 20/20 hindsight, we should have paid more attention too.</p>
  ]]></content:encoded>
  							<wfw:commentRss>https://scanningpages.wordpress.com/2012/08/16/learning-from-mistakes/feed/</wfw:commentRss>
  		<slash:comments>0</slash:comments>
  						
  		<media:content url="https://1.gravatar.com/avatar/42e7367ea86bb9c5fc44b02c814d9e54?s=96&#38;d=identicon&#38;r=G" medium="image">
  			<media:title type="html">scanningpages</media:title>
  		</media:content>
  	</item>
  		<item>
  		<title>PGCon2012 &#8211; Making Maps</title>
  		<link>https://scanningpages.wordpress.com/2012/05/22/pgcon2012-making-maps/</link>
  				<comments>https://scanningpages.wordpress.com/2012/05/22/pgcon2012-making-maps/#comments</comments>
  				<pubDate>Tue, 22 May 2012 01:16:34 +0000</pubDate>
  		<dc:creator><![CDATA[scanningpages]]></dc:creator>
  				<category><![CDATA[openstreetmap]]></category>
  		<category><![CDATA[postgresql]]></category>
  		<category><![CDATA[Presentations]]></category>
  		<category><![CDATA[fosslc]]></category>
  		<category><![CDATA[gis]]></category>
  		<category><![CDATA[mapnik]]></category>
  		<category><![CDATA[maps]]></category>
  		<category><![CDATA[osm]]></category>
  		<category><![CDATA[pgcon]]></category>
  		<category><![CDATA[postgis]]></category>
  		<category><![CDATA[qgis]]></category>
  		<category><![CDATA[tilemill]]></category>

  		<guid isPermaLink="false">http://scanningpages.wordpress.com/?p=515</guid>
  				<description><![CDATA[Slides from my 2012 PGCon presentation Making your Own Maps are now available The presentation covered Common reasons people render their own maps Where to get OpenStreetMap data and how to load it into your PostGIS database How to use Tilemill to design your own map style How to render map tiles, both statically and [&#8230;]]]></description>
  								<content:encoded><![CDATA[<p>Slides from my 2012 PGCon presentation <a href="http://www.pgcon.org/2012/schedule/events/432.en.html" target="_blank"><em>Making your Own Maps</em></a> are now  <a href="https://scanningpages.files.wordpress.com/2012/05/pgcon_makingmaps2012.pdf">available</a></p>
  <p><a href="https://scanningpages.files.wordpress.com/2012/05/pgcon_makingmaps2012.pdf"><img data-attachment-id="516" data-permalink="https://scanningpages.wordpress.com/2012/05/22/pgcon2012-making-maps/pgcon2012/" data-orig-file="https://scanningpages.files.wordpress.com/2012/05/pgcon2012.png" data-orig-size="1058,794" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;}" data-image-title="pgcon2012" data-image-description="" data-medium-file="https://scanningpages.files.wordpress.com/2012/05/pgcon2012.png?w=300" data-large-file="https://scanningpages.files.wordpress.com/2012/05/pgcon2012.png?w=614" src="https://scanningpages.files.wordpress.com/2012/05/pgcon2012.png?w=300&#038;h=225" alt="" title="pgcon2012" width="300" height="225" class="aligncenter size-medium wp-image-516" srcset="https://scanningpages.files.wordpress.com/2012/05/pgcon2012.png?w=300&amp;h=225 300w, https://scanningpages.files.wordpress.com/2012/05/pgcon2012.png?w=600&amp;h=450 600w, https://scanningpages.files.wordpress.com/2012/05/pgcon2012.png?w=150&amp;h=113 150w" sizes="(max-width: 300px) 100vw, 300px" /></a></p>
  <p>The presentation covered</p>
  <ul>
  <li>Common reasons people render their own maps</li>
  <li>Where to get OpenStreetMap data and how to load it into your PostGIS database</li>
  <li>How to use Tilemill to design your own map style</li>
  <li>How to render map tiles, both statically and dynamically </li>
  <li>How to use OpenLayers to display your map</li>
  </ul>
  <p>The presentation was recorded.  I will update this post when the recording comes online.</p>
  <p>I&#8217;ve been a regular attendee of PGCON since the first year it was held in Ottawa.  Like past years I enjoyed the conference and I would like to thank Dan Langille for putting together another first-rate conference.  My favourite part of PostgreSQL conferences is meeting and reconnecting with users people in the community.   </p>
  <p>Updated: The video is available <a href="http://www.youtube.com/watch?v=yctOoHoiQJ4">here</a></p>
  ]]></content:encoded>
  							<wfw:commentRss>https://scanningpages.wordpress.com/2012/05/22/pgcon2012-making-maps/feed/</wfw:commentRss>
  		<slash:comments>3</slash:comments>
  						
  		<media:content url="https://1.gravatar.com/avatar/42e7367ea86bb9c5fc44b02c814d9e54?s=96&#38;d=identicon&#38;r=G" medium="image">
  			<media:title type="html">scanningpages</media:title>
  		</media:content>

  		<media:content url="https://scanningpages.files.wordpress.com/2012/05/pgcon2012.png?w=300" medium="image">
  			<media:title type="html">pgcon2012</media:title>
  		</media:content>
  	</item>
  		<item>
  		<title>Penlake Map May 2012</title>
  		<link>https://scanningpages.wordpress.com/2012/05/06/penlake-map-may-2012/</link>
  				<comments>https://scanningpages.wordpress.com/2012/05/06/penlake-map-may-2012/#respond</comments>
  				<pubDate>Sun, 06 May 2012 18:53:50 +0000</pubDate>
  		<dc:creator><![CDATA[scanningpages]]></dc:creator>
  				<category><![CDATA[openstreetmap]]></category>
  		<category><![CDATA[gis]]></category>
  		<category><![CDATA[huntsville ontario]]></category>
  		<category><![CDATA[maps]]></category>
  		<category><![CDATA[osm]]></category>
  		<category><![CDATA[pen lake]]></category>
  		<category><![CDATA[peninsula lake]]></category>
  		<category><![CDATA[penlakemap]]></category>
  		<category><![CDATA[tilemill]]></category>

  		<guid isPermaLink="false">http://scanningpages.wordpress.com/?p=501</guid>
  				<description><![CDATA[I have published a new version of the Peninsula Lake map. This edition includes new cartography and map updates based on my survey work last summer. I had written the previous map style in Cascadnik, a CSS like language that translates to mapnik XML. Cascadnik has since been replaced by Carto so I have decided [&#8230;]]]></description>
  								<content:encoded><![CDATA[<div data-shortcode="caption" id="attachment_508" style="width: 310px" class="wp-caption aligncenter"><a href="https://scanningpages.files.wordpress.com/2012/05/penlake-2012-05.png"><img aria-describedby="caption-attachment-508" data-attachment-id="508" data-permalink="https://scanningpages.wordpress.com/2012/05/06/penlake-map-may-2012/penlake-2012-05-small/" data-orig-file="https://scanningpages.files.wordpress.com/2012/05/penlake-2012-05-small1.png" data-orig-size="640,354" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;}" data-image-title="penlake.2012.05.small" data-image-description="&lt;p&gt;The May 2012 edition of the Peninsula lake map.&lt;/p&gt;
  " data-medium-file="https://scanningpages.files.wordpress.com/2012/05/penlake-2012-05-small1.png?w=300" data-large-file="https://scanningpages.files.wordpress.com/2012/05/penlake-2012-05-small1.png?w=614" src="https://scanningpages.files.wordpress.com/2012/05/penlake-2012-05-small1.png?w=300&#038;h=165" alt="Peninsula Lake Map" title="penlake.2012.05.small" width="300" height="165" class="size-medium wp-image-508" srcset="https://scanningpages.files.wordpress.com/2012/05/penlake-2012-05-small1.png?w=298&amp;h=165 298w, https://scanningpages.files.wordpress.com/2012/05/penlake-2012-05-small1.png?w=597&amp;h=330 597w, https://scanningpages.files.wordpress.com/2012/05/penlake-2012-05-small1.png?w=150&amp;h=83 150w" sizes="(max-width: 300px) 100vw, 300px" /></a><p id="caption-attachment-508" class="wp-caption-text">Peninsula Lake, Huntsville Ontario</p></div>
  <p>I have published a new version of the Peninsula Lake map.  This edition includes new cartography and map updates based on my survey work last summer. </p>
  <p>I had written the previous map style in Cascadnik, a CSS like language that translates to mapnik XML.   Cascadnik has since been replaced by Carto so I have decided to rewrite the map style using <a href="http://mapbox.com/tilemill" target="_blank">TileMill</a> (Map design software that uses Carto and Mapnik). </p>
  <p>One of my goals while redesigning the map was to put to use what I have learned about cartography over the past year.  The original PenLake map style was based on a sample Cascadnik style adapted to my colour scheme, nautical features and a print scale.   The new style was designed by me from scratch to focus on</p>
  <ul>
  <li>The lake and nearby rivers, streams and marshes</li>
  <li>Recreational attractions around the lake including hotels and golf courses</li>
  <li>Roads near the lake</li>
  <li>The water depth of the lake including suspected rocks</li>
  </ul>
  <p>This style is much smaller than the previous one ( the style is less than 300 lines of CSS) and it behaves well at different zoom levels between 12 and 18.   I suspect the number of layers can still be further reduced for faster rendering times.</p>
  <p>I have made a web version of the map available on <a href="http://www.mapexplorer.org/penlakemap" title="Peninsula Lake Map" target="_blank">Peninsula Lake Map</a> my map <a href="http://www.mapexplorer.org">hosting site</a>.  The map tiles are currently being served by <a href="http://www.mapbox.com" title="MapBox" target="_blank">MapBox.</a></p>
  <p>Some of the water depth data in this version of the map is based on observations I manually made while kayaking on the lake in the summer of 2011.   I have tried to correct this data to match the sounding datum used by the CHS in their observations.  I don&#8217;t expect the observations taken by me with a kayak, rope and a tape measure to match the accuracy of the CHS with their fancy sonar but we have to work with the equipment that we have available to us.  The depths on the map should be based on a datum of 913 feet.</p>
  <p>The Source code is still available from <a href="http://github.com/ssinger/penlakemap" target="_blank">github</a></p>
  ]]></content:encoded>
  							<wfw:commentRss>https://scanningpages.wordpress.com/2012/05/06/penlake-map-may-2012/feed/</wfw:commentRss>
  		<slash:comments>0</slash:comments>
  						
  		<media:content url="https://1.gravatar.com/avatar/42e7367ea86bb9c5fc44b02c814d9e54?s=96&#38;d=identicon&#38;r=G" medium="image">
  			<media:title type="html">scanningpages</media:title>
  		</media:content>

  		<media:content url="https://scanningpages.files.wordpress.com/2012/05/penlake-2012-05-small1.png?w=300" medium="image">
  			<media:title type="html">penlake.2012.05.small</media:title>
  		</media:content>
  	</item>
  		<item>
  		<title>OSM Hack Weekend (Toronto 2012)</title>
  		<link>https://scanningpages.wordpress.com/2012/03/18/osm-hack-weekend-toronto-2012/</link>
  				<comments>https://scanningpages.wordpress.com/2012/03/18/osm-hack-weekend-toronto-2012/#comments</comments>
  				<pubDate>Sun, 18 Mar 2012 15:34:02 +0000</pubDate>
  		<dc:creator><![CDATA[scanningpages]]></dc:creator>
  				<category><![CDATA[openstreetmap]]></category>
  		<category><![CDATA[hack weekend]]></category>
  		<category><![CDATA[osm]]></category>
  		<category><![CDATA[postgresql]]></category>
  		<category><![CDATA[rails port]]></category>
  		<category><![CDATA[ryerson]]></category>
  		<category><![CDATA[toronto]]></category>

  		<guid isPermaLink="false">http://scanningpages.wordpress.com/?p=485</guid>
  				<description><![CDATA[The Toronto OSM hack weekend (2012) started on friday with an introduction to OpenStreetMap presentation at the Ryerson Geography department. Experienced OpenStreetMap presenter, Richard Weait gave a talk to a room of Geography students, staff and faculty. After Richard explained the basics of OpenStreetMap A.J. and Tristen gave a short presentation on some of the [&#8230;]]]></description>
  								<content:encoded><![CDATA[<p>The <a href="http://wiki.openstreetmap.org/wiki/Toronto_Hack_Weekend_March_2012">Toronto OSM hack weekend (2012)</a> started on friday with an introduction to OpenStreetMap presentation at the <a href="http://www.ryerson.ca/geography/">Ryerson Geography</a> department. Experienced OpenStreetMap presenter, <a href="http://wiki.openstreetmap.org/wiki/User:Rw">Richard Weait</a> gave a talk to a room of Geography students, staff and faculty.  After Richard explained the basics of OpenStreetMap A.J. and Tristen gave a short presentation on some of the work that they have been doing for <a href="http://www.mapbox.com">MapBox</a>.</p>
  <p>After the presentations we did a QA session with the audience.  The OSM &#8216;<em>experts</em>&#8216; in the room tried to answer the questions from the audience.  Most people in the audience were geographers and GIS people who had already heard of OSM.  A number of the attendees already had worked with various components in the OSM stack. We had questions on projections, layers, TileMill, Mapnick, tagging and an assortment of other topics.</p>
  <p>After the Q/A  I went with A.J and Tristen to Chiplote for some Burritos. At Chipotle we learned three things. 1) Chipotle in Washington does more of the meat preperation in the back room than they do in Toronto; Toronto should take a hint.  2) Keeping the Burrito in the tinfoil wrap it in keeps it from falling apart as you eat it.  3) Next time I need to remember to bring Buritos back to my fellow hackers. Friday night we had a social at the hack weekend social HQ, there was plenty of beer and the room was packed.</p>
  <p>Saturday morning I took the train back downtown for the first day of hacking. We had local OSM developers and out of town hackers from three different countries participate along with a group of Ryerson Geography students.  I spent some time Saturday morning helping someone get the <a href="http://wiki.openstreetmap.org/wiki/Rails_port">rails port</a> working on their OSX laptop.   The part they seemed to have the most difficulty with was getting PostgresSQL running.  This tells me two things. First that the work people in the OSM community have done to make the rails port easier to install has paid off and some PostgreSQL binary distributions could make their user experience a bit better.</p>
  <p>I had suggested the EDB one-click installer for OSX but they opt&#8217;d for the <a href="http://www.kyngchaos.com/">KyngChao</a> binaries.  They were okay installing the package but they weren&#8217;t clear how to startup PostgreSQL and connect to it.  I feel that the EDB installer makes it easier for an inexperienced PostgreSQL user to get up and running.</p>
  <p>In addition to helping people with installing PostgreSQL I also helped some people with SQL queries and helped with changes to the website look cleaner for users logged in but haven&#8217;t uploaded any GPX traces. </p>
  <p>Saturday night we had another party at social HQ. Party highlights included beer with ingredients from seven continents, comparision of political systems, talk of election rigging. Thai food, wine and beer were also spotted. Most of the geography students from the hack sessions were able to make it out and saw how fun OSM parties can be.  I hope to see them at the regular Toronto OSM pub nights.  </p>
  <p>The <a href="http://www.gotransit.ca">GO train</a> back to Oakville was pretty full. This was St. Patricks day and the streets of Toronto were filled with drunk people in green.  The 11:43pm train to the suburbs is a bit too early for serious leprecons.   Many of the riders had been at a Van Halen concert, it didn&#8217;t seem like many of them were interested in making maps.</p>
  <p>There is a saying in the OpenStreetMap world that if you want to build an local OpenStreetMap community you should import a bunch of Germans.  If you can&#8217;t import some Germans the next best thing is to import Richard Weait.   He almost single-handedly organized the Toronto hack weekend, secured a venue and provided food, drinks and social entertainment.  I also want to thank the Ryerson Geography department and <a href="http://www.ryerson.ca/~crinner/">Claus Rinner </a>for letting us use their facilities and providing eager GIS students.</p>
  ]]></content:encoded>
  							<wfw:commentRss>https://scanningpages.wordpress.com/2012/03/18/osm-hack-weekend-toronto-2012/feed/</wfw:commentRss>
  		<slash:comments>1</slash:comments>
  						
  		<media:content url="https://1.gravatar.com/avatar/42e7367ea86bb9c5fc44b02c814d9e54?s=96&#38;d=identicon&#38;r=G" medium="image">
  			<media:title type="html">scanningpages</media:title>
  		</media:content>
  	</item>
  		<item>
  		<title>Virgin Roads</title>
  		<link>https://scanningpages.wordpress.com/2011/11/17/virgin-roads/</link>
  				<comments>https://scanningpages.wordpress.com/2011/11/17/virgin-roads/#comments</comments>
  				<pubDate>Thu, 17 Nov 2011 03:43:10 +0000</pubDate>
  		<dc:creator><![CDATA[scanningpages]]></dc:creator>
  				<category><![CDATA[openstreetmap]]></category>
  		<category><![CDATA[new houses]]></category>
  		<category><![CDATA[oakville]]></category>
  		<category><![CDATA[osm]]></category>
  		<category><![CDATA[sub-divisions]]></category>

  		<guid isPermaLink="false">http://scanningpages.wordpress.com/?p=421</guid>
  				<description><![CDATA[The neighborhood I live in is between 20 and 30 years old, other parts of Oakville are under constant growth. When a new sub-division is built the developer will put in sewers, roads and start building houses. Sometimes the street signs will look home-made other times proper street signs will already be up. When I [&#8230;]]]></description>
  								<content:encoded><![CDATA[<p><a href="https://scanningpages.files.wordpress.com/2011/11/dsc_0400-1.jpg"><img data-attachment-id="423" data-permalink="https://scanningpages.wordpress.com/2011/11/17/virgin-roads/dsc_0400-1/" data-orig-file="https://scanningpages.files.wordpress.com/2011/11/dsc_0400-1.jpg" data-orig-size="640,429" data-comments-opened="1" data-image-meta="{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;}" data-image-title="DSC_0400.1" data-image-description="&lt;p&gt;Some houses under construction in Oakville&lt;/p&gt;
  " data-medium-file="https://scanningpages.files.wordpress.com/2011/11/dsc_0400-1.jpg?w=300" data-large-file="https://scanningpages.files.wordpress.com/2011/11/dsc_0400-1.jpg?w=614" src="https://scanningpages.files.wordpress.com/2011/11/dsc_0400-1.jpg?w=300&#038;h=201" alt="" title="DSC_0400.1" width="300" height="201" class="aligncenter size-medium wp-image-423" srcset="https://scanningpages.files.wordpress.com/2011/11/dsc_0400-1.jpg?w=300&amp;h=201 300w, https://scanningpages.files.wordpress.com/2011/11/dsc_0400-1.jpg?w=600&amp;h=402 600w, https://scanningpages.files.wordpress.com/2011/11/dsc_0400-1.jpg?w=150&amp;h=101 150w" sizes="(max-width: 300px) 100vw, 300px" /></a></p>
  <p>The neighborhood I live in is between 20 and 30 years old, other parts of Oakville are under constant growth. When a new sub-division is built the developer will put in sewers, roads and start building houses. Sometimes the street signs will look home-made other times proper street signs will already be up.</p>
  <p>When I see a new subdivision going up near my I try drive (or walk) the roads with my GPS  capturing the data for OpenStreetMap. Often my GPS traces in OpenStreetMap will be the first time these roads show up on a map outside of the towns GIS. It might take years before some of the commercial maps send trucks to map these roads.</p>
  <p>On Sunday I was driving past a sub-division that was being built and decided to <a title="map" href="http://www.openstreetmap.org/?lat=43.46939&amp;lon=-79.74569&amp;zoom=16&amp;layers=M" target="_blank">map</a> some of the roads. I got about GPS traces and names for about half of the roads. I need to make a point of going back and finishing the soon. Sub-division construction sites usually have construction workers and heavy equipment moving about.  It is important to pay close attention to your surroundings and the condition of the road your mapping often it is a work in progress. </p>
  <p>I doubt anyone is going to be moving into these houses until the spring. In the spring when people move into their new houses they will be able to use OpenStreetMap to invite there friends to the house-warming party.</p>
  ]]></content:encoded>
  							<wfw:commentRss>https://scanningpages.wordpress.com/2011/11/17/virgin-roads/feed/</wfw:commentRss>
  		<slash:comments>1</slash:comments>
  						
  		<media:content url="https://1.gravatar.com/avatar/42e7367ea86bb9c5fc44b02c814d9e54?s=96&#38;d=identicon&#38;r=G" medium="image">
  			<media:title type="html">scanningpages</media:title>
  		</media:content>

  		<media:content url="https://scanningpages.files.wordpress.com/2011/11/dsc_0400-1.jpg?w=300" medium="image">
  			<media:title type="html">DSC_0400.1</media:title>
  		</media:content>
  	</item>
  		<item>
  		<title>PostGIS Replication @ FOSS4G</title>
  		<link>https://scanningpages.wordpress.com/2011/09/18/postgis-replication-foss4g/</link>
  				<comments>https://scanningpages.wordpress.com/2011/09/18/postgis-replication-foss4g/#respond</comments>
  				<pubDate>Sun, 18 Sep 2011 21:42:23 +0000</pubDate>
  		<dc:creator><![CDATA[scanningpages]]></dc:creator>
  				<category><![CDATA[openstreetmap]]></category>
  		<category><![CDATA[postgresql]]></category>
  		<category><![CDATA[Presentations]]></category>
  		<category><![CDATA[disaster recovery]]></category>
  		<category><![CDATA[foss4g]]></category>
  		<category><![CDATA[gis]]></category>
  		<category><![CDATA[postgis]]></category>
  		<category><![CDATA[replication]]></category>
  		<category><![CDATA[slony]]></category>

  		<guid isPermaLink="false">http://scanningpages.wordpress.com/?p=413</guid>
  				<description><![CDATA[My talk on PostGIS replication at FOSS4G 2011 went well. It looked like there were about 150 people in the room. Most of them had not yet deployed a PostGIS replication solution. My talk covered Slony and streaming replication. It gave an overview of different replication patterns that can crop on in the GIS space. [&#8230;]]]></description>
  								<content:encoded><![CDATA[<p>My talk on PostGIS replication at FOSS4G 2011 went well.  It looked like there were about 150 people in the room.  Most of them had not yet deployed a PostGIS replication solution.</p>
  <p>My talk covered Slony and streaming replication.  It gave an overview of different replication patterns that can crop on in the GIS space.  I then gave an overview of the key features and limitation of Slony and streaming replication.</p>
  <p>A video of the talk is available at <a href="http://www.fosslc.org/drupal/content/postgresql-replication-0">FOSSLC</a></p>
  <p>My slides are available <a href='https://scanningpages.files.wordpress.com/2011/09/postgis_replication_foss4g_2011.pdf'>here</a></p>
  ]]></content:encoded>
  							<wfw:commentRss>https://scanningpages.wordpress.com/2011/09/18/postgis-replication-foss4g/feed/</wfw:commentRss>
  		<slash:comments>0</slash:comments>
  						
  		<media:content url="https://1.gravatar.com/avatar/42e7367ea86bb9c5fc44b02c814d9e54?s=96&#38;d=identicon&#38;r=G" medium="image">
  			<media:title type="html">scanningpages</media:title>
  		</media:content>
  	</item>
  		<item>
  		<title>Big Tent Conferences</title>
  		<link>https://scanningpages.wordpress.com/2011/09/17/big-tent-conferences/</link>
  				<comments>https://scanningpages.wordpress.com/2011/09/17/big-tent-conferences/#comments</comments>
  				<pubDate>Sat, 17 Sep 2011 22:49:50 +0000</pubDate>
  		<dc:creator><![CDATA[scanningpages]]></dc:creator>
  				<category><![CDATA[openstreetmap]]></category>
  		<category><![CDATA[postgresql]]></category>
  		<category><![CDATA[community]]></category>
  		<category><![CDATA[foss4g]]></category>
  		<category><![CDATA[postgis]]></category>
  		<category><![CDATA[sotm]]></category>

  		<guid isPermaLink="false">http://scanningpages.wordpress.com/?p=401</guid>
  				<description><![CDATA[This past week I attended FOSS4G in Denver, a conference run by the open-source geospatial foundation (OSGEO) that also happened to be the largest PostgreSQL conference in North America. FOSS4G is a big tent conference that attracted about 900 attendees from all over the world with over 400 of them from the United States, over [&#8230;]]]></description>
  								<content:encoded><![CDATA[<p>This past week I attended <a href="http://2011.foss4g.org" target="_blank">FOSS4G</a> in Denver, a conference run by the open-source geospatial foundation (<a href="http://www.osgeo.org" target="_blank">OSGEO</a>) that also happened to be the largest PostgreSQL conference in North America.</p>
  <p>FOSS4G is a big tent conference that attracted about 900 attendees from all over the world with over 400 of them from the United States, over 50 from Canada.<br />
  A big tent conference is a conference is similar to a big-tent political party.  The idea is appeal to a broad base of people by catering to as many groups as possible while keeping true to some key common themes.  Out of the 900 people at the conference I&#8217;d say 75% raised their hands when asked if it was their first FOSS4G.   Last years FOSS4G was held in Europe and also had over 800 attendees.<br />
  <span id="more-401"></span><br />
  At FOSS4G the common theme is open source geo-spatial software.  This includes open-source desktop GIS systems like QGIS and GRASS.  It includes open source map rendering platforms like Mapserver and Mapnik.  It also includes open source spatial databases such as PostGIS and MongoDB.   The tent is also big enough to include commercial vendors such as ESRI, Safe(authors of FME) who produce commercial products that interface with some open-source geospartial software.   Many of the software packages (open-source and commercial) represented at the conference compete with each other but they are able to share the same tent.   At the opening reception one attendee (who I won&#8217;t name) said <em>&#8220;I&#8217;m need to find Chris Schmidt and tell him why OpenLayers sucks</em>&#8221;  conferences that get everyone under one roof for a week allow us to learn from each other so we can improve our software, our documentation and our communities.  It is much harder to have a flame-war on a mailing list with someone you have sat down and had a beer with than it is to have a flame-war with someone you have never met.</p>
  <p>At FOSS4G almost everyone I spoke to used PostGIS in some fashion, some of them used couchDB or MongoDB for tasks, others had legacy Oracle Spatial or ESRI systems but everyone was aware of PostGIS and most had used it in some fashion.   If you are running PostGIS then you are running PostgreSQL. FOSS4G 2011 was the largest collection of PostgreSQL users under one roof in North America to date. As a member of the PostgreSQL community I feel we need to do a better job at making sure that geospatial types feel welcome in our community.   Excluding the core PostGIS developers I recognized less than 5 people from the PostgreSQL community at the conference and most of us were there because of our connections to other communities.  Having said that, most people I spoke to said that they loved PostGIS and PostgreSQL.</p>
  <p>The flight home from FOSS4G is also a good time to reflect on what it means to be a community conference.</p>
  <ul>
  <li> Each year OSGEO asks for submissions/proposals from the community on groups that want to host the next FOSS4G.  An OSGEO committee examines these submissions and selects a host city for the next conference.</li>
  <li> OSGEO moves the conference around the world each year.  They won&#8217;t hold the conference in the same continent two years in a row.  This makes it easier for people in different parts of the world too attend.  It also means that when the conference is in your backyard you make a special effort to attend because it probably isn&#8217;t coming back for at least three years.</li>
  <li> FOSS4G has a call for papers, the papers are then voted on by the wider community as an input into the paper selection processes.  Anyone in the community can have input into which talks are selected.</li>
  <li> After FOSS4G a financial report on the conference along with many other documents used in planning the conference are published in a subversion repository.  This acts as a resource for future groups considering hosting FOSS4G.  It also means that the revenues and expenses are public so people can can see where the money goes.</li>
  <li> FOSS4G does not just appeal to developers.  In addition to developers and sysadmins there where many cartographers and geographers at the conference</li>
  </ul>
  <p>Next year FOSS4G will be held in Bejing.  Various political, cultural, logistical and financial reasons will make it hard for many of the American attendees to go to FOSS4G.  Some people in Denver were talking about how there should be a North American FOSS4G next year as well. My thoughts on this are</p>
  <ul>
  <li>It is important to not dilute the FOSS4G brand, FOSS4G is the big geo-spatial conference that travels around the world.  If you start splitting the FOSS4G brand you will soon end up with  a FOSS4G-EU, FOSS4G-US , FOSS4G-Asia instead of a FOSS4G.   Other communities do this (*BSD as an example).   Figure out the relationship between FOSS4G, OSGEO and regional conferences before someone announces FOSS4G-US, maybe these regional conferences should be called something other than FOSS4G</li>
  <li> If you are thinking about creating another North American geo-spatial conference think about what other communities your conference can reach out to  (OSM, PostgreSQL, Python, etc&#8230;).  State of the Map was fun for a lot of the same reasons PGCon is fun.  You get to spend a few days meeting and hanging out with people that you regularly chat with online and everyone shares a deep passion of a very specific project.  A lot of the fun I had with FOSS4G was from the energy of being at such a large conference.</li>
  <li> If you do hold some sort of regional conference, for god sakes don&#8217;t hold the conference two weeks before the Bejing conference.  Space the conferences out so that it is easier for speakers, attendees and sponsors to attend both.  I think the next year will turn out to be a great year for the struggling OpenStreetMap community in the US because of the interest and enthusiasm generated at  SOTM and FOSS4G.  I hope FOSS4G in Bejing brings a lot more great Asian contributors into the open source geospatial tent next year.</li>
  </ul>
  <p>It feels like the plane is descending.  I will try to send this off once I&#8217;m home.</p>
  ]]></content:encoded>
  							<wfw:commentRss>https://scanningpages.wordpress.com/2011/09/17/big-tent-conferences/feed/</wfw:commentRss>
  		<slash:comments>1</slash:comments>
  						
  		<media:content url="https://1.gravatar.com/avatar/42e7367ea86bb9c5fc44b02c814d9e54?s=96&#38;d=identicon&#38;r=G" medium="image">
  			<media:title type="html">scanningpages</media:title>
  		</media:content>
  	</item>
  	</channel>
  </rss>


key: osmand
encoding: utf8
format: rss 2.0
title: OsmAnd
url: http://osmand.net/blog
feed_url: http://osmand.net/rss.xml
twitter: osmandapp
items_last_updated: 2019-09-03 12:00:00
fetched: 2019-10-20 15:30:23.069255
http_server: nginx/1.14.0 (Ubuntu)
md5: 59c454e3635ea94186b3d0162611fef2
created_at: 2019-06-09 18:11:33.501479
updated_at: 2019-10-20 15:30:23.070433
body:
  <?xml version="1.0" encoding="UTF-8"?>
  <rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  	<channel>
  		<title>OsmAnd blog</title>
  		<link>https://osmand.net/blog</link>
  		<atom:link href="https://osmand.net/rss.xml" rel="self" type="application/rss+xml" />
  		<description/>
  		<generator></generator>
  		<item>
  			<title>OsmAnd 3.0 (iOS)</title>
  			<link>https://osmand.net/blog/osmand-ios-3-0-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-ios-3-0-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd 3.0 (iOS)</h2><div class="date">September 03, 2019</div><p>As we said, we are working on improving the ios version of the application. The proof is this release. <a href="https://itunes.apple.com/us/app/osmand-maps-travel-navigate/id934850257">We're glad to announce the new OsmAnd version for iOS.</a></p><p><b>• Added Quick Action feature: instant access to various map actions (e. g. changing a map style, showing specific POI types etc.) directly from the main screen</b></p><p>The Quick Action button is a flexible menu of shortcuts providing direct access to whatever features you need to access most frequently: You select and add them to this 'toolbar'. In this way, the next time you need to add a new favorite, add a navigation destination, enable or disable voice guidance, etc. These actions are now directly accessible.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-3-0/1.jpg"/><img src="https://osmand.net/images/blog/ios-3-0/2.jpg"/><img src="https://osmand.net/images/blog/ios-3-0/3.jpg"/><img src="https://osmand.net/images/blog/ios-3-0/4.jpg"/><div style="clear:both;"></div></div><p>The quick action button can be added via the Configure screen menu. To add actions to the menu, please press 'Add action' right on its menu bar. To remove an action, go to Configure screen and tap the 'Quick action' menu, then remove any shortcuts not needed.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-3-0/5.jpg"/><img src="https://osmand.net/images/blog/ios-3-0/6.jpg"/><div style="clear:both;"></div></div><p><b>• Context menu improvements: ability to download the missing region map directly from the context menu</b></p><p>Now clicking on a point on the map, provided there is no detailed map, a menu appears with a quick download of the map.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-3-0/7.jpg"/><img src="https://osmand.net/images/blog/ios-3-0/8.jpg"/><div style="clear:both;"></div></div><p><b>• OpenStreetMap editing: added hints for tags and values to the advanced editing screen</b></p><p>Added hints for tags and values to the advanced editing screen for better experience. Now it is possible to quickly select tags from existing ones. <a href="https://osmand.net/features/osm-editing-plugin#How_to_use_it_ios">Additional information.</a></p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-3-0/11.jpg"/><div style="clear:both;"></div></div><p><b>• Improved the trip recording widget UI</b></p><p>Fixed the UI of the track recording widget, now it is clearly visible - when recording is enabled, and when disabled.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-3-0/9.jpg"/><img src="https://osmand.net/images/blog/ios-3-0/10.jpg"/><div style="clear:both;"></div></div><p><b>• Fixed an issue with navigation alerts not being refreshed</b></p><p>Sound and visual warnings now work correctly while navigating.</p><p><b>• Various improvements and bugfixes</b></p><p>And remember that only together we can achieve the best results!</p><p>New features are coming SOON!</p><p>____________________________ </p><p>OsmAnd at <a href="https://www.facebook.com/osmandapp/">Facebook</a>, <a href="https://www.twitter.com/osmandapp/">Twitter</a>, and <a href="https://www.reddit.com/r/OsmAnd/">Reddit</a>!</p> <p>Join us at our <a href="https://t.me/OsmAndMaps">group of Telegram (EN)</a>.</p> <p>Join us at our <a href="https://t.me/deosmand">group of Telegram (DE)</a>.</p> <p>Join us at our <a href="https://t.me/ruosmand">group of Telegram (RU)</a>.</p> <p><a href="https://itunes.apple.com/us/app/osmand-maps-travel-navigate/id934850257"><img src="http://osmand.net/images/app-store-badge.png" alt="Get it on App Store" height="60"/></a>]]></description>
  			<pubDate>03 Sep 2019 12:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>OsmAnd Online GPS Tracker 0.7</title>
  			<link>https://osmand.net/blog/osmand-tracker-0-7-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-tracker-0-7-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd Online GPS Tracker 0.7</h2><div class="date">August 01, 2019</div><p>Hi!</p><p>The new version of our OsmAnd Tracker is ready!</p><p>We prepared new feature for you!</p><img src="https://osmand.net/images/blog/tracker-0-7/8.png" alt="(Image not available offline)"/><h3>Live track</h3><p>Now you can see the online moving of your contacts on the map of OsmAnd with online GPS-track. For this, you should switch on "Live" at your contact in "Timeline menu". If there are many sharing contacts at this moment OsmAnd app chooses the color of tracks by random.</p><div class="blogimageblock">    <img src="https://osmand.net/images/blog/tracker-0-7/1.jpg"/>    <img src="https://osmand.net/images/blog/tracker-0-7/2.jpg"/>    <img src="https://osmand.net/images/blog/tracker-0-7/3.jpg"/>    <img src="https://osmand.net/images/blog/tracker-0-7/4.jpg"/>        <div style="clear:both;"></div></div><p>When you click "Show in OsmAnd" or mini-map you can choose special settings for GPX-track your contact in OsmAnd app and see it live moving with track on OsmAnd map.</p><div class="blogimageblock">    <img src="https://osmand.net/images/blog/tracker-0-7/5.jpg"/>    <img src="https://osmand.net/images/blog/tracker-0-7/6.jpg"/>        <div style="clear:both;"></div></div><h3>"Timeline" modified</h3><p>We added information to "Timeline" screen if you don't have data collected in the selected day. You can go to select the nearest date with one click.</p><div class="blogimageblock">    <img src="https://osmand.net/images/blog/tracker-0-7/7.jpg"/>        <div style="clear:both;"></div></div><p>Further instructions and helpful advice can be found <a href="https://osmand.net/features/tracker">here</a>.</p><p>OsmAnd at <a href="https://www.facebook.com/osmandapp/">Facebook</a>, <a href="https://www.twitter.com/osmandapp/">Twitter</a>, and <a href="https://www.reddit.com/r/OsmAnd/">Reddit</a>!</p> <p>Join us at our <a href="https://t.me/OsmAndMaps">group of Telegram (EN)</a>.</p> <p>Join us at our <a href="https://t.me/deosmand">group of Telegram (DE)</a>.</p> <p>Join us at our <a href="https://t.me/ruosmand">group of Telegram (RU)</a>.</p><p>Get OsmAnd Tracker today from <a href="https://play.google.com/store/apps/details?id=net.osmand.telegram&hl">Google play</a>.</p>.</p><p><a href="https://play.google.com/store/apps/details?id=net.osmand.telegram" target="_blank"><img src="https://raw.githubusercontent.com/osmandapp/osmandapp.github.io/9fc59e5136b6a07045eb96d052b3ce6ddde805c3/website//images/help/badge_store_google_play.png" alt="Get it on Google Play" height="60"/></a>]]></description>
  			<pubDate>01 Aug 2019 18:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>OsmAnd 2.90 (iOS)</title>
  			<link>https://osmand.net/blog/osmand-ios-2-9-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-ios-2-9-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd 2.90 (iOS)</h2><div class="date">July 25, 2019</div><p><a href="https://itunes.apple.com/us/app/osmand-maps-travel-navigate/id934850257">We're glad to announce the new OsmAnd version for iOS.</a></p><p><b>• Added support for import modified render.xml and routing.xml.</b></p><p>Now it is possible to transfer a file to the application for setting up navigation and routing (parameters for building a route).</p><p><b>• Share link now directly launch OsmAnd on a shared place.</b></p><p>The ability to open a link <i>(https://osmand.net/go?lat=11.11&lon=11.11&z=14)</i> to the coordinates through the application.</p><p><b>• Added back swipe gestures to all screens.</b></p><p>The ability to quickly navigate the screens of the application.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-2-90/2.jpg"/><div style="clear:both;"></div></div><p><b>• Added support for display and view coordinates with more formats.</b></p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-2-90/1.jpg"/><div style="clear:both;"></div></div><p><b>• Fixed Mapillary photo viewer on iPad.</b></p><p><b>• OSM Edits: added ability to select POI type without category, added upload progress, "close changeset" option is set by default, fix few bugs with uploading edits.</b></p><p>And remember that only together we can achieve the best results!</p><p>New features are coming SOON!</p><p>OsmAnd at <a href="https://www.facebook.com/osmandapp/">Facebook</a>, <a href="https://www.twitter.com/osmandapp/">Twitter</a>, and <a href="https://www.reddit.com/r/OsmAnd/">Reddit</a>!</p> <p>Join us at our <a href="https://t.me/OsmAndMaps">group of Telegram (EN)</a>.</p> <p>Join us at our <a href="https://t.me/deosmand">group of Telegram (DE)</a>.</p> <p>Join us at our <a href="https://t.me/ruosmand">group of Telegram (RU)</a>.</p> <p><a href="https://itunes.apple.com/us/app/osmand-maps-travel-navigate/id934850257"><img src="http://osmand.net/images/app-store-badge.png" alt="Get it on App Store" height="60"/></a>]]></description>
  			<pubDate>25 Jul 2019 14:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>OsmAnd 3.4</title>
  			<link>https://osmand.net/blog/osmand-3-4-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-3-4-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd 3.4</h2><div class="date">July 11, 2019</div><p>We are glad to announce the release of version 3.4! We proudly present some major improvements and new features, and hope you will enjoy them:</p><img src="https://osmand.net/images/blog/3.4/17.png" alt="(Image not available offline)"/><h3><b>Application profiles</b></h3><p>Now you can create your own profile for your needs, with a custom icon and color. Go to <i>General menu-> Settings-> Application settings</i>. Here you can select <i>Base profiles</i> to display in the navigation.</p><p>After clicking to "+Add" you can build your own profile with your own parametres: icon, avoid, elevation, configure map, screen, navigation settings...</p><p>You can add own modified version of <b>routing.xml</b> to osmand folder: ...osmand/routing.</p><div class="blogimageblock"><img src="https://osmand.net/images/blog/3.4/1.jpg"/><img src="https://osmand.net/images/blog/3.4/2.jpg"/><img src="https://osmand.net/images/blog/3.4/3.jpg"/><img src="https://osmand.net/images/blog/3.4/4.jpg"/><div style="clear:both;"></div></div><h3><b>Settings and min/max speed for profiles</b></h3><p>We add settings for default and min/max speed for profiles. You can improve base profiles for your own conditions. Go to <i>General menu-> Settings-> Application settings</i>. Here you can set special settings for <i>Base profiles</i>: <i>Routing preferences, Navigation preferences, Voice</i>. Please, click <i>Navigation settings</i> for setting up your own speed: min/max speed.</p><div class="blogimageblock"><img src="https://osmand.net/images/blog/3.4/5.jpg"/><img src="https://osmand.net/images/blog/3.4/6.jpg"/><img src="https://osmand.net/images/blog/3.4/7.jpg"/><img src="https://osmand.net/images/blog/3.4/8.jpg"/><div style="clear:both;"></div></div><h3><b>Add a widget with current coordinates</b></h3><p>For switch on this widget, please go to <i>General menu-> Configure screen (Remaining elements)</i>. After you can see your own coordinates on top of the screen.</p><p>You can change coordinate format  <i>General menu-> Settings-> General settings-> Coordinate format</i>.</p><div class="blogimageblock"><img src="https://osmand.net/images/blog/3.4/9.jpg"/><img src="https://osmand.net/images/blog/3.4/10.jpg"/><div style="clear:both;"></div></div><h3><b>The compass at the Radius ruler</b></h3><p> We add the option to show the compass on the map with the Radius ruler. Go to settings of the Radius Ruler (menu Configure screen) and choose <i>"Show compass ruler"</i>. Now you can see the compass at the Radius ruler (if you don't see it, please to click at the widget of Radius ruler).</p><div class="blogimageblock"><img src="https://osmand.net/images/blog/3.4/11.jpg"/><img src="https://osmand.net/images/blog/3.4/12.jpg"/><img src="https://osmand.net/images/blog/3.4/13.jpg"/><img src="https://osmand.net/images/blog/3.4/14.jpg"/><div style="clear:both;"></div></div><h3><b>'Turn Screen On' option</b></h3><p>At this release, we returned 'Turn Screen On' option. Now, this feature doesn't ask any permissions for correctly working. You can use it to save phone energy consumption. You should go to <i>General menu-> Settings-> Navigation settings</i>, choose a profile for setting the time for which the screen will turn on and use proximity sensor.</p><p>Proximity sensor is a very useful feature during a long trip. You should wave your hand over the top of the screen to turn on the screen while navigating.</p><div class="blogimageblock"><img src="https://osmand.net/images/blog/3.4/15.jpg"/><img src="https://osmand.net/images/blog/3.4/16.jpg"/><div style="clear:both;"></div></div><h3><b>What is more?</b></h3><p><i>- Fix background track logging.</i></p><p><i>- Improved map downloads in the background.</i></p><p><i>- Fixed Wikipedia language selection.</i></p><p><i>- Fixed compass button behavior during navigation.</i></p><p><i>- Bug fixes.</i></p><p>_________________________________________________</p><p>OsmAnd at <a href="https://www.facebook.com/osmandapp/">Facebook</a>, <a href="https://www.twitter.com/osmandapp/">Twitter</a>, and <a href="https://www.reddit.com/r/OsmAnd/">Reddit</a>!</p> <p>Join us at our <a href="https://t.me/OsmAndMaps">group of Telegram (EN)</a>.</p> <p>Join us at our <a href="https://t.me/deosmand">group of Telegram (DE)</a>.</p> <p>Join us at our <a href="https://t.me/ruosmand">group of Telegram (RU)</a>.</p><p><a href="https://play.google.com/store/apps/details?id=net.osmand" target="_blank"><img src="https://raw.githubusercontent.com/osmandapp/osmandapp.github.io/9fc59e5136b6a07045eb96d052b3ce6ddde805c3/website/images/help/badge_store_google_play.png" alt="Get it on Google Play" height="60"/></a><a href="https://www.amazon.com/OsmAnd-Maps-Navigation/dp/B00D0SA8I8" target="_blank"><img src="https://raw.githubusercontent.com/osmandapp/osmandapp.github.io/9fc59e5136b6a07045eb96d052b3ce6ddde805c3/website/images/help/badge_store_amazon.png" alt="Get it on Amazon" height="60"/></a><a href="https://itunes.apple.com/app/apple-store/id934850257?pt=2123532&amp;ct=WebSite&amp;mt=8" target="_blank"><img src="https://raw.githubusercontent.com/osmandapp/osmandapp.github.io/9fc59e5136b6a07045eb96d052b3ce6ddde805c3/website/images/app-store-badge.png" alt="Get it on App Store" height="60"/></a></p>]]></description>
  			<pubDate>11 Jul 2019 17:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>OsmAnd 2.80 (iOS)</title>
  			<link>https://osmand.net/blog/osmand-ios-2-8-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-ios-2-8-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd 2.80 (iOS)</h2><div class="date">June 09, 2019</div><p><a href="https://itunes.apple.com/us/app/osmand-maps-travel-navigate/id934850257">OsmAnd 2.80 is now available!</a></p><img src="https://osmand.net/images/blog/ios-2-72/01.png" alt="(Image not available offline)"/><h4>Mapillary</h4><p>We continue to follow our <a href="https://osmand.net/blog/2019-ny-resolutions">New Year’s resolutions</a>, where we have promised to catch the iOS version of OsmAnd up with the one on Android, and we are getting there.</p><p>We're proud to have collaborated with Mapillary for the new release. Now you can view the street-level imagery in your location or anywhere in the world using the Mapillary in OsmAnd for iOS.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-2-72/1.jpg"/><img src="https://osmand.net/images/blog/ios-2-72/2.jpg"/><img src="https://osmand.net/images/blog/ios-2-72/3.jpg"/><img src="https://osmand.net/images/blog/ios-2-72/4.jpg"/><div style="clear:both;"></div></div><p>Using the widget on the map, you'll be able to view images or videos and even add your own. To view the images and videos, you'll have to go to 'Configure screen' menu and enable the 'Mapillary' layer.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-2-72/7.jpg"/><img src="https://osmand.net/images/blog/ios-2-72/8.jpg"/><img src="https://osmand.net/images/blog/ios-2-72/9.jpg"/><img src="https://osmand.net/images/blog/ios-2-72/10.jpg"/><div style="clear:both;"></div></div> <p>Now you have access to an extensive database created by volunteers. To view media attached to a certain point on the map, just make a long tap there and open the context menu. You'll be able to scroll through images of the location easily.</p><p>Zoom in closer to view the images even more conveniently in a Mapillary window over your map. You can also press the white arrows to 'move' along the streets. To contribute to Mapillary, just press on the widget and take a picture.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-2-72/5.jpg"/><img src="https://osmand.net/images/blog/ios-2-72/6.jpg"/><div style="clear:both;"></div></div><h3>What's more in the latest version of 2.80?</h3><p>• Fixed city and postal code search.</p><p>• Improved performance of Hillshade and Contour Lines map rendering.</p><p>• Added Turkish TTS.</p><p>And remember that only together we can achieve the best results!</p><p>New features are coming SOON!</p><p><a href="https://itunes.apple.com/us/app/osmand-maps-travel-navigate/id934850257"><img src="http://osmand.net/images/app-store-badge.png" alt="Get it on App Store" height="60"/></a><p>OsmAnd at <a href="https://www.facebook.com/osmandapp/">Facebook</a>, <a href="https://www.twitter.com/osmandapp/">Twitter</a>, and <a href="https://www.reddit.com/r/OsmAnd/">Reddit</a>! Join us at our <a href="https://t.me/OsmAndMaps">group of Telegram</a>.</p>]]></description>
  			<pubDate>09 Jun 2019 14:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>OsmAnd Online GPS Tracker 0.5</title>
  			<link>https://osmand.net/blog/osmand-tracker-0-5-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-tracker-0-5-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd Online GPS Tracker 0.5</h2><div class="date">June 03, 2019</div><p>Hello everybody!</p><p>We prepared a new version of our OsmAnd Tracker - 0.5.</p><p>What the new functions are?</p><img src="https://osmand.net/images/blog/tracker-0-5/01.png" alt="(Image not available offline)"/><h3>Added contact search</h3><p>Yes, it was a problem to find the contact or your group from a telegram list. Now it's easy!</p><div class="blogimageblock">    <img src="https://osmand.net/images/blog/tracker-0-5/1.jpg"/>    <img src="https://osmand.net/images/blog/tracker-0-5/2.jpg"/>        <div style="clear:both;"></div></div><h3>Added proxy settings</h3><p>Many of our users asked us to add proxy. We did it. Use proxy inside of OsmAnd Tracker. Go to Settings and write your proxy data: <i>Settings-> Privacy-> Proxy Settings</i></p><div class="blogimageblock">    <img src="https://osmand.net/images/blog/tracker-0-5/3.jpg"/>    <img src="https://osmand.net/images/blog/tracker-0-5/4.jpg"/>        <div style="clear:both;"></div></div><h3>GPS settings: minimum speed, accuracy, and distance between points</h3><p>Now, you can control your points. You can set settings of points. If you stay at one place some time without moving, for example, your points don't send and your GPX track will be nicer. Go to <i>Settings-> GPX settings</i></p><div class="blogimageblock">    <img src="https://osmand.net/images/blog/tracker-0-5/5.jpg"/>        <div style="clear:both;"></div></div><h3>Active marker on the OsmAnd map: distance, speed, and angle of movement of your contacts</h3><p>It's very useful to see distance, speed, and angle movement of your contacts at  OsmAnd map and text in Telegram. Now you can understand at once where and how your contact move.</p><div class="blogimageblock">    <img src="https://osmand.net/images/blog/tracker-0-5/6.jpg"/>    <img src="https://osmand.net/images/blog/tracker-0-5/7.jpg"/>        <div style="clear:both;"></div></div><p>Further instructions and helpful advice can be found <a href="https://osmand.net/features/tracker">here</a>.</p><p>Get OsmAnd Tracker today from <a href="https://play.google.com/store/apps/details?id=net.osmand.telegram&hl">Google play</a>.</p>.</p><p><a href="https://play.google.com/store/apps/details?id=net.osmand.telegram" target="_blank"><img src="https://raw.githubusercontent.com/osmandapp/osmandapp.github.io/9fc59e5136b6a07045eb96d052b3ce6ddde805c3/website//images/help/badge_store_google_play.png" alt="Get it on Google Play" height="60"/></a>]]></description>
  			<pubDate>03 Jun 2019 18:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>OsmAnd 2.70 (iOS)</title>
  			<link>https://osmand.net/blog/osmand-ios-2-7-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-ios-2-7-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd 2.70 (iOS)</h2><div class="date">May 10, 2019</div><p><a href="https://itunes.apple.com/us/app/osmand-maps-travel-navigate/id934850257">OsmAnd 2.70 is now available!</a></p><img src="https://osmand.net/images/blog/ios-2-7/ios-2-7.png" alt="(Image not available offline)"/><h3>OpenStreetMap (OSM) Editing</h3><p>We continue to follow our <a href="https://osmand.net/blog/2019-ny-resolutions">New Year’s resolutions</a>, where we have promised to catch the iOS version of OsmAnd up with the one on Android, and we are getting there. OSM Editing allows you to make contributions to <a href="https://www.openstreetmap.org">OpenStreetMap</a>, a global community aimed at creating a comprehensive map of the world and providing up-to-date open-source data to every user.</p><p>The feature lets you create new objects, so-called <a href="https://wiki.openstreetmap.org/wiki/Points_of_interest">points of interest or POI</a> on the map and <a href="https://wiki.openstreetmap.org/wiki/Notes">OSM notes</a>.</p><p>So how to do it? Tap on the map where the new POI has to be placed -> 'Actions' and press 'Create POI' or 'Open OSM Note' add its name and other details like working hours, website, etc. You'll also have to register at <a href="https://www.openstreetmap.org">OpenStreetMap</a> and then provide your OSM credentials to introduce changes.</p></ul><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-2-7/ios-2-7-1.PNG"/><img src="https://osmand.net/images/blog/ios-2-7/ios-2-7-2.PNG"/><img src="https://osmand.net/images/blog/ios-2-7/ios-2-7-3.PNG"/><div style="clear:both;"></div></div><p>There are all your "Edits" and "Notes" at general menu -> 'My Places' -> 'OSM Edits'. From this menu, you can delete or upload all your edits and notes.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-2-7/ios-2-7-4.PNG"/><div style="clear:both;"></div></div><h3>My places menu</h3><p>Combined "Favorites", "Trips" and "OSM Edits" screens into a new 'My Places' screen.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-2-7/ios-2-7-5.PNG"/><img src="https://osmand.net/images/blog/ios-2-7/ios-2-7-6.PNG"/><div style="clear:both;"></div></div><h3>What's new in the latest version of 2.70?</h3><p>• Added options to control the size of map elements: use “Text Size” option to control the size of the text and icons on the map and “Map Magnifier” to control the overall size of map elements.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-2-7/ios-2-7-8.PNG"/><img src="https://osmand.net/images/blog/ios-2-7/ios-2-7-9.PNG"/><div style="clear:both;"></div></div><p>• Opening a search result on the map will now zoom in on the selected point.</p><p>• Fixed icon sizes on map.</p><p>• Optimized multiple menus for iPhone X screen.</p><p>• Added Hungarian and Hungarian (formal) TTS, added alphabetical sorting to the TTS language list.</p>	<p>And remember that only together we can achieve the best results!</p><p>New features are coming SOON!</p><p><a href="https://itunes.apple.com/us/app/osmand-maps-travel-navigate/id934850257"><img src="http://osmand.net/images/app-store-badge.png" alt="Get it on App Store" height="60"/></a>]]></description>
  			<pubDate>10 May 2019 18:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>Guideline for public transport</title>
  			<link>https://osmand.net/blog/guideline-pt</link>
  			<guid isPermaLink="true">https://osmand.net/blog/guideline-pt</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>Guideline for public transport</h2><div class="date">March 16, 2019</div><ul class="list"><li><a href="#Guidelines">Guidelines on how to build or correct public transport routes</a></li><li><a href="#mistakes">The most common mistakes</a></li><li><a href="#Validation_1">"OSM Inspector" final check (Data validator)</a></li><li><a href="#Validation_2">"JOSM validator" final check (Data validator)</a></li><p>Hi!</p><p>As you already know we have <a href="https://osmand.net/blog/osmand-3-3-released">recently released</a> an update of our app on Android!</p><p>And guess what? It allows our users to work out their routes using public transport, which will certainly help one to move through the concrete jungle faster. The OSM database, however, has some unchecked and dubious info, so we decided to give you a hand and show how to use the newest update to your advantage. Our instructions will be detailed, but, overall, not too complex, because we want you to engage in the process of forming OSM. Try not to break it, ok? Just kidding! If you need a bit more support use <a href="https://learnosm.org/en/josm/start-josm/">LearnOSM</a>.</p> <div class="subtitle" id="Guidelines"><h3>Guidelines on how to build or correct public transport routes</h3></div><p>So, let us go through the process of mapping out a route together using <a href="https://wiki.openstreetmap.org/wiki/JOSM">JOSM</a>. We have chosen to create a bus route. For starters, let’s mark our stops. There are two options in the settings, so one can either mark it as a regular bus stop (<a href="https://wiki.openstreetmap.org/w/index.php?title=Proposed_features/Public_Transport&oldid=625726#Stop_Position">public_transport=stop_position</a>), or as a place where the passengers will be waiting for the arrival of their bus (<a href="https://wiki.openstreetmap.org/w/index.php?title=Proposed_features/Public_Transport&oldid=625726#Platform">public_transport=platform</a>). These stops should be included directly into your route. Then, you should attach some tags, such as <i>public_transport=stop_position</i>, <i>name</i>, etc. You also can use presets: find them in Presets - Transport - Public transport.</p><img src="https://osmand.net/images/blog/guideline-pt/image_1.png"/><p>As a general rule, the platform for waiting passengers is located in front of a bus stop. It can be a node, a line or even a polygon. We will set two tags, one is <i>public_transport=platform</i>, and the other - <i>highway=bus_stop</i> (to meet the requirements of PT v1). We also encourage you to give it a name.</p><img src="https://osmand.net/images/blog/guideline-pt/image_2.png"/><p>You need to match the first stop with the starting point of your route. Logically, the last stop has to be near the finish. If, however, you have to put it in the middle, you will need to split the segment there.</p><p>After checking whether we have tagged all the stops correctly, we will go on mapping out the route. We will need to search a panel on the right side of the screen for “Relations”. If you cannot find it, go directly to Windows -> “Relations”. Found it? Click on “Create a new relation”.</p> <img src="https://osmand.net/images/blog/guideline-pt/image_3.png"/><p>A window will pop up. Here, we will create the relations. One can find the description of any tag they may need <a href="https://wiki.openstreetmap.org/w/index.php?title=Proposed_features/Public_Transport&oldid=625726#Route">here</a>.</p><p>Let’s go ahead and add some tags:</p><p><i>type=route</i></p><p><i>route=bus</i></p><p><i>from=Bus stop 1</i></p><p><i>to=Bus stop 3</i></p><p><i>name=Bus 5: Bus stop 1 => Bus stop 3</i></p><p><i>ref=5</i></p><p>Our next task is to add other elements of our route. Go to the main window, do not close the window where you have been creating the relation, though. Successively select all the segments of the route, using the CTRL button for multiple marking. These segments will appear in the tab Selection in the window where we have been creating the relation.</p><p>Using the buttons circled in red, we will add them to the “Members” tab.</p><img src="https://osmand.net/images/blog/guideline-pt/image_4.png"/><img src="https://osmand.net/images/blog/guideline-pt/image_5.png"/><p>Our next move is to add nodes <i>public_transport=stop_position</i> and <i>public_transport=platform</i>. Select them on the map and, then, add to the relation (higher than ways). Enumerate them to avoid further difficulties. For example, <i>stop_position 1, platform 1, stop position 2, platform 2</i> and etc. As a result, our relation editing window looks like this:</p><img src="https://osmand.net/images/blog/guideline-pt/image_6.png"/><p>The roles for stops and platforms, however, will be automatically filled out. For instance, in the column called “Role” stop_position has an assigned “stop” role, while platform has “platform” role. We will change it just a bit here. In particular, we will change the first and the last platform and stop_position roles. Add “_entry_only” to the objects related to the first stop (<i>stop_entry_only, platform_entry_only</i>), it means that this stop has just one purpose - entrance. The last stop, accordingly, should be marked “_exit_only” (<i>stop_exit_only, platform_exit_only</i>). Sometimes the roles are not assigned automatically, then you will need to fill them in manually.</p><img src="https://osmand.net/images/blog/guideline-pt/image_7.png"/><p>We have marked the buttons for moving and sorting the relation members as 1 and 2 (in red).</p><p><b>If the order of stops is incorrect, OsmAnd will NOT be able to plan a proper route for you!</b></p><p>To check whether the route has any gaps, use small arrows located on the right side of the way. They also will help you to determine the direction of the route.</p><p>Now, as you can see, we have a complete route, which starts at <i>Bus stop 1</i> (on Fedinskaya street) and ends at <i>Bus stop 3</i> (A-108).</p><p>As a rule, buses go both ways. So, why not to create a backtrack? Just start from copying the relations by pressing “Relations - Create copy of this relation”. You can also access the button in the redactor. In order not to get confused, we can delete all <i>stop_position</i> and <i>platform</i> from the relation copy and leave only the ways. By clicking on a button called “Reverse the order of the relation members” in the redactor window, we will reverse the direction of the route and, furthermore, will be able to check whether they are correct. Now, let us add stops and platforms, adding tags in the process.</p><p>All the directions, variants of route, and details attached to the route should be united relation type "route_master" which is required by the scheme PTv2. Our task now is to create a new relation and fill in the tags: </p><p><i>type=route_master</i></p><p><i>route_master=bus</i></p><p><i>ref=5</i></p><p>The tag “ref” is reserved for the number of the route.</p><p>Not closing the redactor, let’s find the previously created route and choose “Select relation” from the context menu. If you did alright, which we are sure of, you will see your route change color. It will become purple, and the tags will appear on the panel “Tags / Membership”.</p><img src="https://osmand.net/images/blog/guideline-pt/image_8.png"/><p>We will add it to “Members” using the corresponding buttons and save the relation.</p><img src="https://osmand.net/images/blog/guideline-pt/image_9.png"/><div class="subtitle" id="mistakes"><h3>The most common mistakes</h3></div><div class="subtitle" id="Validation_1"><h3>"OSM Inspector" final check (Data validator)</h3></div><p><a href="http://tools.geofabrik.de/osmi/?view=pubtrans_routes&lon=13.39599&lat=52.49897&zoom=12&overlays=ptv2_routes_invalid,ptv2_error_,ptv2_error_ways,ptv2_error_nodes">OSM Inspector</a> can help you to check whether routes are correct. <a href="https://blog.geofabrik.de/?p=436">Use it</a>, it does a pretty nice work!</p><p>In case there is a mistake, click on it in the tab “Selection” and detailed info will appear. A vertical scroll bar will show up automatically if there are multiple mistakes on different levels. Each mistake has a letter J next to it. Click on it, and the mechanism of <a href="https://josm.openstreetmap.de/wiki/Help/Preferences/RemoteControl">the remote control</a> will open the corresponding area in JOSM.</p><img src="https://osmand.net/images/blog/guideline-pt/image_10.png"/><p>Should the message “The OSM server ‘api.openstreetmap.org’ reported a bad request. The area you tried is too big or your request was too large. Either request a smaller area or use an export file provided by the OSM community” appear on the screen, the easiest way to resolve it would be to download this area through JOSM -> File -> Download data -> Download from Overpass API. Using <a href="https://wiki.openstreetmap.org/wiki/Overpass_API">Overpass API</a> allows you to get more data from the server in one request as well as to download only the required type of data.</p><p><i>Gap / Gap or unordered before this way</i></p><p>Any unintentional split of your track will result in this mistake. Search for the relation with this mistake in the editing panel.</p><img src="https://osmand.net/images/blog/guideline-pt/image_11.png"/><p>On this screenshot, we can see that the line which shows our route indeed has some gaps. To get to the root of the problem, click on the route segment near the split in the Members list. This way will be marked with red coloring in the main window of the program.</p><img src="https://osmand.net/images/blog/guideline-pt/image_12.png"/><p>To move across the segments of a relation you can also use the keyboard. By going through the track manually, it is easier to spot the segment that was placed incorrectly. And when we already know what has caused the mistake, we can just drag it to its right place. You can also use buttons on your left, if you wish.</p><p>The neighboring segments may have an inverted order, which is a fairly common mistake. To correct it, we can select the disorganized tracks in the list of Members and click the button  on the left, which is “Reverse the order of relation members”.</p><p>The image of a bus has been placed on the button in the latest versions of JOSM to automatically correct simple mistakes. And, finally, when we understand how to make corrections manually, we can run the program, fully controlling the process!</p><img src="https://osmand.net/images/blog/guideline-pt/image_13.png"/><p><i>Empty role for non-way object</i></p><p>In most cases, it means that we have left out one stop_position or platform role. We will need to fill in the empty role.</p> <p><i>Road vehicle route over non-road</i></p><p>This message will appear when the road is closed or does not allow a certain type of transport to pass through it. Just check the track and place some corrections.</p>  <p><i>Stop without proper tags</i></p><p>At the place of stop, there are no tags indicating the type of transport (bus=yes, train=yes and etc.).</p><p><i>Route has only stops/platforms</i></p><p>There are no ways in the route. Just add them.</p> <p><i>Trolley bus without trolley wire</i></p> <p>Tag trolley ways with trolley_wire=yes to mark contact electrical network.</p><p><i>Unknown route type</i></p> <p>This type of the route is unknown. Add some details.</p> <p><i>Unknown role</i></p> <p>The type of the role should be specified.</p><p><i>Stop is not node</i></p> <p>The place of a stop (stop_position) is not a node.</p>  <p><i>Stop not on way</i></p> <p>A stop is not implemented in your route, it should be put directly in it.</p><div class="subtitle" id="Validation_2"><h3>"JOSM validator" final check (Data validator)</h3></div><p>JOSM has an inbuilt validator which checks data. It will open automatically should you try to send the data to the server, it can also be handled manually. In order to do this, search for the tab “Validation Results” on the right of the main window.   Still cannot see it? Let’s turn it on through Windows -> Validation results. Click “Check” button and the whole area (if none selected) validation should start. Should you try sending data to the server, this check will work out only recently changed info.</p>  <img src="https://osmand.net/images/blog/guideline-pt/image_14.png"/><p>Let us go through the results related to public transport:</p> <p>- <i>Route relation contains a ‘forward/backward/alternate’ role</i></p><p>The PTv1 scheme implies additional role assignment to a route, but it goes against PTv2 schema. Delete all the roles and, if necessary, create your route again from scratch.</p>  <p>- <i>Route relation contains a gap</i></p><p>The same as <i>Gap / Gap or unordered before this way</i> in OSM Inspector.</p><p>- <i>Route scheme is unspecified. Add public_transport:version</i></p><p>If a route matches with the PTv2 scheme (as mentioned above), add the tag <i>public_transport:version=2</i> to the relation.</p> <p>- <i>Stop position is not part of the route</i></p><p>Look up <i>Stop not on way</i> in OSM Inspector, because they are identical.</p><p>____________</p> <p>As you already know, a public transport function has been added in <a href="https://osmand.net/blog/osmand-3-3-released">the newest release</a> of our app.  We have received some valuable feedback from our users who had already tried this function out. Unfortunately, not all the data in OSM is reliable, which causes a few problems. We sincerely hope that, with the help of these guidelines, it will be much easier for you to create and check the public transport routes.</p><p>P.S. We will do our best to upgrade these and other functional aspects of the app in the future. And remember that each user makes all the difference. Let’s create OSM together!</p>  ]]></description>
  			<pubDate>16 Mar 2019 17:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>OsmAnd 3.3</title>
  			<link>https://osmand.net/blog/osmand-3-3-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-3-3-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd 3.3</h2><div class="date">March 7, 2019</div><p>We will be releasing the new version patch by patch during the next two weeks, and the info about app updates will be published in this article. Here is what has been done so far.</p><p>We follow our <a href="https://osmand.net/help-online/privacy-policy">Privacy Policy</a> and now we have completely <b>removed Facebook and Firebase analytics from the free version (OsmAnd+ didn't include it)</b>.</p><h3>Navigation on public transport: Metro, Buses, Tram and so on.</h3><p>First, we would like to introduce Navigation for public transport which is currently in the beta phase. <b>Each of you can try to navigate in your city by public transport now with OsmAnd</b>. We want to receive your feedback for customization of this feature. Most importantly, we use the latest New Public Transport Schema that is also called Public Transport Version 2 (PTv2) for our Public Transport navigation algorithm. You can check your public transport <a href="http://tools.geofabrik.de/osmi/">here</a>. <a href="https://osmand.net/blog/guideline-pt">Guidelines on how to build or correct public transport routes </a>is provided in our blog.</p></ul> <div class="centeredimageblog"><img src="https://osmand.net/images/blog/3.3/pt_1_blog.png"/><img src="https://osmand.net/images/blog/3.3/pt_2_blog.png"/><img src="https://osmand.net/images/blog/3.3/pt_3_blog.png"/><img src="https://osmand.net/images/blog/3.3/pt_4_blog.png"/><div style="clear:both;"></div></div><h3>Redesigned Directions menu</h3><p>We have made a more detailed Directions menu to help you navigate to your Home and Work and see the History of your destinations. When you click “Options”, you can see Settings for Navigation like the list of active GPX and others.</p></ul><div class="centeredimageblog"><img src="https://osmand.net/images/blog/3.3/pt_5_blog.png"/><img src="https://osmand.net/images/blog/3.3/pt_6_blog.png"/><div style="clear:both;"></div></div><h3>Improve Quick action</h3><p>As requested by many users, we have <b>added Show/Hide tracks, Day/Night mode switches in Quick action</b>. Quick action is a flexible menu of shortcuts providing direct access to whatever features you need to access most frequently: just select and add them to this 'toolbar'. In this way, the next time you need to add a new favorite, add a navigation destination, enable or disable voice guidance, etc, you can do it in one click using Quick action. The Quick action button can be added via the Configure screen menu. To add actions to the menu, please press 'Add action' right on its menu bar. To remove an action, go to Configure screen and tap the 'Quick action' menu, then remove any shortcuts not needed.</p></ul><div class="centeredimageblog"><img src="https://osmand.net/images/blog/3.3/pt_7_blog.png"/><div style="clear:both;"></div></div><h3>Additional info in the Route details.</h3><p>While building a route,  you can view information about the road itself like <b>road types, surface, steepness and smoothness</b> which will help you in understanding the road conditions of your route.</p></ul><div class="centeredimageblog"><img src="https://osmand.net/images/blog/3.3/pt_8_blog.png"/><img src="https://osmand.net/images/blog/3.3/pt_9_blog.png"/><div style="clear:both;"></div></div><p>Besides these changes, we've also made a range of fixes of some critical bugs, including flooded territories and crashes at some public transport stops.</p><p>Please update OsmAnd and share your opinion on our <a href="https://www.facebook.com/osmandapp/">Facebook</a>, <a href="https://www.twitter.com/osmandapp/">Twitter</a>, and <a href="https://www.reddit.com/r/OsmAnd/">Reddit</a>!</p><p><a href="https://play.google.com/store/apps/details?id=net.osmand" target="_blank"><img src="https://raw.githubusercontent.com/osmandapp/osmandapp.github.io/9fc59e5136b6a07045eb96d052b3ce6ddde805c3/website/images/help/badge_store_google_play.png" alt="Get it on Google Play" height="60"/></a><a href="https://www.amazon.com/OsmAnd-Maps-Navigation/dp/B00D0SA8I8" target="_blank"><img src="https://raw.githubusercontent.com/osmandapp/osmandapp.github.io/9fc59e5136b6a07045eb96d052b3ce6ddde805c3/website/images/help/badge_store_amazon.png" alt="Get it on Amazon" height="60"/></a></p>]]></description>
  			<pubDate>07 Mar 2019 20:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>OsmAnd Online GPS Tracker 0.4</title>
  			<link>https://osmand.net/blog/osmand-tracker-0-4-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-tracker-0-4-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd Online GPS Tracker 0.4</h2><div class="date">Feb 19, 2019</div><p>Hi, Everybody!</p><p>Guess what - we have got something new for you! OsmAnd Tracker version 0.4 is fresh from the oven and ready to be downloaded. It does not matter whether you are an amateur or a professional sportsperson, or a traveler  - the app will pleasantly surprise you anyway.</p><p>In <a href="https://osmand.net/blog/osmand-telegram-released">our blog</a> we have already announced that we have chosen Telegram to access all the necessary data. It is an open social platform that perfectly satisfies our requirements. Not only does it provide the list of contacts, but it also allows sharing one’s location. A Telegram account will be used during the process of registration in OsmAnd Tracker. The cherry on top is that you do not have to download the Telegram app to use your account in OsmAnd Tracker, and vice versa - one does not need OsmAnd app to share their location with another user via Telegram. You do not have to worry about the safety of personal information and chats, though. Select the accessible data and do not worry. By the way, your messages will stay intact and secure.</p><p>The Tracker has an amazing new function that is Timeline. We used to have two tabs “Live now” and “My location”. Now our users can organize their tracks the way they like, sending their location with a timer. The transmission will stop the moment the set time runs out. You have time options up to 24 hours to choose from. The users you have sent the GEO position to will see either a text message with written coordinates or a dot on an actual map. And what if you want to share a road to work that you made a few days ago? Well, now - with OsmAnd - you can! Here comes “Timeline”, where you can find the exact time span and send it. That is it! As easy as it possibly could be. In this tab, you can also view the history of tracks that were either sent by you or to you on a chosen day. One also can have multiple chats with different settings and levels of integration. The application also allows you to access saved info chronologically to avoid any confusion and difficulties in finding a particular moment in your movement history, including the locations that have been shared with you by other users. Any user <a href="https://osmand.net/features/trip-recording-plugin">can master their tracks and work on the details directly in OsmAnd app</a>, which is fantastic!</p><div class="blogimageblock">    <img src="https://osmand.net/images/features/4-1-Timeline.png"/>    <img src="https://osmand.net/images/features/Timeline_2.png"/>        <div style="clear:both;"></div></div><p>Further instructions and helpful advice can be found <a href="https://osmand.net/features/tracker">here</a>.</p><p>Get OsmAnd Tracker today from <a href="https://play.google.com/store/apps/details?id=net.osmand.telegram&hl">Google play</a>.</p>.</p><p><a href="https://play.google.com/store/apps/details?id=net.osmand.telegram" target="_blank"><img src="https://raw.githubusercontent.com/osmandapp/osmandapp.github.io/9fc59e5136b6a07045eb96d052b3ce6ddde805c3/website//images/help/badge_store_google_play.png" alt="Get it on Google Play" height="60"/></a>]]></description>
  			<pubDate>19 Feb 2019 20:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>OsmAnd 2.5 (iOS)</title>
  			<link>https://osmand.net/blog/osmand-ios-2-5-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-ios-2-5-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd 2.5 (iOS)</h2><div class="date">February 12, 2019</div><p><a href="https://itunes.apple.com/us/app/osmand-maps-travel-navigate/id934850257">OsmAnd 2.5 is now available!</a></p><h3>OsmAnd Live</h3><p>In a nutshell - you can access OsmAnd Live from an IPhone now. Do you remember our <a href="https://osmand.net/blog/2019-ny-resolutions">New Year’s resolutions</a>? We have promised to catch the iOS version of OsmAnd up with the one on Android, and we are getting there. This update is just the beginning, so we will continue gradually adjusting the features of the app. OsmAnd Live gives you a unique chance to collaborate with other active app users to create a verified detailed map. Once you have inserted a correction in OpenStreetMap, it takes an hour for the online map to adjust. And, finally, there are no limits on downloading maps and contour lines.</p><h3>Subscription options</h3><p>Even though, the subscription for OsmAnd Live is inexpensive, it gives you numerous benefits, such as unlimited downloads, the access to contour lines, and hourly updates. In fact, by subscribing to the app you help to support our developers and <a href="https://osmand.net/osm_live">the contributors that edit the info on OpenStreetMap</a>. Just choose your subscription plan and you no longer need to update the map data on the device, it will be done automatically every hour or so. There are three options to choose from: a monthly, trimester, and annual. The subscribed users have all the downloads and plugins available. You can also manage the subscription and payment details using App Store or IPhone Settings.</p><div class="centeredimageblog"><img src="https://osmand.net/images/blog/ios-2-5/ios-2-5.png"/><div style="clear:both;"></div></div><p>And remember that only together we can achieve the best results!</p><p>New features are coming SOON!</p><p><a href="https://itunes.apple.com/us/app/osmand-maps-travel-navigate/id934850257"><img src="http://osmand.net/images/app-store-badge.png" alt="Get it on App Store" height="60"/></a>]]></description>
  			<pubDate>12 Feb 2019 21:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>2019 New Year resolutions</title>
  			<link>https://osmand.net/blog/2019-ny-resolutions</link>
  			<guid isPermaLink="true">https://osmand.net/blog/2019-ny-resolutions</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>2019 New Year resolutions</h2><div class="date">Dec 31, 2018</div><p>Happy New 2019 Year!</p><p>Following the nice tradition of creating the list of New Year Resolutions, on behalf of our team, I will create and share it here and hope it would be an appropriate post for the community.</p><h3>2018 achievements</h3><img src="https://osmand.net/images/blog/2018/2018-NY-1.png"/><ul class="textlist">	<li><b>Measure Distance.</b><br> This is a long-expected UI simplification and enhancement to create GPX tracks as lines and as route points using a mobile phone. It allows:		<ul>			<li>To draw lines and measure straight distances</li>			<li>To draw a route between selected points as car, pedestrian, bike even as boat on channels route. Basically it calculates a route between all intermediate points and so far we've tested up to 50-100 points.</li>			<li>To save all edits as GPX track.</li>			<li>Edit an existing GPX track.</li>			<li>Easily accessible via Drawer menu.</li>		</ul>	</li><br>	<li><b>OsmAnd Live maps.</b><br> In 2018 we've moved our implementation to Overpass and managed to achieve average delay 20 minutes after the improvements (90th percentile delay is less than 40 minutes). If we exclude all medium outages for 2-3 hours, which happens once a month, then average time between your edits and appearance on OsmAnd would be 10-15 minutes. So it gives a great tool to quickly verify changes on a spot. I've made a presentation at SOTM 2018 about it - https://youtu.be/ZHY0KE4IzWI</li><br>	<li><b>Travel - Wikivoyage.</b><br> We've launched a new travel category based on open data https://www.wikivoyage.org/ project. There were already some improvements how to open GPX files in OsmAnd and select specific categories i.e. Eat / Accommodation / See. We went further and made all that data available offline with cached images. By downloading 300 MB, you can have all travel articles around the World and enjoy your trips. Of course, Wikivoyage data is linked to Wikipedia articles about specific places, so for a complete picture, you will need to download Wikipedia as well.</li><br>	<li><b>iOS Navigation, Ruler and other.</b><br> We've finally made huge progress in our iOS version and managed to organize a small &amp; stable development group. One of the primary important features was Navigation which is a full copy of Android version including navigation on GPX and all parameters and vehicle support. This also included a variety of widgets added to iOS with an advanced version of Ruler which works in 3D as well.</li><br>	<li><b>OsmAnd Telegram.</b><br> It was a hard decision for us but we decided to go into a completely new area and make a product for active travelers OsmAnd Telegram. This is an online GPS tracker which allows you to share your location within a group of people and also see everybody on the map. Sharing your location is always linked with chat-messaging, so we decided to build on the most open social technology Telegram. We've put the OsmAnd API on a completely different level to support a complex application such as OsmAnd Telegram and we also plan to enhance that API, so more and more side-apps could enrich OsmAnd and make it easier to use for specific cases.</li><br></ul><img src="https://osmand.net/images/blog/2018/2018-NY-2.png"/><h3>2019 resolutions</h3><p>We knew our list up front and it was based on your emails and issues. We selected 5 most important directions for our small team of 9 people to make it happen.</p><ul class="textlist">	<li><b>Public Transport.</b><br> This feature is almost ready and we planned it for 2018 release though we will need to polish it and do a proper beta test. OsmAnd already has all transport lines clickable which dramatically improved in 2018, so you can view and validate the public transport lines.</li><br>	<li><b>Place Reviews!</b><br> This is probably the most important goal for 2019. We fully understand that we need to do something to fight for Open Reviews Data against Google, TripAdvisor who takes the data bound to its own products, rules and etc. It is time to make something like OpenStreetMap for Reviews and we've already found connections with other OSM app developers to build a unified platform. Obviously the data will be open for everybody and that is the only way to get enough reviews to rate the places.</li><br>	<li><b>Travel + Wikivoyage.</b><br> We've got positive feedback for integrating with Wikivoyage data but it is very obvious we need to do more. We plan to let people contribute back to Wikivoyage more actively, so the open data will be more up to date. We also need to split it by regions to make the maps smaller and generate them every month. So, there are lots of small and very important features to make it useful :)</li><br>		<li><b>iOS improvements: OsmAnd Live, Quick Action, OSM Editing.</b><br> 2018 was decisive for iOS version and we've started to work hard on features to reduce the gap between Android / iPhone version. In 2019, we are planning to release the most requested features, and the good sign is that iOS audience is growing very fast, reacting to the changes we do. OsmAnd Live, OSM Editing and Quick Action are even a shorter list what we plan to achieve in 2019.</li><br>	<li><b>OpenGL-rendering (speed up rendering).</b><br> This is a long-awaited feature and it is ready for iOS users but unfortunately, we didn't publish it for Android due to a big variety of devices on Android. It is available for public test as https://download.osmand.net/latest-night-build/OsmAnd-qt-arm-nightly.apk. Obviously, there are pros and cons: it is faster but more power consuming; it has 2.5D but not all layers are currently supported; it requires more resources for the same operations.</li><br></ul><p><b>Happy New 2019 Year!</b></p>]]></description>
  			<pubDate>31 Dec 2018 16:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>Giveaway</title>
  			<link>https://osmand.net/blog/giveaway-start</link>
  			<guid isPermaLink="true">https://osmand.net/blog/giveaway-start</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>Giveaway</h2><div class="date">Dec 29, 2018</div><p>We know that not everyone is able to purchase OsmAnd on Google Play / Amazon store or to purchase maps for OsmAnd on iTunes store due to various reasons, but many of you would still like to support the application by using it and spreading it around the world.</p><p>OsmAnd team decided to do a small present for our subscribers during this Christmas. And we started our True Giveaway based on Bitcoin. A little bit technical information: the number of promocodes is limited and each round 1 participant is selected for 1 promocode based on random number. The source of random is the latest available Bitcoin  Block hash. More information you can read and find out <a href="https://osmand.net/giveaway">on our website</a>. </p><h3>Christmas - our first giveaway</h3><p>During this Christmas weekend we started our first giveaway and we sent a special letter for our subscribers. At the end of the letter there was a special button to uniquely identify user with participant id.</p><img src="https://osmand.net/images/blog/giveaway-start/giveaway_sub_1.png"/><p>At first time we gave away <b>75</b> promocodes for OsmAnd+. At the last round there were <b>322</b> participants, so <b>25%</b> were winners.</p><img src="https://osmand.net/images/blog/giveaway-start/giveaway_win_1.png"/><p>Each hour (round) the winner was selected by latest available Bitcoin Block, so our giveaway is totally transparent and also respects privacy. </p><p>We were very surprised with your activity and engagement, so we decided to do on a regular basis. Also we plan to give away OsmAnd+, OsmAnd iOS and other paid plugins. Right now you can only subsribe for a giveaway at the Download screen in OsmAnd Android but we plan to enable subscription soon on the <a href="https://osmand.net">main page</a></p><p>Subscribe and Good luck!</p><p>We love you, our users!</p>]]></description>
  			<pubDate>29 Dec 2018 18:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>Detailed OSM changes stats</title>
  			<link>https://osmand.net/blog/osmand-live-support-changes-count</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-live-support-changes-count</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>Detailed OSM changes stats</h2><div class="date">Sep 30, 2018</div><p>This is a tremendous example of how a small change in the OpenStreetMap API could make an impact on other applications. As part of our OsmAnd Live initiative, we've collected detailed statistics about OSM changes over the last 2 years. As of today, we've gathered information about all changesets, all users and all editors. What is unique about our stats is that we are also collecting them by multiple world regions. </p><p>But we've always been asked to focus on how many objects were changed rather than how many changesets are submitted. This month that change was finally merged into the platform <a href="https://github.com/openstreetmap/openstreetmap-website/issues/1947">https://github.com/openstreetmap/openstreetmap-website/issues/1947</a>. So the rest of the implementation was pretty clear for us and it is finally in place as well.</p><h3>URLs and APIs</h3><p>You can view all stats at <a href="https://osmand.net/osm_live">https://osmand.net/osm_live</a> and browse by specific month and regions. It is also available via JSON API i.e. <a href="https://osmand.net/reports/query_report?report=all_contributions_requests&month=2018-09&region=">Worldwide</a> and <a href="https://osmand.net/reports/query_report?report=all_contributions_requests&month=2018-09&region=belarus_europe">Belarus</a> for example. </p><img src="https://osmand.net/images/blog/changes-count/stats.png"/><h3>Stats for 29 September 2018</h3><table border="1">	<tr>		<th>Changesets</th>		<th>Unique users</th>		<th>Object edits</th>		<th>Editor</th>	</tr>	<tr><td>14223</td><td>1991</td><td>810839</td><td>iD 2.1</td></tr>	<tr><td>7334</td><td>1171</td><td>1591351</td><td>JOSM</td></tr>	<tr><td>1169</td><td>247</td><td>7382</td><td>StreetComplete</td></tr>	<tr><td>818</td><td>476</td><td>1324</td><td>MAPS.ME</td></tr>	<tr><td>623</td><td>142</td><td>55098</td><td>Potlatch</td></tr>	<tr><td>305</td><td>115</td><td>3666</td><td>Vespucci</td></tr>	<tr><td>305</td><td>149</td><td>537</td><td>OsmAnd</td></tr>	<tr><td>304</td><td>99</td><td>8582</td><td>Go Map</td></tr>	<tr><td>124</td><td>10</td><td>8716</td><td>Merkaator</td></tr></table><p>Don't forget statistics lie.</p>]]></description>
  			<pubDate>30 Sep 2018 11:00:00 GMT</pubDate>
  		</item>
  		<item>
  			<title>OsmAnd Telegram &#x2014; Location Sharing</title>
  			<link>https://osmand.net/blog/osmand-telegram-released</link>
  			<guid isPermaLink="true">https://osmand.net/blog/osmand-telegram-released</guid>
  			<description><![CDATA[<head><style type="text/css">
  .blogimageblock img {
      max-width: 300px;
      padding: 10px;
      margin-top: 20px;
      margin-right: 10px;
      margin-bottom: 20px;
  }
  .blogimageblock {
      margin: 30px auto 0;
      text-align: center;
      border-radius: 6px;
      background-color: #fafafa;
      border: 1px solid #e6e6e6;
  }

  </style></head>
  <h2>OsmAnd Telegram — Location Sharing</h2><div class="date">Sep 14, 2018</div><p>We are pleased to announce that we have a new application in OsmAnd family. This application helps to see your contacts on the map in OsmAnd. It has a social aspect; you need to have contacts, that's why it was challenging for us to choose the platform to build it on.</p><p>We have chosen Telegram as the most open social platform since it has an Open API, Open SDK and, moreover, it will eventually have an Open Server implementation (Blockchain).</p><div class="blogimageblock">        <img src="https://osmand.net/images/blog/telegram/1-My_Location.jpg"/>        <img src="https://osmand.net/images/blog/telegram/4-Map.jpg"/>        <div style="clear:both;"></div></div><p>It is important that we have a great level of interoperability between the applications. You can easily exchange your location with your contacts without having 'Telegram mobile app' (OsmAnd Telegram is an independent telegram client) and, the other way around, your contacts should not necessarily have OsmAnd Telegram in order to see your location. We designed OsmAnd Telegram so you could still use your Telegram app and OsmAnd app naturally without flaws.</p><p>The application sends the Live Location messages to the selected chats for the time you specified, displays a list of your contacts and groups, and checks the chats for the presence of the messages with a location, which are then displayed on the map in OsmAnd.The application does not send or view your text messages.</p><span>Log in using your phone number registered in Telegram to enable the following:<ul>    <li>Manage the list of contacts and groups that send you their location</li>    <li>View the location of contacts and groups in real time on a map in OsmAnd</li>    <li>Set the time for sharing the location separately for each chat</li>    <li>Set the location update frequency</li></ul></span><div class="blogimageblock"><img src="https://osmand.net/images/blog/telegram/2-My-location.jpg"/><img src="https://osmand.net/images/blog/telegram/3-Live.jpg"/><div style="clear:both;"></div></div><h3>Settings:</h3><p>There are several settings for managing the frequency of sharing locations and controlling the display of the locations sent to you.</p><p><b>Send my location</b> — allows you to set the interval for sending your position from once per second (for increasing accuracy) to once in 5 minutes (for reducing power consumption).</p><p><b>Stale location</b> — allows you to set the time (from 1 minute to 1 hour) after which the location of your contact will be considered outdated. If the location becomes outdated, the position becomes gray.</p><p><b>Location history</b> — allows setting the time (from 5 minutes to 24 hours) after which the contact with the deleted location will be hidden from the list and the OsmAnd map.</p><p>The <b>OsmAnd Connect</b> setting allows you to select the version of OsmAnd in which your contacts with the position will be displayed on the map.</p><div class="blogimageblock">    <img src="https://osmand.net/images/blog/telegram/Settings-1.jpg"/>    <img src="https://osmand.net/images/blog/telegram/Settings-2.jpg"/><div style="clear:both;"></div></div><h3>OsmAnd Assistant:</h3><p>We have also prepared a Telegram bot called 'OsmAnd Assistant,' so you can integrate OsmAnd Telegram with 3rd party trackers or APIs. Hence, you can not only share your location but also the locations of your bicycle or a group of individuals taking part in racing which is available via API, and OsmAnd Telegram will make sure you have a nice presentation of locations on the map in OsmAnd itself.</p><p>OsmAnd Telegram represents the idea that we are not trying to lock-in our users into a single app, and we are in line with our mission to bring the value to end-users by focusing on the strong side of our app, meanwhile integrating with others. Recently we have also expanded a lot of our <a href="http://osmand.net/build_it">OsmAnd APIs</a>, so you can fine-tune or write another app that will seamlessly integrate with OsmAnd. Please share with us what you think about this new app on <a href="https://twitter.com/osmandapp">Twitter</a> and other social channels.</p><p>To view the GPS positions of contacts on the map, you need the latest version of <a href="https://play.google.com/store/apps/details?id=net.osmand">OsmAnd</a> or <a href="https://play.google.com/store/apps/details?id=net.osmand.plus">OsmAnd+</a>. The minimum supported version of OsmAnd or OsmAnd + is 3.0.4.</p><p>Get OsmAnd Telegram - Location Sharing <a href="https://play.google.com/store/apps/details?id=net.osmand.telegram">Google Play</a>.</p>]]></description>
  			<pubDate>14 Sep 2018 10:00:00 GMT</pubDate>
  		</item>
  	</channel>
  </rss>


key: toby_murray
encoding: utf8
format: rss 2.0
title: Toby Murray
url: http://ksmapper.blogspot.com/
feed_url: http://ksmapper.blogspot.com/feeds/posts/default
summary: My thoughts on OpenStreetMap, open source and working for the state.
generator: Blogger
updated: 2019-10-25 13:50:32
twitter: KSUToeBee
items_last_updated: 2015-02-04 00:43:00
fetched: 2019-10-25 14:30:26.938078
http_etag: A+VflO39LAw/4okaI1DHkBIO8wc
http_last_modified: Fri, 25 Oct 2019 14:30:26 GMT
http_server: GSE
md5: 2bb9632b5ca4db5fc2730d6ae80f750b
created_at: 2019-06-09 18:11:33.519956
updated_at: 2019-10-25 14:30:27.241007
body:
  <?xml version="1.0" encoding="UTF-8"?>
  <?xml-stylesheet type="text/xsl" media="screen" href="/~d/styles/rss2full.xsl"?><?xml-stylesheet type="text/css" media="screen" href="http://feeds.feedburner.com/~d/styles/itemcontent.css"?><rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:openSearch="http://a9.com/-/spec/opensearchrss/1.0/" xmlns:blogger="http://schemas.google.com/blogger/2008" xmlns:georss="http://www.georss.org/georss" xmlns:gd="http://schemas.google.com/g/2005" xmlns:thr="http://purl.org/syndication/thread/1.0" version="2.0"><channel><atom:id>tag:blogger.com,1999:blog-7044416767097884399</atom:id><lastBuildDate>Fri, 25 Oct 2019 13:50:32 +0000</lastBuildDate><category>OSM</category><category>TIGER</category><category>mapping</category><category>data integrity</category><category>imports</category><category>postgis</category><category>ODbL</category><category>osmosis</category><category>pgsnapshot</category><category>FOSS</category><category>JOSM</category><category>Linux</category><category>airport</category><category>cartography</category><category>rendering</category><category>Maperitive</category><category>QGIS</category><category>hardware</category><category>hello world</category><category>java</category><category>license</category><category>picture</category><category>tiles</category><category>update</category><title>Toby's Blog of open/mappy things</title><description>My thoughts on OpenStreetMap, open source and working for the state.</description><link>http://ksmapper.blogspot.com/</link><managingEditor>noreply@blogger.com (Unknown)</managingEditor><generator>Blogger</generator><openSearch:totalResults>25</openSearch:totalResults><openSearch:startIndex>1</openSearch:startIndex><openSearch:itemsPerPage>25</openSearch:itemsPerPage><atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="self" type="application/rss+xml" href="http://feeds.feedburner.com/ksmapper" /><feedburner:info xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0" uri="ksmapper" /><atom10:link xmlns:atom10="http://www.w3.org/2005/Atom" rel="hub" href="http://pubsubhubbub.appspot.com/" /><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-8346970140380752493</guid><pubDate>Wed, 04 Feb 2015 00:43:00 +0000</pubDate><atom:updated>2015-02-15T18:36:23.089-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">FOSS</category><category domain="http://www.blogger.com/atom/ns#">JOSM</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><title>A Note About JOSM</title><description>Over the past couple of months I have been slowly working to add support for OSM notes into JOSM. Back in July 2013, Ian Dees and I initially did a rewrite of the OpenStreetBugs plugin to support notes. We actually both started out in a friendly competition to see who could get a working version out first as we traded words on IRC. But soon Ian gave me access to his GitHub &lt;a href="https://github.com/iandees/josm-notes" target="_blank"&gt;repository&lt;/a&gt; where we continued to work on the plugin for a while.&lt;br /&gt;&lt;br /&gt;This plugin basically worked but I decided that since notes are now a core feature of OSM, they should also be more integrated into JOSM. In particular I wanted the workflow of notes to follow the standard JOSM download-work-upload sequence. The plugin was designed to be always online and issued a new query against the notes API every time you panned or zoomed the map. It had an "offline" mode which would stop this behavior and allow you to work without constant network traffic. However in this mode, changes to notes and newly created notes were not shown on the map which was a bit confusing. And unfortunately it was not a simple fix because the plugin just wasn't designed that way. In offline mode, it held actions in queue which were then executed when you went back online. So there was no note object in queue to draw on the map until after the upload happened and another download was initiated from the API. Also, the most useful way of using it was to zoom out, let it download notes and then go into offline mode to work. This was not at all intuitive.&lt;br /&gt;&lt;br /&gt;After receiving a favorable response from the &lt;a href="https://lists.openstreetmap.org/pipermail/josm-dev/2014-August/007033.html" target="_blank"&gt;josm-dev mailing list&lt;/a&gt;, I began submitting patches in September 2014. Throughout the rest of 2014 I slowly built up behind-the-scenes functionality to download, parse and display notes in JOSM. As of the December release, the notes functionality is actually usable. It is not turned on by default yet because there is still some polishing that needs to be done, especially in the user interface department. But I would love to get some more people testing it. The secret handshake to enable notes in JOSM is as follows:&lt;br /&gt;&lt;ol start="0"&gt;&lt;li&gt;Download some data. Anything, anywhere. This just initializes a setting that you will change&lt;/li&gt;&lt;li&gt;Open JOSM preferences&lt;/li&gt;&lt;li&gt;Go to the advanced preferences&lt;/li&gt;&lt;li&gt;Search for "notes"&lt;/li&gt;&lt;li&gt;Change the preference &lt;i&gt;osm.notes.enableDownload&lt;/i&gt; to "true"&lt;/li&gt;&lt;li&gt;Restart JOSM&lt;/li&gt;&lt;/ol&gt;&lt;div&gt;In graphical format:&amp;nbsp;&lt;/div&gt;&lt;div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;a href="http://4.bp.blogspot.com/-MkutEB88e4I/VM6zw1NSXcI/AAAAAAAAJAE/lgDUEiMcHzY/s1600/josm_note_screenshot.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://4.bp.blogspot.com/-MkutEB88e4I/VM6zw1NSXcI/AAAAAAAAJAE/lgDUEiMcHzY/s1600/josm_note_screenshot.png" height="440" width="625" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;When you open the download dialog after restarting JOSM, you will notice a new "Notes" checkbox alongside the "OpenStreetMap Data" and "Raw GPS data":&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;/div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-hDWQ2l5wVGU/VM7AOul50XI/AAAAAAAAJAg/fl2uHvvUWuc/s1600/josm_note_download_screenshot.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://4.bp.blogspot.com/-hDWQ2l5wVGU/VM7AOul50XI/AAAAAAAAJAg/fl2uHvvUWuc/s1600/josm_note_download_screenshot.png" height="251" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;After your first download of notes, you will also need to open the note panel or you won't be able to manipulate notes. Click the note icon in the left hand panel. If you still have the notes plugin installed, there will be two identical buttons. Either figure out which one is which or uninstall the plugin. Here is a view of things:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-ZUl2WVluywU/VM7P7nRUcvI/AAAAAAAAJAw/kujSZ99gTzc/s1600/josm_note_view.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://4.bp.blogspot.com/-ZUl2WVluywU/VM7P7nRUcvI/AAAAAAAAJAw/kujSZ99gTzc/s1600/josm_note_view.png" height="490" width="625" /&gt;&lt;/a&gt;&lt;/div&gt;The area download limit for the notes API is far greater than the limit on downloading OSM data. You can download 1 square degree at a time if you are only downloading notes. The limit for OSM data is 0.25 square degrees (although realistically you usually run into the 50,000 object limit before you run into the square degree limit). The above screenshot was downloaded in one request. It is about half the state of Kansas.&lt;br /&gt;&lt;br /&gt;Protip: By default, JOSM will only download 1,000 notes per download request. The OSM API has a hard limit of 10,000 per request. What JOSM asks for is controlled by the&amp;nbsp;&lt;i&gt;osm.notes.downloadLimit&lt;/i&gt; setting&amp;nbsp;in the advanced preferences. The other setting&amp;nbsp;worth mentioning is &lt;i&gt;osm.notes.daysClosed&lt;/i&gt;. This determines whether or not closed notes are downloaded. If set to -1, it will download all notes ever created within the requested bounding box. If it is set to 0 it will only download currently open notes. If it is set to a positive number, it will download notes which have been closed for that many days. By default, osm.org shows notes that have been closed within the last week. To copy this behavior, this setting should be set to 7. I believe the default right now is 1 day.&lt;br /&gt;&lt;br /&gt;There is some discussion around these options as well as some of the other polishing that is still needed in the&amp;nbsp;&lt;a href="https://josm.openstreetmap.de/query?status=!closed&amp;amp;component=Core+notes" target="_blank"&gt;JOSM bug tracker&lt;/a&gt;. And of course that is also where any new issues can be reported.&lt;br /&gt;&lt;br /&gt;In particular, one of the open tickets which I have already submitted a patch for, but hasn't been accepted yet is the ability to download notes via the note search API. I use this periodically to download all the notes created by Craigslist users (they all start with the word "bounds" so that's the query I send to the note search API) and close all the useless ones.&lt;br /&gt;&lt;br /&gt;One last screenshot. One of the features of the notes code is that it can read note dump files. It is actually a byproduct of being able to save and load the note layer. Anyway, here is a screenshot of the latest note dump from&amp;nbsp;&lt;a href="http://planet.osm.org/notes/"&gt;http://planet.osm.org/notes/&lt;/a&gt;&amp;nbsp;loaded into JOSM. All 309,000 of them.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-LgIndMEM91U/VM9DRt0W3GI/AAAAAAAAJBA/bDFIQRxahSM/s1600/Screenshot%2B2015-02-02%2B03.21.28.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://4.bp.blogspot.com/-LgIndMEM91U/VM9DRt0W3GI/AAAAAAAAJBA/bDFIQRxahSM/s1600/Screenshot%2B2015-02-02%2B03.21.28.png" height="360" width="625" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;Happy noting!&lt;br /&gt;&lt;br /&gt;P.S. Dear iD: &lt;a href="https://github.com/openstreetmap/iD/pull/1895" target="_blank"&gt;Your move&lt;/a&gt;!&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/xzQ71tvzErg" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2015/02/a-note-about-josm.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://4.bp.blogspot.com/-MkutEB88e4I/VM6zw1NSXcI/AAAAAAAAJAE/lgDUEiMcHzY/s72-c/josm_note_screenshot.png" height="72" width="72" /><thr:total>1</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-4415861582832030737</guid><pubDate>Wed, 05 Feb 2014 18:32:00 +0000</pubDate><atom:updated>2014-02-05T21:25:57.165-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">data integrity</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><title>Workflow For Fixing County Borders</title><description>Well, &lt;a href="http://ksmapper.blogspot.com/2014/02/county-borders-in-openstreetmap.html" target="_blank"&gt;my last post&lt;/a&gt; got some requests for workflow and I got another snow day off from work today so why not! Follow along as I fix Wyoming's county borders.&lt;br /&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Two things you will need to have open: JOSM with the mirrored_download plugin installed and the Wikipedia page for the list of counties in the state. Do a search on wikipedia for "List of counties in &amp;lt;state&amp;gt;" to find it. Here is the one for &lt;a href="http://en.wikipedia.org/wiki/List_of_counties_in_Wyoming" target="_blank"&gt;Wyoming&lt;/a&gt;.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Next you need to download all the county relations and their members. I do this using the &lt;a href="http://wiki.openstreetmap.org/wiki/Overpass_API" target="_blank"&gt;Overpass API&lt;/a&gt;. JOSM has a "Download from Overpass API" option in the File menu (Update: turns out this is a feature of the mirrored_download plugin, not in JOSM core) but we aren't going to use that one just yet because it always adds a bbox parameter to the query which we don't want in this one. Instead, use the "Open Location" option and paste in the following URL after changing the 56 to the appropriate FIPS code for your state which should be listed on the Wikipedia page.&lt;/div&gt;&lt;pre class="source-code"&gt;http://overpass-api.de/api/interpreter?data=relation["nist:state_fips"="56"];&amp;gt;&amp;gt;;out meta;&lt;br /&gt;&lt;/pre&gt;&lt;div&gt;&lt;br /&gt;This tells the Overpass API to return all relations with a nist:state_fips=56 tag plus all of the relation members (&amp;gt;&amp;gt;) and finally to add OSM metadata that is needed to edit and upload (out meta). This includes things like version number, last edit timestamp and last user to touch the object. By default it excludes this metadata to reduce download size.&lt;br /&gt;&lt;br /&gt;After that finishes you should have a nice outline of the state and all its counties. Make sure the number of downloaded relations matches the number of counties in the state. If not, it probably means the nist:state_fips tag is missing from some counties. If you can mange to find the relation through another means, download it by ID using the "Download object" option in JOSM. Or... I don't know... Ping me on &lt;a href="http://irc.openstreetmap.org/" target="_blank"&gt;IRC&lt;/a&gt; I guess :)&lt;br /&gt;&lt;br /&gt;Next, let's load up all the place nodes in the state. For this I use the "Download from Overpass API" option in JOSM. Start off by drawing a box around the desired state on the map. So far I have only done mostly square states. If your state is an odd shape then you can download the bounding box and then select things outside of your state and use the JOSM "Purge" option in the edit menu to remove them from your dataset. Or just ignore them. Then paste this in this text into the query box:&lt;/div&gt;&lt;pre class="source-code"&gt;[timeout:300];node["place"];out meta;&lt;br /&gt;&lt;/pre&gt;&lt;div&gt;Now the screen will look a little busy so I set up a filter to only show me place=county nodes until I need the other stuff. To do this, create a new filter with the filter string&lt;/div&gt;&lt;pre class="source-code"&gt;place=county or type:way or type:relation&lt;br /&gt;&lt;/pre&gt;&lt;div&gt;This will initially hide the county borders and nodes that you want to see so after the filter is created, make sure all three checkboxes in the filter panel are checked. The far right one will invert the filter so instead of hiding whatever matches the filter string it will hide everything &lt;b&gt;except&lt;/b&gt; the things that match the filter string. The second checkbox on the left makes them completely disappear instead of just making them inactive but still visible. The first checkbox can be used to quickly disable the filter when you need to see everything.&lt;br /&gt;&lt;br /&gt;At this point my JOSM window looks like this:&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-ctczbpgXw5E/UvJnp6LxJoI/AAAAAAAAFQw/3C4C-qnFlEw/s1600/WY_counties.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://2.bp.blogspot.com/-ctczbpgXw5E/UvJnp6LxJoI/AAAAAAAAFQw/3C4C-qnFlEw/s1600/WY_counties.png" height="384" width="632" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Note the "Relations: 23" on the relation toolbox matches the expected number of counties in Wyoming.&lt;br /&gt;&lt;br /&gt;At this point I usually remove the is_in tag from all the place=county nodes just to get that out of the way. Do a search (CTRL-F) for "type:node place=county" and it should select all of the county nodes. If you have some for a neighboring state loaded that you don't want to touch you can deselect them by holding CTRL while clicking. Then just delete the is_in tag.&lt;br /&gt;&lt;br /&gt;Now for the real meat. I just go down the list of counties in Wikipedia. Bring up the first county's Wikipedia page and copy the page title for use in the wikipedia=* tag and note the name of the county seat.&lt;br /&gt;&lt;br /&gt;Now in JOSM hit CTRL-F to bring up the search dialog and search for the county seat. If you are lucky it will find exactly one match. Note that because of the filter, you won't actually see the selected object but it is still selected! This could be considered a bug but in this case it works out. If the search found multiple things then you will have to disable the filter and find the right one.&lt;br /&gt;&lt;br /&gt;Now:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Hold down shift and select the county node in addition to the city node that is already selected&lt;/li&gt;&lt;li&gt;Select the county relation in the relation list and click the "Edit" button&lt;/li&gt;&lt;li&gt;Add the two selected nodes to the bottom of the relation member list using the 4th button down on the lower right panel of the window&lt;/li&gt;&lt;li&gt;Add the role "admin_centre" to the city node&lt;/li&gt;&lt;li&gt;Add the role "label" to the county node&lt;/li&gt;&lt;li&gt;Add the alt_name tag to the relation in the top panel&lt;/li&gt;&lt;li&gt;Add the wikipedia tag. Remember how you copied the page title from Wikipedia? Just type "en:" and then paste the title.&lt;/li&gt;&lt;/ul&gt;Here is a shot of Albany county (Wyoming, not New York!) when I'm done with it:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-mDtYRRPR9rg/UvJzaIDPNbI/AAAAAAAAFRA/joJTFCWHazQ/s1600/Relation_editor.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://2.bp.blogspot.com/-mDtYRRPR9rg/UvJzaIDPNbI/AAAAAAAAFRA/joJTFCWHazQ/s1600/Relation_editor.png" height="615" width="632" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;And that's pretty much it. Now just repeat the process through the list of counties.&lt;br /&gt;&lt;br /&gt;After I'm done I go back and look for any place=hamlet or place=village county seats and bump them up to place=town. I do this by doing CTRL-A in the relation list, then right clicking and doing a "Select members" which selects all the relation members. Then I use the find dialog to find all place=village and make sure to select the "find in selection" radio button so you are only finding things that are already selected. Otherwise you will promote ALL villages in the whole state to towns! As a sanity check, use the selection toolbox to make sure the number of objects selected matches your expectation. It should be fewer than the number of counties in the state.&lt;/div&gt;&lt;br /&gt;&lt;div&gt;I will leave you with a quick video of me editing the next county in Wyoming.&lt;/div&gt;&lt;iframe allowfullscreen="" frameborder="0" height="375" src="//www.youtube.com/embed/8CAt9Uec5R4" width="500"&gt;&lt;/iframe&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/jNgij8BF9EE" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2014/02/workflow-for-fixing-county-borders.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://2.bp.blogspot.com/-ctczbpgXw5E/UvJnp6LxJoI/AAAAAAAAFQw/3C4C-qnFlEw/s72-c/WY_counties.png" height="72" width="72" /><thr:total>1</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-8114342771946681699</guid><pubDate>Wed, 05 Feb 2014 07:21:00 +0000</pubDate><atom:updated>2014-02-05T12:36:53.790-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">data integrity</category><category domain="http://www.blogger.com/atom/ns#">imports</category><category domain="http://www.blogger.com/atom/ns#">mapping</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><title>County Borders in OpenStreetMap</title><description>I have had a fairly steady relationship with county borders in OSM. They were originally imported before I joined the project in 2010. However they were imported as overlapping closed ways which is not ideal. In some places people had started working on de-duplicating the ways and turning them into relations. However this is tedious work and some of the people who attempted it didn't quite understand how relations worked so it was pretty much a big mess.&lt;br /&gt;&lt;br /&gt;Throughout 2011 and 2012 I periodically came back and worked on county borders and eventually got to a place where there was a relation for every county in the nation. There are still some minor differences in tagging but I wasn't as concerned with that and tried not to change the work of other mappers as long as the relation existed and had valid geometry. At the same time I also reduced complexity at state borders. There used to be a mess of admin boundary ways. One for the state, one for each of the counties on either side of the state border and possibly also some city boundaries. They usually didn't match up very well either so it looked terrible. I didn't mess with cities but at least I made the state and county borders share a single way to reduce clutter.&lt;br /&gt;&lt;br /&gt;At some point I also noticed (thanks to the MapQuest Open rendering which renders county names) that the place=county nodes were all placed at the extreme eastern border of all the counties. This caused the county names to be rendered in unexpected locations. So I ended up fixing those as well. Here is a screenshot of the last changeset where I moved over 1,100 nodes to the middle(ish) of their respective counties:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-cSoG666Bk9s/UvE9ZqZ6wOI/AAAAAAAAFOI/ts6ch7_tfEU/s1600/county_nodes_move.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://2.bp.blogspot.com/-cSoG666Bk9s/UvE9ZqZ6wOI/AAAAAAAAFOI/ts6ch7_tfEU/s1600/county_nodes_move.png" height="425" width="632" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;&lt;h4&gt;The Effects&lt;/h4&gt;So that was nice. But there was still some weirdness with counties, particularly when geocoding in Nominatim. Especially if there was a city with the same name as a county. For example, I searched for "Kearney, NE" and the first result was the place=county node for Kearney County in Nebraska. What I was looking for and expected to find first was the city of Kearney which is in Buffalo county. Then there is the issue of Nominatim finding both the county node and the boundary relation when you search for a county. The same thing often happens with cities. Really, they are the same thing and should show up as one result. And sometimes it might. Nominatim tries to be smart about this and does remarkably well but sometimes it needs help.&lt;br /&gt;&lt;br /&gt;You might say we should just delete the node since we already have the boundary information. But very few renderers use boundary relations to render place names from. They all rely on these nodes. The same goes for cities, states and even countries I believe. There is also the oddness of what to use for county name. The relations all have a "name=Kearney County" tag while the nodes just have "name=Kearney".&lt;br /&gt;&lt;br /&gt;After poking around a little and bugging&amp;nbsp;&lt;a href="http://wiki.openstreetmap.org/wiki/User:Lonvia"&gt;lonvia&lt;/a&gt;&amp;nbsp;(maintainer of nominatim.osm.org) about it a couple of times, I learned a few things about Nominatim and was able to make some fixes. It was my intent to come back to this some day and figure it all out and fix up county relations once and for all. I still hadn't gotten around to that but just in the last few days, new user &lt;a href="http://www.openstreetmap.org/user/revent"&gt;revent&lt;/a&gt;&amp;nbsp;was getting annoyed at the geocoding problems in Texas and really dove into Nominatim to figure out the best practices. He wrote up a &lt;a href="http://www.openstreetmap.org/user/revent/diary/20934" target="_blank"&gt;diary entry&lt;/a&gt; on the subject and we have since then talked together some more on IRC and since I had work cancelled today because of snow I thought I would write up this post about my current understanding of things:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Nominatim is able to "link" multiple elements into a single feature. It tries to do this automatically but doesn't always get it right.&lt;/li&gt;&lt;li&gt;Nominatim uses Wikipedia to help determine the importance of a place.&lt;/li&gt;&lt;li&gt;Nominatim makes use of the alt_name tag.&lt;/li&gt;&lt;li&gt;After editing the map, Nominatim does not always fully reindex an area. It might depend on current workload of the server so things can seem inconsistent. But a manual reindex can be forced by an admin after a large area has undergone a lot of edits.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;The linking is important because this allows the county border and the node to be linked into a single feature so that only one result is returned when you search for a particular county. It also improves addressing since Nominatim no longer has to choose between two different things to associate addresses with. This linking can be accomplished by adding the node to the county border relation with a role of "label".&lt;br /&gt;&lt;br /&gt;The use of Wikipedia is interesting and (I think) the source of my problems with Kearney, NE that I mentioned above. Nominatim was linking the county node to the &lt;a href="http://en.wikipedia.org/wiki/Kearney,_Nebraska"&gt;Kearney, Nebraska&lt;/a&gt;&amp;nbsp;Wikipedia page based on name tag. This helped bump its importance up to where it was returned before the city. Now that I have added the Kearney county node to the Kearney county relation and specified a "wikipedia=en:Kearney County, Nebraska" tag, it links things to the correct Wikipedia page and the search results return the city first, as expected.&lt;br /&gt;&lt;h4&gt;&lt;br /&gt;&lt;/h4&gt;&lt;h4&gt;The Fix&lt;/h4&gt;So to summarize, I believe revent and I have settled on the ideal way to map a county. Using Kearney county as an example:&lt;br /&gt;One relation with the following tags:&lt;br /&gt;&lt;pre class="source-code"&gt;type=boundary&lt;br /&gt;boundary=administrative&lt;br /&gt;admin_level=6&lt;br /&gt;name=Kearney County&lt;br /&gt;alt_name=Kearney&lt;br /&gt;wikipedia=en:Kearney County, Nebraska&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;In addition, the following tags are in widespread use:&lt;br /&gt;&lt;pre class="source-code"&gt;border_type=county&lt;br /&gt;nist:state_fips=31&lt;br /&gt;nist:fips_code=31099&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;I'm not sure the border_type tag is actually used for much but it has a fair amount of existing use and seems to further differentiate borders that are the same admin_level based on local conventions - county vs borough and such. The two nist: tags were part of the import and I have actually used the state_fips as an easy way to download all county relations in a given state to work on. These FIPS codes have been withdrawn by NIST recently but are still commonly used. We may need to look at replacing or augmenting these tags with an ISO standard at some point but for now they work.&lt;br /&gt;&lt;br /&gt;In addition to the relation tags the following relation memberships are recommended:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Obviously the ways that make up the border with a role of "outer" in the correct order so that they form a closed ring&lt;/li&gt;&lt;li&gt;The node with the following two tags: place=county and name=Kearney with a role of "label"&lt;/li&gt;&lt;li&gt;The place=city/town node of the county seat with a role of "admin_centre"&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;The county seat city node as an admin centre (yes, use the British spelling) is kind of a new thing to me. I'm not 100% sold on it but it does seem like useful information to have.&lt;br /&gt;&lt;br /&gt;If you have looked at very many county nodes you might notice that the "is_in" tag is conspicuously absent. It turns out, it causes more problems than it solves and should probably be left off or removed. I have always found it a bit silly anyway. OSM is a geodatabase - there is no need to indicate location in a tag.&lt;br /&gt;&lt;br /&gt;I have also been promoting any county seats that were place=village to place=town. I know I saw this as a thing on the OSM wiki at one point although I can't find it right now. While they may not meet the population guidelines of a "town" according to the wiki, I think county seats carry extra importance due to their governmental functions. It also means that at least some city names are rendered on maps at low zoom levels in low population areas. Without this promotion you could pan around in the Kansas/Nebraska/Dakotas area for several screens at zoom level 9 or 10 and not see a single city name on the map on osm.org. In some ways this is a rendering issue but I don't feel like this is an egregious case of tagging for the renderer.&lt;br /&gt;&lt;br /&gt;&lt;h4&gt;Your Turn!&lt;/h4&gt;So if you are looking for something to do, jump in and make sure your county is correctly linked and tagged. I have already done Kansas, Nebraska and South Dakota but I probably won't have time to do everything in the near future and I will concentrate on lower population areas first so please feel free to jump in! If there is interest, I may do another post about my workflow to do a state-wide edit. Otherwise you can always find me on the #osm or #osm-us &lt;a href="http://irc.openstreetmap.org/" target="_blank"&gt;IRC channels&lt;/a&gt; if you have questions.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;UPDATE!&lt;/b&gt;&amp;nbsp;I have since written another post detailing my workflow: &lt;a href="http://ksmapper.blogspot.com/2014/02/workflow-for-fixing-county-borders.html" target="_blank"&gt;http://ksmapper.blogspot.com/2014/02/workflow-for-fixing-county-borders.html&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;Incidentally, a lot of the same concepts apply to cities as well. If you do a Nominatim search for "&lt;a href="http://nominatim.openstreetmap.org/search.php?q=Los+Angeles%2C+CA" target="_blank"&gt;Los Angeles, CA&lt;/a&gt;" right now it will return two results. One for the node and one for the boundary relation. Making the node a member of the relation should fix that.&lt;br /&gt;&lt;span style="font-size: x-small;"&gt;(sigh. In doing that search I just saw that the relation for LA is completely broken anyway. Admin boundaries suck.)&lt;/span&gt;&lt;br /&gt;&lt;span style="font-size: x-small;"&gt;&lt;br /&gt;&lt;/span&gt; In closing, here is a screen shot of the completed county relations in South Dakota:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-2ILEFfM1e_8/UvHkmYVvEJI/AAAAAAAAFO4/yrarFtR5VJo/s1600/SD_counties.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://1.bp.blogspot.com/-2ILEFfM1e_8/UvHkmYVvEJI/AAAAAAAAFO4/yrarFtR5VJo/s1600/SD_counties.png" height="425" width="632" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/WgdEEIS7cC0" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2014/02/county-borders-in-openstreetmap.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://2.bp.blogspot.com/-cSoG666Bk9s/UvE9ZqZ6wOI/AAAAAAAAFOI/ts6ch7_tfEU/s72-c/county_nodes_move.png" height="72" width="72" /><thr:total>1</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-6006119363182568753</guid><pubDate>Wed, 23 Jan 2013 07:13:00 +0000</pubDate><atom:updated>2013-01-23T01:13:36.340-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">imports</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">pgsnapshot</category><category domain="http://www.blogger.com/atom/ns#">postgis</category><category domain="http://www.blogger.com/atom/ns#">QGIS</category><title>Too Much of a Good Thing</title><description>What makes OpenStreetMap great? Well there may be multiple answers from different people. But one thing is certain: Our geodata is awesome. I mean it is kind of the reason that we exist. So more is better, right? We are, after all, trying to map the entire planet. &lt;br /&gt;&lt;br /&gt;But it turns out, there can actually be too much of a good thing. One of my first encounters with such a situation was while doing some fixing up of imported TIGER data somewhere out west. I came across this:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-8wUDDLz28FM/UO5PohggKAI/AAAAAAAABkA/FccY7q8W7SI/s1600/TIGER_exponential_nodes.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="448" src="http://1.bp.blogspot.com/-8wUDDLz28FM/UO5PohggKAI/AAAAAAAABkA/FccY7q8W7SI/s640/TIGER_exponential_nodes.png" width="635" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Notice the scale. We are looking at a little over 100 meters of country highway and there are over 300 nodes in this image. This is obviously beyond ridiculous and I can assure you that this particular instance was cleaned up long ago and those nodes met their appropriate end in a special level of /dev/null.&lt;br /&gt;&lt;br /&gt;But this is not an isolated incident. I keep coming across such over-noding. So I decided to use my local planet database (which I &lt;a href="http://ksmapper.blogspot.com/2011/04/parse-planet-into-pgsnapshot-database.html" target="_blank"&gt;previously blogged&lt;/a&gt; about setting up) to try and find out how much this really happens and where such things come from.&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;Well, the "ways" table in the database is 85 GB in size and has over 165 million entries in it. Doing anything with the whole data set tends to be slow. So I decided to filter out some things. While discussing it on IRC, Andrew Buck suggested starting by ignoring ways with fewer than 10 nodes in them. After some consideration this actually seemed reasonable. It is hard to over-node something with 10 nodes. So I made a new table with only these ways in it and with only a subset of columns to make it smaller and more manageable. This actually ended up eliminating a lot more ways than I anticipated. It turns out, 75% of the ways in the OSM database have fewer than 10 nodes in them. This left me with about 40.5 million ways to investigate.&lt;br /&gt;&lt;br /&gt;The next question is what does it mean to be "over-noded" anyway? Since this is a geospatial database, I can use functions to determine the length of a given way. And I also know how many nodes each way has in it. So a logical unit is nodes per meter. &lt;br /&gt;&lt;br /&gt;Next up is to determine what is "normal" for typical mapping. Taking the 40 million ways with more than 10 nodes, I ran some basic statistical functions and found the following numbers:&lt;br /&gt;Minimum nodes/meter: 0.00000282&lt;br /&gt;Median nodes/meter: 0.043&lt;br /&gt;Average nodes/meter: 0.088&lt;br /&gt;Maximum nodes/meter: 65.23&lt;br /&gt;&lt;br /&gt;Well that is quite a range. Let's see what is going on here. The way with the fewest nodes/meter turns out to be &lt;a href="http://www.openstreetmap.org/browse/way/105818922" target="_blank"&gt;way 105818922&lt;/a&gt; which is part of the border of Indonesia. It has 14 nodes and stretches almost 5,000 kilometers, averaging one node every 350 kilometers. It could maybe use a little more refinement but it's a national border in the middle of the ocean... it is probably not too bad.&lt;br /&gt;&lt;br /&gt;On the other end of the spectrum is &lt;a href="http://www.openstreetmap.org/browse/way/44342937" target="_blank"&gt;way 44342937&lt;/a&gt;. It supposedly represents a round "building" with a diameter of about 1.5 meters. It has 339 nodes. What the... &lt;b&gt;WHAT?!&lt;/b&gt; What is this, the world's most accurately mapped dog house? Since it is likely that this way is going to be deleted in the very near future, I include here a screen shot of it loaded up in JOSM:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-NtISA2J1tcE/UO5o7s4CLpI/AAAAAAAABkQ/h7HTXMOBfw8/s1600/most_overnoded.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="327" src="http://1.bp.blogspot.com/-NtISA2J1tcE/UO5o7s4CLpI/AAAAAAAABkQ/h7HTXMOBfw8/s400/most_overnoded.png" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Yes, that is a tiny circle in the middle of someone's back yard. All that red around it is JOSM trying to draw its normal way direction arrows on such a ridiculous object.&lt;br /&gt;&lt;br /&gt;Now that we have examined both ends of the spectrum, what's in the middle? Well, there is &lt;a href="http://www.openstreetmap.org/browse/way/101521179" target="_blank"&gt;way 101521179&lt;/a&gt;. 174 meters long with 14 nodes which yields a nodes/meter reading of 0.08000019. That is... well... entirely reasonable. The fact that the median is half of the average would seem to indicate that the numbers are skewed towards the high end by a comparatively few number of ways with a very high number of nodes - like that one with 65 per meter.&lt;br /&gt;&lt;br /&gt;So if 65 nodes/meter is the extreme upper limit and most people map at less than 0.1 then what exactly is "too much"? Well I don't really know but I'm just going to throw out a number for further analysis. Let's make it... 1 node per meter. This limits the number of ways down to about 105 thousand.&lt;br /&gt;&lt;br /&gt;Since my database has geographic knowledge about these ways and I have the number down to a reasonable level, let's hook up&amp;nbsp;&lt;a href="http://www.qgis.org/" target="_blank"&gt;QGIS&lt;/a&gt;&amp;nbsp;to it and make a map of the ways with more than 1 node per meter. Let's start with the US:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-fJlA8bkjm2w/UPrnIj5SzkI/AAAAAAAABpI/ygwGpcA-JLE/s1600/over_noded_US.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="448" src="http://2.bp.blogspot.com/-fJlA8bkjm2w/UPrnIj5SzkI/AAAAAAAABpI/ygwGpcA-JLE/s640/over_noded_US.png" width="635" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Well that's actually not as dense as I was expecting. It looks like some of the TIGER over-noding might actually come in at under 1 node/meter. Which kind of makes sense if you think about it. A lot of TIGER ways are really long and the over-noding is sometimes clumped into one section so on average the nodes/meter ratio will be less than 1. But for fun let's look at what other interesting things might be in this data set. How about Europe?&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-Ei22FD6v07I/UProXdkS_pI/AAAAAAAABpY/AaI4ONjTwbE/s1600/over_noded_EU.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="449" src="http://3.bp.blogspot.com/-Ei22FD6v07I/UProXdkS_pI/AAAAAAAABpY/AaI4ONjTwbE/s640/over_noded_EU.png" width="635" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;Well now that's different. What's with the outbreak in France?! I have my suspicions. Let's see how this works out...&lt;br /&gt;&lt;br /&gt;There is a "source" tag that is often used in OSM data to indicate where a certain map object came from. It is especially often used during imports. Technically it usually makes more sense to put the source tag on the changeset instead of the map object itself, especially for large imports but a lot of people still put it on the map objects anyway. So let's see what these over-noded ways reveal in their source tags. Here are the results of a query that groups things by source tag and counts how many ways with a node/meter ratio of over 1 have that source tag. I truncated the source information at 70 characters for display purposes.&lt;br /&gt;&lt;br /&gt;&lt;pre class="source-code"&gt;&amp;nbsp;WAYS  | SOURCE&lt;br /&gt;-------|-----------------------------------------------------------------------&lt;br /&gt; 62995 | cadastre-dgi-fr source : Direction Générale des Impôts - Cadastre. Mis&lt;br /&gt; 22713 | &lt;br /&gt;  3535 | extraction vectorielle v1 cadastre-dgi-fr source : Direction Générale &lt;br /&gt;  1313 | Bing&lt;br /&gt;  1045 | 3dShapes&lt;br /&gt;   744 | bing&lt;br /&gt;   687 | Kolding Kommune&lt;br /&gt;   628 | NHD&lt;br /&gt;   608 | dcgis&lt;br /&gt;   602 | WakeGIS&lt;br /&gt;   585 | WroclawGIS&lt;br /&gt;   526 | Planimetria de Vitoria&lt;br /&gt;   482 | MassGIS Buildings (http://www.mass.gov/mgis/lidarbuildingfp2d.htm)&lt;br /&gt;   454 | http://www.bakersfieldcity.us/gis/downloads/gis_spatial_data.htm&lt;br /&gt;   393 | NextView&lt;br /&gt;   390 | Regione Emilia Romagna&lt;br /&gt;   369 | cadastre-dgi-fr source : Direction Générale des Impôts - Cadastre ; mi&lt;br /&gt;   349 | kapor2&lt;br /&gt;   336 | Bing Sat&lt;br /&gt;   324 | SO!GIS Import&lt;br /&gt;   260 | geoimage.at&lt;br /&gt;   244 | Regione_del_Veneto_LR28_16.7.1976_Formazione_CTR_auth_39164-5700-1100_&lt;br /&gt;   230 | http://www.sogis.ch SO!GIS Import&lt;br /&gt;   225 | MGC&lt;br /&gt;   214 | Kreis_Viersen_Katasteramt_2012_06&lt;br /&gt;   214 | lukr&lt;br /&gt;   206 | Ajuntament de Girona&lt;br /&gt;   184 | CCH&lt;br /&gt;   183 | NRCan-CanVec-10.0&lt;br /&gt;   175 | Orthophotos 2011 du SITG (Système d'Information du Territoire Genevois&lt;br /&gt;   133 | DEP Wetlands (1:12,000) - April 2007 (http://www.mass.gov/mgis/wetdep.&lt;br /&gt;   128 | vuv:dibavod:a05&lt;br /&gt;   125 | http://www.roseville.ca.us/services/maps_n_data/data_clearinghouse.asp&lt;br /&gt;   115 | City of Kamloops&lt;br /&gt;   112 | cadastre-dgi-fr source : PaysDeBrest - 20100331&lt;br /&gt;   105 | OS_OpenData_VectorMapDistrict&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Well then. I recognize that top source tag as the ongoing import of french building outlines and some other features from some cadastre data they got their hands on over there. A spot check shows that some of this isn't actually because of excessive nodes being used but rather the odd way in which the import is creating building geometries. For example, here is&amp;nbsp;&lt;a href="http://www.openstreetmap.org/browse/way/67157454" target="_blank"&gt;way&amp;nbsp;67157454&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-otQ5Y7wm0IA/UPx2hj3kfRI/AAAAAAAABuc/1hmKTcG0Qcs/s1600/overnoded_FR.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="429" src="http://3.bp.blogspot.com/-otQ5Y7wm0IA/UPx2hj3kfRI/AAAAAAAABuc/1hmKTcG0Qcs/s640/overnoded_FR.png" width="635" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;Note the wall=no tag. I believe this implies that it is some kind of porch or veranda. It has a rounded front which is where all the nodes are that make it have a nodes/meter ratio of just over 1.0. If a human mapper had mapped this building it would have likely been a single way that included the porch as well as the rest of the house instead of the 8 individual areas that the import created. This would have resulted in a single 70 meter long way with 47 nodes which comes out to 0.67 nodes/meter.&lt;br /&gt;&lt;br /&gt;Checking a few more of these ways in France, it does look like some care was taken to prevent over-noding. Most of them are just barely over 1.0 nodes/meter. And at this point it should also be noted that closed ways are actually having their first/last node counted twice because of the way the database stores the node membership information. Since I was originally looking for over-noded highways I didn't take this into account. So technically a lot of these French buildings might be just under 1 node/meter. But the fact that there are so many of them right on this (arbitrary) limit is still interesting.&lt;br /&gt;&lt;br /&gt;There are a few other source tags I recognize as well. CanVec is imported from Canadian government data. NHD is the National Hydrography Dataset here in the US from which some people have imported rivers and lakes. I know several of these water feature imports have had problems with ridiculous over-noding. Some of it has been fixed but a lot remains. MassGIS is the&amp;nbsp;Massachusetts&amp;nbsp;GIS office which has been used for some local imports. In particular, the most over-noded way that I showcased above is from this MassGIS building import.&lt;br /&gt;&lt;br /&gt;Of course the second most popular source tag is blank which doesn't tell us much. A tiny random sample shows there are a couple of imports that didn't use any source tag and some are just very detailed manual mapping.&lt;br /&gt;&lt;br /&gt;I think finding what I was originally looking for (over-noded TIGER ways) is going to take some more digging. This post kind of got hijacked by the big red blob in France but I think I'll call it quits for now and do some more poking to find the TIGERs I'm after. If there is a lesson so far, I think it can be summed up as:&lt;br /&gt;&lt;br /&gt;&lt;ol&gt;&lt;li&gt;&lt;b&gt;Sanity check your imports!&lt;/b&gt; Tiny objects with a large number of nodes are an obvious sign that there is something weird going on in your conversion to .osm format. It is possible that the object was represented as a parametric curve in the source and the conversion to OSM format tried to recreate that as closely as possible within the constraints of our x/y coordinate system.&lt;/li&gt;&lt;li&gt;&lt;b&gt;Imports are not human data.&lt;/b&gt; Even in France where this building import is generally viewed as a good thing and a lot of checking and verifying of the data has taken place, the data is still very distinctly different from most of the OSM database that has been created by hand. This may not be a bad thing in every case but it is definitely something to think about when proposing and executing an import.&lt;/li&gt;&lt;/ol&gt;Lastly, I leave you with one more way. As I was randomly browsing the 105 thousand ways, I happened to click on way number&amp;nbsp;&lt;a href="http://www.openstreetmap.org/browse/way/129485933" target="_blank"&gt;129485933&lt;/a&gt;&amp;nbsp;which has 1.00526 nodes per meter. (again, with the first/ last node double counted) If you look back at version 1 and 2 you will see a certain someone's user name. Turns out I am the culprit! This way was originally created on my phone during the &lt;a href="http://2011.stateofthemap.org/friday-night-fun-rockies-game/" target="_blank"&gt;baseball game&lt;/a&gt; we attended during SOTM 2011 in Denver. How random is that?&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/1bUQg-G41gU" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2013/01/too-much-of-good-thing.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://1.bp.blogspot.com/-8wUDDLz28FM/UO5PohggKAI/AAAAAAAABkA/FccY7q8W7SI/s72-c/TIGER_exponential_nodes.png" height="72" width="72" /><thr:total>7</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-4177457460665795371</guid><pubDate>Wed, 21 Nov 2012 17:10:00 +0000</pubDate><atom:updated>2012-11-21T11:18:12.844-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">imports</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">rendering</category><category domain="http://www.blogger.com/atom/ns#">TIGER</category><title>Preparing for Operation Cowboy</title><description>If you haven't heard, there's a major mapping event about to happen. Dubbed &lt;a href="http://wiki.openstreetmap.org/wiki/Operation_Cowboy" target="_blank"&gt;Operation Cowboy&lt;/a&gt;, this will be the second event of its kind, the first being &lt;a href="http://wiki.openstreetmap.org/wiki/Night_of_the_living_maps" target="_blank"&gt;Night of the Living Maps&lt;/a&gt; which happened back in February. As described on the wiki, these events are kind of like a global LAN party for maps. For one weekend people who choose to participate will focus their mapping efforts on a common goal. This time the organizers decided to focus on the USA. &lt;br /&gt;&lt;br /&gt;We of course have plenty of mapping to do around here. Cleaning up and updating the TIGER road import is kind of a never-ending task. In some of the sparsely populated areas of the country there aren't really any local mappers to speak of so there is ample work for "armchair mappers" to come in and do some map maintenance.&lt;br /&gt;&lt;br /&gt;Considering the audience and the event, I decided to try and get a few things tweaked before Operation Cowboy goes down.&lt;br /&gt;&lt;br /&gt;&lt;h3&gt;TIGER 2012 road name tiles&lt;/h3&gt;&lt;br /&gt;First up was improving the &lt;a href="http://wiki.openstreetmap.org/wiki/TIGER_2012" target="_blank"&gt;TIGER 2012 road name tiles&lt;/a&gt; being served up by the OSM-US tile server. They were originally set up by Ian Dees and are a great resource for tracing roads that were built since the original TIGER data was imported. The only issue I had with them is that they show the street name in the Census Bureau's abbreviated format. In OSM we prefer to use unabbreviated names because it reduces ambiguity. Does "St" mean "Street" or "Saint"? While that one might not be too hard to figure out and expanding "Ave" to "Avenue" might be trivial, there are a lot of abbreviations in the data and some of them are not easy to guess if you aren't familiar with the area.&lt;br /&gt;&lt;br /&gt;Any guesses as to what the following mean? Brg. Cll. Natl Lkshr. Ofc. Pso. RM. BIA Rte? &lt;span style="font-size: xx-small;"&gt;(see answer key at end of post)&lt;/span&gt; Most people in the US can probably guess some of them but there are definitely some regional ones in there that many people may not be familiar with. Then add in abbreviated directional prefixes. And now suppose you are a European trying to figure all of this out during Operation Cowboy. It's going to look like a secret code.&lt;br /&gt;&lt;br /&gt;I won't go into the technical details in this post (might save that for another one after I clean up the process doumentation) but it did involve 27 GB of data in 3,200 .dbf files, about 10 different command line utilties and several large SQL queries. The end result was a mapping of 10.8 million line IDs (linearid field in TIGER speak) to fully expanded road names which I applied to the rendering database. The result can be seen on this &lt;a href="http://tile.openstreetmap.us/tiger2012_roads_expanded/preview.html#16/37.8112/-122.2747" target="_blank"&gt;tile preview page&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;&lt;h3&gt;Imagery preset updates&lt;/h3&gt;&lt;br /&gt;Related to the road name tiles, I noticed that Potlatch2 had a TIGER road name tiles preset in the "Background" menu but it was still pointing at tiles with 2011 data in them. Thanks to the magic of github, I was able to &lt;a href="https://github.com/systemed/potlatch2/pull/90" target="_blank"&gt;get that fixed&lt;/a&gt; easily.&lt;br /&gt;&lt;br /&gt;Then I noticed that the &lt;a href="http://josm.openstreetmap.de/wiki/Maps" target="_blank"&gt;JOSM imagery sources&lt;/a&gt; page didn't have this tile set listed at all. That one was a little trickier to fix. It involved coming up with a simplified polygon of the US border to define the available extent of the tile layer as well as putting a base64 encoded image into an XML tag on the JOSM wiki. The image+base64+XML+wiki thing seems a little nasty but the JOSM functionality that this enables is actually pretty neat. If you download OSM data inside of the US, it will pop up a notice that informs you that that there is additional background imagery available and lets you add it to the "Imagery" menu with a single click. The base64 encoded image is displayed next to the menu option. In this case it is the TIGER logo:&lt;br /&gt;&lt;br /&gt;&lt;table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td style="text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-Qx5r646Pjec/UKxy_pMqK6I/AAAAAAAABas/NpOb_1FOc7s/s1600/smalltiger.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"&gt;&lt;img border="0" src="http://3.bp.blogspot.com/-Qx5r646Pjec/UKxy_pMqK6I/AAAAAAAABas/NpOb_1FOc7s/s1600/smalltiger.gif" /&gt;&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td class="tr-caption" style="text-align: center;"&gt;RAWR!&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;span id="goog_1398736279"&gt;&lt;/span&gt;&lt;br /&gt;&lt;h3&gt;TIGER tag removal&lt;/h3&gt;&lt;br /&gt;The TIGER road import put some tags on imported objects that weren't really necessary. As people edit roads, some of the tags become even less useful as the information from the tiger tags diverges from the "normal" OSM tags. Some of the tags were useful at the time for the upload process itself but are now just annoying bloat that confuses new users.&lt;br /&gt;&lt;br /&gt;Back in July I started a &lt;a href="http://lists.openstreetmap.org/pipermail/talk-us/2012-July/008830.html" target="_blank"&gt;mailing list thread&lt;/a&gt; about having editors automatically drop some of these tags like they already drop the created_by tag. Out of that discussion came a JOSM &lt;a href="https://josm.openstreetmap.de/ticket/7915" target="_blank"&gt;trac ticket and patch&lt;/a&gt; that does exactly this. Right before uploading, JOSM checks to see if any of the modified objects it is about to upload have "discardable" TIGER tags on them and removes them.&lt;br /&gt;&lt;br /&gt;In September, I opened &lt;a href="https://trac.openstreetmap.org/ticket/4552" target="_blank"&gt;a ticket&lt;/a&gt; against Potlatch2 with the same feature request. Since nothing had happened with that ticket and I already had the Potlatch2 repository cloned from the tile URL change pull request, I decided to take matters into my own hands. I had never so much as glanced at ActionScript but fortunately P2 was a breeze to get running in a development environment. The &lt;a href="http://wiki.openstreetmap.org/wiki/Potlatch_2/Developer_Documentation" target="_blank"&gt;developer documentation page&lt;/a&gt;&amp;nbsp;on the wiki got me up and running in a matter of minutes. Although I did have to give ant more PermGen space to get through compilation. I should probably update the wiki with that tidbit...&lt;br /&gt;&lt;br /&gt;After a little over an hour I had &lt;a href="https://github.com/systemed/potlatch2/pull/91" target="_blank"&gt;another pull request&lt;/a&gt; fashioned which Richard was kind enough to merge the next day. I believe this went live on osm.org within the last few hours. So now every way that is touched during Operation Cowboy will help to clean this clutter out of the database without the user having to do anything.&lt;br /&gt;&lt;br /&gt;&lt;h3&gt;YEEEEHAW!&lt;/h3&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So that's how I prepared for Operation Cowboy. Unfortunately the actual event will take place over the Thanksgiving weekend which means I will be in Nebraska with very limited internet access so I won't be able to participate much with actual mapping. I will probably manage to hang out in the IRC channel a bit to try and field some questions if they come up. Please join me! (#opc2012 on irc.oftc.net)&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;span style="font-size: x-small;"&gt;Answer key for TIGER abbreviations:&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;div&gt;&lt;span style="font-size: x-small;"&gt;Brg = Bridge&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span style="font-size: x-small;"&gt;Cll = Calle (Spanish)&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span style="font-size: x-small;"&gt;Natl Lkshr = National Lakeshore&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span style="font-size: x-small;"&gt;Ofc = Office&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span style="font-size: x-small;"&gt;Pso = Paseo (Spanish)&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span style="font-size: x-small;"&gt;RM = Ranch to Market Road&lt;/span&gt;&lt;/div&gt;&lt;div&gt;&lt;span style="font-size: x-small;"&gt;BIA Rte = Bureau of Indian Affairs Route&lt;/span&gt;&lt;/div&gt;&lt;/div&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/c4_i5cJBUME" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2012/11/preparing-for-operation-cowboy.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://3.bp.blogspot.com/-Qx5r646Pjec/UKxy_pMqK6I/AAAAAAAABas/NpOb_1FOc7s/s72-c/smalltiger.gif" height="72" width="72" /><thr:total>1</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-6660965256320040292</guid><pubDate>Wed, 24 Oct 2012 03:53:00 +0000</pubDate><atom:updated>2012-10-23T22:53:43.605-05:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">data integrity</category><category domain="http://www.blogger.com/atom/ns#">ODbL</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">TIGER</category><title>Licensed to Map (What happened to Los Angeles?!)</title><description>I recently got back from the &lt;a href="http://stateofthemap.us/" target="_blank"&gt;State of the Map - USA&lt;/a&gt;&amp;nbsp;conference. It was great and you should have been there! But for those who weren't... I presented one session on Saturday afternoon with the same title as this post. Instead of just putting my slides out there I thought I would write a blog post to tell more of the story. This is a fairly long post but don't worry, it has a lot of pictures! (click to see full resolution)&lt;br /&gt;&lt;br /&gt;So, the license change. It happened. We lost some data. But what happened before that to try and save as much as we could? And what exactly did we lose?&lt;br /&gt;&lt;br /&gt;First, a brief timeline:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Many moons ago, the OSM Foundation voted to change our license from Creative Commons to the Open Database License. This actually happened before I knew that OSM existed.&lt;/li&gt;&lt;li&gt;In order to make the change, permission had to be secured from everyone who had contributed map data to the database.&lt;/li&gt;&lt;li&gt;Contributions from anyone who did not agree to the new terms had to be removed from the database.&lt;/li&gt;&lt;li&gt;At the end of 2011 the board set April 1st, 2012 as a target date for the license switch. This turned out to be wildly optimistic but at least it was a concrete goal to shoot for instead of just an ongoing process with no end in sight.&lt;/li&gt;&lt;li&gt;The process of removing non-relicensable data was done by a bot that went through the entire database in the 2nd half of June&lt;/li&gt;&lt;li&gt;After a little more cleanup, the first ODbL planet file was finally delivered in September.&lt;/li&gt;&lt;/ul&gt;Even though the April 1st target date was not really reasonable to hit, (especially in hindsight of course) it still gave the community something to work towards. And work we did.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;&lt;span style="font-size: large;"&gt;Contacting inactive mappers&lt;/span&gt;&lt;/h2&gt;The first priority was to contact undecided users to make them aware of the change and that they needed to log in and indicate a decision. There were several rounds of emails sent out from the foundation to undecided users, trying to get them to respond. In addition to the emails from the foundation, users actively contacted other people in their area to try and make them aware of the decision. Lastly, the foundation supplied the account email address of some of the accounts with the most map data to a small group of volunteers for more targeted contact.&lt;br /&gt;&lt;br /&gt;The result of this contact effort can be seen in this graph that I have been presenting since the beginning of the process:&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-TnzlkN1nvXQ/UH4qdtYkgOI/AAAAAAAABUo/5KjReAixVZs/s1600/osm_license_acceptance.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="217" src="http://4.bp.blogspot.com/-TnzlkN1nvXQ/UH4qdtYkgOI/AAAAAAAABUo/5KjReAixVZs/s640/osm_license_acceptance.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;The green line&amp;nbsp;corresponds&amp;nbsp;to the scale on the left while the red line follows the scale on the right. Of those who responded, it was a 99.5% landslide in favor of&amp;nbsp;re-licensing their data. You can see that the people who were opposed to the change were very quick to log in and enter their decision. The big bumps in the green line clearly show when the mass emails were being sent out by the foundation.&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;I was the volunteer who did targeted contact here in the U.S. It was an interesting project. &lt;a href="http://wiki.openstreetmap.org/wiki/User:SimonPoole" target="_blank"&gt;Simon Poole&lt;/a&gt; came up with a prioritized list of users to contact per country/region based on his site&amp;nbsp;which&amp;nbsp;shows undecided users ranked by how much map data they have in the database. (&lt;a href="http://odbl.poole.ch/" target="_blank"&gt;http://odbl.poole.ch/&lt;/a&gt;)&lt;br /&gt;&lt;br /&gt;I ended up contacting people through a variety of websites including obvious social networking sites like Facebook, Google+, Twitter, Flickr and LinkedIn. I also sent a few though more obscure sites like tripadvisor and soundcloud. Beyond that, I attempted to determine their real name and any contact info I could get my hands on. A simple web search with their OSM user name plus the location of their first edit was remarkably effective. Often it would yield a twitter account which would then give me a real name and lead to accounts on other sites. Here is a screenshot of the spreadsheet I used to track my contact with users:&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-4qTQLT3RbwQ/UH4xE1HaTWI/AAAAAAAABU4/wLhS0odL_CU/s1600/user_contact_list.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="400" src="http://1.bp.blogspot.com/-4qTQLT3RbwQ/UH4xE1HaTWI/AAAAAAAABU4/wLhS0odL_CU/s400/user_contact_list.png" width="342" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;I kept track of dates I sent messages, when I heard back and any accounts around the internet that were possibly related to the user in question. All in all, I attempted to contact a total of 168 users. 71% of them accepted the new&amp;nbsp;contributor terms. Only one or two of those who responded did not accept the new terms. The rest I was unable to contact.&lt;br /&gt;&lt;br /&gt;As part of my contact efforts I ended up calling 7 people on the telephone. This led to a few awkward conversations. One person said "I don't remember giving OSM my phone number" to which I replied "Well... you didn't. I stalked you across the internet." Another call landed me in a conversation with a high school student's mother. When I asked to speak to him she asked "who is this?" to which I responded that her son did not know me but that I am from a website that her son had used. I am happy to report that both of these users agreed to the new terms within 5 minutes of talking to me! However a few times I felt kind of like this:&lt;br /&gt;&lt;br /&gt;&lt;table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td style="text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-R4q8iTfeMEQ/UH416ryUwAI/AAAAAAAABVI/4iJh9V_UGy0/s1600/fromtheinternet.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"&gt;&lt;img border="0" height="320" src="http://1.bp.blogspot.com/-R4q8iTfeMEQ/UH416ryUwAI/AAAAAAAABVI/4iJh9V_UGy0/s320/fromtheinternet.png" width="308" /&gt;&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td class="tr-caption" style="text-align: center;"&gt;Found on&amp;nbsp;http://www.freewebs.com/jhnbytwoo/&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;br /&gt;&lt;h2&gt;&lt;span style="font-size: large;"&gt;Remapping&lt;/span&gt;&lt;/h2&gt;Besides contacting undecided users, there was also an effort to remove and replace the contributions of users who would not agree to the new terms. There were several tools to assist us in this effort. Unfortunately I didn't get good screenshots of most of them. At the time I wasn't planning on doing a presentation about this and now that the license change is over, they are either shut down or not displaying useful information.&lt;br /&gt;&lt;br /&gt;But they basically all had a different way of highlighting map objects that were likely going to be either deleted or modified by the removal of license tainted data. Both JOSM and Potlatch 2 had a license change plugin/mode that would draw red halos around objects that were in danger of deletion and yellow ones around things that would be modified. Next was the OSM Inspector license change view. It also displayed objects that would be touched by the bot and let you click on them to get details as well as links to the object history and edit links to go fix it. These were based on the "&lt;a href="http://wiki.openstreetmap.org/wiki/WTFE" target="_blank"&gt;Quick History Service&lt;/a&gt;" which was a pretty good approximation of the algorithm the bot would use to remove data. There was also&amp;nbsp;&lt;a href="http://wiki.openstreetmap.org/wiki/CLEANMAP" target="_blank"&gt;cleanmap/badmap&lt;/a&gt; which was two different map renderings. Cleanmap showed a rendering of what the map would look like after the license change and badmap showed only things that needed to be remapped.&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;&lt;span style="font-size: large;"&gt;The license bot&lt;/span&gt;&lt;/h2&gt;In the second half of July, the "license bot" was run over the planet to remove all license tainted data from the database. After the full bot run there was some additional cleanup to do. For example there were some obvious cases of copy/paste remapping where people "remapped" by straight up copying the data of users who had not agreed to the new terms. This is obviously copyright infringement and needed to be taken care of. In the bot removal process, data which was originally created by decliners or non-responders was completely removed from the database. Objects which had been modified by decliners/non-responders were reverted to an older state to remove the data that couldn't be relicensed.&lt;br /&gt;&lt;br /&gt;The official number I heard after the process was done is that OpenStreetMap lost 1.2% of its data. Of course removing 1% of the Interstate system makes it completely useless for routing across the country so the true impact of the bot was much higher but I'm not sure it is possible to put a specific number on it. The effects of the bot were also highly variable by location. Australia and Poland got hit disproportionately hard as did some cities here in the U.S.&lt;br /&gt;&lt;br /&gt;The bot had a slight blind spot having to do with splitting ways. When a way is split in OSM, a brand new way is created and attributed entirely to the user who split the way. No link is kept to the user who originally created the long way. This issue was known about ahead of time but the effort required to try and guess on way splits was deemed too high in comparison to the difference it would have made on the data.&lt;br /&gt;&lt;br /&gt;So, what does 1.2% of OSM data look like? Here are two graphs from&amp;nbsp;&lt;a href="http://osmstats.alltogetherlost.com/"&gt;http://osmstats.alltogetherlost.com/&lt;/a&gt;&amp;nbsp;which show the total number of nodes and ways in the OSM database over time:&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-DGQxalunzKc/UH5F83GI-LI/AAAAAAAABVY/efqr95m2H3g/s1600/bot_nodes_graph_clipped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="151" src="http://1.bp.blogspot.com/-DGQxalunzKc/UH5F83GI-LI/AAAAAAAABVY/efqr95m2H3g/s320/bot_nodes_graph_clipped.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-3VNAxQCiDPc/UH5F9Zx8PGI/AAAAAAAABVg/NHd_J_6eM-I/s1600/bot_ways_graph_clipped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="154" src="http://1.bp.blogspot.com/-3VNAxQCiDPc/UH5F9Zx8PGI/AAAAAAAABVg/NHd_J_6eM-I/s320/bot_ways_graph_clipped.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;You can clearly see a small bump in July. But really, it's little more than a blip. The growth of data in OSM appears to be kind of unstoppable.&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;h2&gt;&lt;span style="font-size: large;"&gt;The bot in America&lt;/span&gt;&lt;/h2&gt;&lt;div&gt;Since I presented this information at SOTM-US, I focused on some U.S. specific issues. We have some unique considerations, mostly having to do with the fact that a lot of our road network is imported from the TIGER/Line data set which is public domain. This means that the vast majority of our road network is not impacted by license considerations. However some of the TIGER fixup work that has happened since the import was definitely impacted. For example, there was one armchair mapper from Europe who did a lot of work on the interstate system in the eastern 1/3 of the country which was lost in the license change. As I mentioned above, way splitting was not taken into account by the license bot. Since splitting ways is a relatively common thing to do when fixing up TIGER data, this did lead to some data being removed that was technically from a public domain source. Luckily it tends to be relatively easy to restore this data, especially on interstates where a lot of the splitting happened to map bridges which are clearly visible in aerial imagery.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;To visualize the bot's activities, I made a map by analyzing the daily diffs that were published during the time the license bot was running in America. It only shows node edits. Red means the bot deleted a node, blue means it modified a node, likely reverting it to an earlier version.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-m7Kx9BYnJdw/UIcsP3RuPpI/AAAAAAAABV4/axn6as7w2To/s1600/bot_affected_nodes_USA.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="395" src="http://4.bp.blogspot.com/-m7Kx9BYnJdw/UIcsP3RuPpI/AAAAAAAABV4/axn6as7w2To/s640/bot_affected_nodes_USA.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Well that looks kind of bad. But it is exaggerated a bit because of the rendering style and low zoom. Let's look at some individual locations. First up: Kansas.&amp;nbsp;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-iJYg63hstwI/UIcszeIm7oI/AAAAAAAABWI/MlJmCrcvJQY/s1600/bot_affected_nodes_Kansas.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="378" src="http://2.bp.blogspot.com/-iJYg63hstwI/UIcszeIm7oI/AAAAAAAABWI/MlJmCrcvJQY/s640/bot_affected_nodes_Kansas.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Well that's pretty good. I got most of the license problems here in Kansas taken care of either by contacting users and getting them to agree to the change or by remapping things.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Next up is Dallas. On the national map it looks like a fairly decent sized blob.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-4RngWLQJnuo/UIcsy_agKvI/AAAAAAAABWA/vFzj7Zf209g/s1600/bot_affected_nodes_Dallas.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="380" src="http://1.bp.blogspot.com/-4RngWLQJnuo/UIcsy_agKvI/AAAAAAAABWA/vFzj7Zf209g/s640/bot_affected_nodes_Dallas.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;Upon closer inspection, it isn't really that bad. There are a few neighborhoods that were removed from the map and some damage to that highway heading east. But the overall structure of the road network isn't too bad.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Next, South Carolina:&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-8LE19I9KC7Y/UIcs1f4JYRI/AAAAAAAABWY/2i2-hoKlm9I/s1600/bot_affected_nodes_SouthCarolina.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="378" src="http://2.bp.blogspot.com/-8LE19I9KC7Y/UIcs1f4JYRI/AAAAAAAABWY/2i2-hoKlm9I/s640/bot_affected_nodes_SouthCarolina.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;That's a bit more serious. Obviously there was a lot of damage to the interstate system here.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;And finally, Los Angeles:&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-igx_Vp15xPM/UIcs0cO9LRI/AAAAAAAABWQ/B6f7rJzpsC8/s1600/bot_affected_nodes_LA.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="382" src="http://1.bp.blogspot.com/-igx_Vp15xPM/UIcs0cO9LRI/AAAAAAAABWQ/B6f7rJzpsC8/s640/bot_affected_nodes_LA.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;OK, now that's pretty heavy. Most of it is due to a single, very prolific, mapper who refused to agree to the new terms.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;So we know where the license bot was most active. But what exactly did it do to our data? Turns out, many different things. Here is an example of some pretty obvious damage:&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-6hu8wgRcs7Q/UIcy0busdaI/AAAAAAAABWs/2XCtPRfMpzk/s1600/oj_co_bot_damage.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="412" src="http://3.bp.blogspot.com/-6hu8wgRcs7Q/UIcy0busdaI/AAAAAAAABWs/2XCtPRfMpzk/s640/oj_co_bot_damage.jpg" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;The original TIGER import was pretty terrible in this area. A declining user corrected a bunch of the road geometry. Then an accepting user came along and added some details like turning circles and gates. When the bot ran, it did not touch the new additions made by the accepting user but moved all the nodes touched by the declining user back to their original positions, shearing the map rather severely.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Here is a somewhat amusing example:&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-apHSW0kuiN0/UIczscIpvZI/AAAAAAAABW0/KR3gUr_wZEQ/s1600/redaction_weird_offramp.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="302" src="http://3.bp.blogspot.com/-apHSW0kuiN0/UIczscIpvZI/AAAAAAAABW0/KR3gUr_wZEQ/s320/redaction_weird_offramp.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;In this case, the original TIGER had this onramp going the wrong direction. A declining user reversed the direction of the way. Then an accepting user came along and refined the shape of the way by adding more nodes to it. The bot reversed the order of the original nodes but not of the new nodes added by the accepting user, leading to the odd "zig" in the middle.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Now for a little more subtle damage:&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-T8vID_DMNg0/UIc0tJ1tIQI/AAAAAAAABW8/Y44iXMe5faI/s1600/license_bot_LA_slippy.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="452" src="http://3.bp.blogspot.com/-T8vID_DMNg0/UIc0tJ1tIQI/AAAAAAAABW8/Y44iXMe5faI/s640/license_bot_LA_slippy.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;You can see some obviously weird zigs and zags in the roads but nothing seems to be completely out of whack here. However, when viewed in JOSM, the damage becomes clearer.&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-CspfEJs_0oE/UIc0vpushiI/AAAAAAAABXE/avERPOaIWj4/s1600/license_bot_LA_JOSM_annotated.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="448" src="http://3.bp.blogspot.com/-CspfEJs_0oE/UIc0vpushiI/AAAAAAAABXE/avERPOaIWj4/s640/license_bot_LA_JOSM_annotated.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;In the upper left, there are nodes that used to be part of a way. Both the nodes themselves and the way made it through the license bot. But the fact that the nodes were a part of the way was lost leaving empty nodes and a misshapen way. In the lower part of the image you can see that there is a way which does not actually connect to any of the ways it crosses. So much for routability.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;h2&gt;&lt;span style="font-size: large;"&gt;Fixing the damage&lt;/span&gt;&lt;/h2&gt;&lt;/div&gt;&lt;div&gt;After the license bot ran, the community immediately started focusing on how to fix the damage. Several tools were developed or adapted to help guide and focus this effort. At the end of the day, we have risen to the challenge and I have actually been surprised at how quickly most of the damage has been fixed.&amp;nbsp;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I made a rough graph of remapping activity based on changeset comments. I used the weekly changeset dump along with my&amp;nbsp;my&amp;nbsp;&lt;a href="https://github.com/ToeBee/ChangesetMD" target="_blank"&gt;ChangesetMD&lt;/a&gt;&amp;nbsp;tool&amp;nbsp;to create a database of changeset metadata which I then queried for these keywords in the changeset comments:&amp;nbsp;odbl, license, remapping, redaction.&amp;nbsp;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/--bOCbXxoMW0/UIc21_wVDfI/AAAAAAAABXM/GGoWdefxQy4/s1600/remapping_activity_graph.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="426" src="http://3.bp.blogspot.com/--bOCbXxoMW0/UIc21_wVDfI/AAAAAAAABXM/GGoWdefxQy4/s640/remapping_activity_graph.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;It starts back in December 2011 when remapping started in earnest. The orange line follows the axis on the left and indicates the number of objects touched in these remapping changesets. This includes deletes, modifications and additions. The blue line follows the axis on the right and is just the number of remapping changesets created per day.&amp;nbsp;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;I believe the huge spike in the orange line at the end of January was me and a few other people really putting our backs into getting the coastlines cleaned up. &lt;a href="http://www.paulnorman.ca/blog/" target="_blank"&gt;Paul Norman&lt;/a&gt; came up with a good way to highlight shoreline ways that were in danger of being damaged by the bot and it turns out there were a lot of them. I nuked most of the shoreline along the west coast and replaced it with NHD data. Some others did the same or manually remapped the shoreline around the Great Lakes and the east coast.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;In May and June, remapping activity seems to have slowed to a crawl. I think people were just getting tired of it. I certainly was. The big blue spike on the right happened a couple of days after the license bot started running. It was actually still in progress at the time. But since Europe was hit first, it seems that the Europeans made quick work of cleaning up after it.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;My own contribution to the remapping effort was a map that highlighted ways that seemed to be missing their highway=* tag. I noticed that a fair number of interstates and on/off ramps had a lot of their tags stripped but one of the following tags remained: oneway, lanes, bridge, tunnel. Any of these tags without a highway tag is rather suspicious so I put them on a map to highlight and guide people to them for remapping. Once in the area of such a way I often found other problems in addition to the missing highway tag. Here is an animation of how my map evolved between August 8th and September 30th:&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-pEgarfZJWyY/UIc8dPZ9Q6I/AAAAAAAABXg/QXHymdZCp8I/s1600/oneway_map.gif" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="384" src="http://4.bp.blogspot.com/-pEgarfZJWyY/UIc8dPZ9Q6I/AAAAAAAABXg/QXHymdZCp8I/s640/oneway_map.gif" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;It started out only showing ways with a oneway tag. You can see some new red appear on the map when I added the other tags to the list. By August 28th, the U.S. was completely clear on this map. The Europeans still have quite a mess to clean up. You can find the map still online here:&amp;nbsp;&lt;a href="http://ni.kwsn.net/~toby/OSM/maps/redaction.html" target="_blank"&gt;http://ni.kwsn.net/~toby/OSM/maps/redaction.html&lt;/a&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Another effort to detect problems on major roads was the routing grid that Kai Krueger set up. It queried &lt;a href="http://project-osrm.org/" target="_blank"&gt;OSRM&lt;/a&gt; periodically for a route between 40 cities and compared the distance to a reference distance. If it was off by a lot, it indicated a likely problem along an interstate or US highway between the two cities. The difference was color coded and put into a display matrix. Here is another animation showing how the routing grid evolved between July 21st and September 30th:&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-F6p29qzO_6w/UIc-a6xIbII/AAAAAAAABXo/7NvGTXYKY_I/s1600/routing_grid_3.gif" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="574" src="http://4.bp.blogspot.com/-F6p29qzO_6w/UIc-a6xIbII/AAAAAAAABXo/7NvGTXYKY_I/s640/routing_grid_3.gif" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;Again, you can see that things got fixed up pretty quickly. Also, I'm not sure why everything went orange again right at the very end.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;One other strategy I used to fill in hard hit areas was to re-import the area from new TIGER 2011 and even some 2012 data when it became available. To do this, I downloaded both the shapefile and the feature names relationship file from the &lt;a href="http://www.census.gov/geo/www/tiger/shp.html" target="_blank"&gt;Census Bureau&lt;/a&gt;. The feature names file allowed me to do a better job of expanding street names from their abbreviated forms. I joined the two files in QGIS and then ran them through ogr2osm using my own &lt;a href="https://github.com/ToeBee/ogr2osm-translations" target="_blank"&gt;translation file&lt;/a&gt; which extracted only a few of the shapefile attributes into OSM tags and expanded abbreviations. This gave me a .osm file which I was able to load into JOSM and then selectively copy roads from there into OSM.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;Here is a screen shot of JOSM while I had some TIGER data loaded in the background with my "inactive" color set to a bright green to contrast the red of selected ways in the OSM layer on top of it. This highlighted areas that were missing in OSM. My apologies to any red-green colorblind readers.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-5jWKsj_KfeU/UIdA2tmCmcI/AAAAAAAABXw/CeEpdInMGfw/s1600/riverside_JOSM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="458" src="http://3.bp.blogspot.com/-5jWKsj_KfeU/UIdA2tmCmcI/AAAAAAAABXw/CeEpdInMGfw/s640/riverside_JOSM.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div&gt;After zooming in on the desired area I used the search string "type:way allinview" to automatically select all the ways in the TIGER layer, copied them and then pasted them into the OSM layer. Then of course I had to do some quality assurance to connect the new ways to the existing data and remove some common TIGER problems such as over-noded ways and duplicate ways that were created when TIGER had more than one name for a road. Typically this was just things like "Drive" vs "Avenue" or some other minor spelling difference.&lt;br /&gt;&lt;br /&gt;One excellent tool that&amp;nbsp;&lt;a href="https://oegeo.wordpress.com/" target="_blank"&gt;Martijn van Exel&lt;/a&gt; came up with is called "Remap-a-tron." I did not talk about it at the conference at all because&amp;nbsp;Martijn did a presentation about it right before my presentation. But you can read about it in his own &lt;a href="http://oegeo.wordpress.com/2012/08/29/remapathon/" target="_blank"&gt;blog post&lt;/a&gt;. Basically, he did some digging to find ways that had been deleted by the license bot. Then when you hit the web page, it would display one deleted way at a time and give you the opportunity to go remap it in your favorite editor using aerial imagery, surrounding data that wasn't removed or whatever other source of license acceptable data you could find. After remapping the way you would come back and indicate that you had remapped it. It would record the remapping and immediately show you the next way that needed attention. This tool has now been rebranded as "Maproulette" and is available at&amp;nbsp;&lt;a href="http://maproulette.org/" target="_blank"&gt;http://maproulette.org&lt;/a&gt;&amp;nbsp;where it has been retasked to find other common map errors that are easy to fix.&lt;br /&gt;&lt;br /&gt;Finally, there is also a "&lt;a href="http://tools.geofabrik.de/osmi/debug.html?view=redactionbot" target="_blank"&gt;Redaction Bot&lt;/a&gt;" view in the Geofabrik OSM Inspector tool. It shows all objects that were touched by the bot. You can click on them to get details about what the bot did to a particular object. I don't think it tries to match things that were deleted by the bot to new mapping done to replace them so over time the signal to noise ratio of this tool will decrease. Here is a screenshot of it with one way selected.&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-6eUopxuY498/UIdgoCZOjsI/AAAAAAAABYQ/cXzesoT31HI/s1600/geofabrik_OSMI.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="394" src="http://4.bp.blogspot.com/-6eUopxuY498/UIdgoCZOjsI/AAAAAAAABYQ/cXzesoT31HI/s640/geofabrik_OSMI.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;h2&gt;&lt;span style="font-size: large;"&gt;The End&lt;/span&gt;&lt;/h2&gt;&lt;/div&gt;&lt;div&gt;In conclusion, the license change definitely had a major impact on OpenStreetMap. Both our map data and the community were impacted, sometimes to great detriment. However we are getting past it remarkably quickly thanks to the hard work of mappers like you. Some damage will remain in the map for a long time to come. Deleted points of interest aren't nearly as obvious as broken interstates and will take much longer to fill back in. But at the end of the day, we will survive and continue to thrive. In some ways, 2012 really was the year of OpenStreetMap, despite the license change. Let's make 2013 even better!&lt;br /&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;h2&gt;&lt;span style="font-size: large;"&gt;Map the planet!&lt;/span&gt;&lt;/h2&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/hIuqAK-Ssgk" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2012/10/licensed-to-map-what-happened-to-la.html.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://4.bp.blogspot.com/-TnzlkN1nvXQ/UH4qdtYkgOI/AAAAAAAABUo/5KjReAixVZs/s72-c/osm_license_acceptance.png" height="72" width="72" /><thr:total>0</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-2574779849454093341</guid><pubDate>Fri, 08 Jun 2012 02:50:00 +0000</pubDate><atom:updated>2012-09-21T18:34:38.172-05:00</atom:updated><title>Making a location badge using OsmAnd and the MapQuest Open static map API</title><description>I'm about to embark on my second &lt;a href="http://www.bak.org/" target="_blank"&gt;Biking Across Kansas&lt;/a&gt; tour. When I did it last year, I used Google Latitude to create a little "My Location" badge which I put here on my blog to let people track me across the state. The Google one looks like this:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-f3H4To1O__c/T88CsQsnd0I/AAAAAAAAAvk/SlOCc_mg3tU/s1600/google_location_badge.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="200" src="http://4.bp.blogspot.com/-f3H4To1O__c/T88CsQsnd0I/AAAAAAAAAvk/SlOCc_mg3tU/s200/google_location_badge.png" width="141" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;However at some point during the last year I noticed that &lt;a href="http://www.osmand.net/" target="_blank"&gt;OsmAnd&lt;/a&gt; added a feature that allows it to periodically ping a web service of your choosing with the current location of the phone. This allows me to build my own location publishing service. &lt;br /&gt;&lt;br /&gt;I did a little searching for how other people have used this feature in OsmAnd and found a few people talking about feeding the data into a Google map display. The example in the OsmAnd wiki posts the location to a Google doc which then somehow magically updates a Google map display. No offense to Google but I'd rather use OSM based products. Fortunately the good people over at &lt;a href="http://open.mapquest.com/" target="_blank"&gt;MapQuest Open&lt;/a&gt; provide a service that suits this situation: The &lt;a href="http://open.mapquestapi.com/staticmap/" target="_blank"&gt;static map API&lt;/a&gt;.&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;Behold my creation (and location):&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-jZTj5KLUAiA/UFz5DUwyDtI/AAAAAAAABG4/kF4JgODSwus/s1600/tracker.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="640" src="http://2.bp.blogspot.com/-jZTj5KLUAiA/UFz5DUwyDtI/AAAAAAAABG4/kF4JgODSwus/s640/tracker.png" width="622" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Note that this is no longer an active link since BAK is over. This is just a screenshot of what it looked like.&lt;br /&gt;&lt;br /&gt;Now for the details. The system has two components. A web service to collect the location information from the phone and a display page that creates a pretty map to look at.&lt;br /&gt;&lt;br /&gt;I hacked together some PHP in about an hour. Not the biggest fan of PHP and I'm pretty sure I'm doing some things wrong but I got something working and don't have time to make any improvements before I leave for BAK in under 12 hours so there you have it. &lt;a href="https://github.com/ToeBee/OsmAnd-Tracker" target="_blank"&gt;The code is on github&lt;/a&gt;.&lt;br /&gt;&lt;br /&gt;The location web service takes the GET request from the phone and writes the information contained in it to a file. The display page reads the file, pulls out the coordinates and uses them to construct a URL that retrieves a static map image from the MapQuest Open static map API. It isn't an interactive map, just a quick "I am here" image. However I have wrapped the image in a link that takes youto an interactive map on openstreetmap.org. Throw that into an iframe tag and you're good to go!&lt;br /&gt;&lt;br /&gt;Possible future improvements might include the following&lt;br /&gt;&lt;ul&gt;&lt;li&gt;&lt;div&gt;Perform a nominatim query to get a nearby city name and display that&lt;/div&gt;&lt;/li&gt;&lt;li&gt;Better controls over the accuracy of the position displayed. The google badge lets you only reveal which city you are in if you wish. Currently I only allow rounding coordinates to a given number of decimals (default is two)&lt;/li&gt;&lt;li&gt;Dynamic sizing of the map based on parameters passed in to the iframe from the parent page. This example is rather huge and not really a "badge" but that's what I want in this situation.&lt;/li&gt;&lt;li&gt;Track more than just the last location ping and display distance traveled between updates or something else interesting. The static map API is pretty flexible with displaying multiple POIs or even lines or shapes.&lt;/li&gt;&lt;li&gt;...&lt;/li&gt;&lt;li&gt;PROFIT!&lt;/li&gt;&lt;/ul&gt;I am packing my bags as I finish this post so I don't really have time to brainstorm anything else. I'm sure others will have ideas :)&lt;br /&gt;&lt;div&gt;&lt;br /&gt;&lt;div&gt;See you in 500 miles!&lt;/div&gt;&lt;/div&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/W7GoQ5RWHaU" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2012/06/making-location-badge-using-osmand-and.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://4.bp.blogspot.com/-f3H4To1O__c/T88CsQsnd0I/AAAAAAAAAvk/SlOCc_mg3tU/s72-c/google_location_badge.png" height="72" width="72" /><thr:total>2</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-5540953464009813785</guid><pubDate>Sun, 11 Mar 2012 22:36:00 +0000</pubDate><atom:updated>2012-03-11T19:01:34.430-05:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">mapping</category><category domain="http://www.blogger.com/atom/ns#">ODbL</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">TIGER</category><title>Remapping using TIGER 2011</title><description>The license change mapping process I talked about in my &lt;a href="http://ksmapper.blogspot.com/2012/01/license-change-mapping.html"&gt;previous post&lt;/a&gt; works well where there is still some clean data to go off of and you're just having to redo some work that a decliner did in splitting ways or adding some nodes to refine the geometry. But what about when an entire area is on the chopping block? I present the &lt;a href="http://tools.geofabrik.de/osmi/?view=wtfe"&gt;OSM Inspector&lt;/a&gt; view in Glendale, CA:&lt;br /&gt;&lt;span id="goog_639475318"&gt;&lt;br /&gt;&lt;/span&gt;&lt;span id="goog_639475319"&gt;&lt;/span&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-d8qLLPU5sJQ/T10KNM0Z2wI/AAAAAAAAAnE/MFOar-B8CYM/s1600/glendale_dirty.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="440" src="http://1.bp.blogspot.com/-d8qLLPU5sJQ/T10KNM0Z2wI/AAAAAAAAAnE/MFOar-B8CYM/s640/glendale_dirty.png" width="630" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;The area I worked on was just south of here and was actually even worse although I didn't get a "before" screenshot. Less yellow, more red. Here is what it looked like on Simon Poole's "&lt;a href="http://cleanmap.poole.ch/?layers=00B0"&gt;badmap&lt;/a&gt;"&amp;nbsp;rendering. Pretty much the entire road network is going to get nuked:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-Ef7upQxWi6Q/T10MjBS7ftI/AAAAAAAAAnM/uhKgHH2C954/s1600/remapping_before.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="352" src="http://2.bp.blogspot.com/-Ef7upQxWi6Q/T10MjBS7ftI/AAAAAAAAAnM/uhKgHH2C954/s640/remapping_before.png" width="630" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Most of this data is from a single, very prolific contributor who is not likely to agree to the new terms. It is also highly unlikely that anyone in the area is going to be able to redo all of his work before April 1st. So I reached for a bigger paintbrush: &lt;a href="http://www.census.gov/geo/www/tiger/tgrshp2011/tgrshp2011.html"&gt;TIGER 2011&lt;/a&gt;.&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;To use TIGER data in OSM is no simple matter. For one thing, the Census Bureau uses abbreviations rather heavily and OSM is strongly anti-abbreviation, and for good reason. Unfortunately it is not a simple matter of throwing a regular expression at it and expanding all "Ave" to "Avenue" and "N" to "North" - what if there is a street named "N Street" which runs parallel to "O Street"? But then again, there may be a legitimate "North Street" somewhere. The TIGER data does indicate what pre- and post-fixes are present on a given road so the full name can be reconstructed with some effort. But to use this information is a little cumbersome. And of course the accuracy is always an issue but that's kind of a given with this data.&lt;br /&gt;&lt;br /&gt;First I downloaded the shapefile for Los Angeles county. Then I had to get the "Feature Names Relationship File" and use QGIS to join the two based on the LINEARID field. I ended up with a shapefile with all the name information present. The complication here is that sometimes roads have multiple entries in the relationship file and I'm not sure what QGIS does in this case. Since I was only working on a small area, I reviewed the data and made sure it all looked sane.&amp;nbsp;Since a whole county's worth of data can be hard to work with in JOSM, I clipped out just the area I wanted to work on and exported that as a new shapefile in QGIS.&lt;br /&gt;&lt;br /&gt;The next step was to convert this data to .osm format. I used a new version of the &lt;a href="https://github.com/andrewguertin/ogr2osm"&gt;ogr2osm&lt;/a&gt; python script written by Andrew Guertin to do this. To translate shapefile attributes to OSM tags requires a translation file. I cooked one up based on some previous traffic on the talk-us mailing list for mapping TIGER road types to OSM highway=* tags. For the name, I did some more looking at the data and came up with a bare minimum implementation for the few hundred highways I had to work with. I don't expect this to work well in other places without modification. But here is what I came up with. Patches welcome :)&lt;br /&gt;&lt;a href="https://github.com/ToeBee/ogr2osm-translations/blob/master/tiger2011.py"&gt;https://github.com/ToeBee/ogr2osm-translations/blob/master/tiger2011.py&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;And that was the easy part.&lt;br /&gt;&lt;br /&gt;Now to integrate this data into OSM. First I downloaded the entire area in JOSM, selected everything and ran the &lt;a href="http://wiki.openstreetmap.org/wiki/JOSM/Plugins/LicenseChange"&gt;LicenseChange&lt;/a&gt; plugin against it. Of course the whole screen lit up in red. Then I opened the .osm file generated by ogr2osm. I also changed my JOSM settings to render the "inactive" layer in a bright color so I could easily see where things were missing.&lt;br /&gt;&lt;br /&gt;Then it was just a matter of spending a few hours deleting ways in the area, switching to the TIGER layer, copying the missing roads and then pasting them into the "live" layer of OSM data. Then of course I had to find all the places where the old and the new data intersected and recombine them to maintain correct topology. The JOSM validator can help with this although I'm sure some mistakes made it in. The larger an area you can do at once, the less fiddling you have to do with the topology since all the data you copy/paste from the TIGER data will (for the most part) have correct internal topology. But bigger areas are harder to keep track of as you go. I usually tried to find relatively self-contained areas with as few as possible connections to the rest of the world. This spot on the map is a little unique in that it seems to be in a hilly area and it isn't your average grid pattern of streets. So there were actually isolated networks that I could deal with in chunks.&lt;br /&gt;&lt;br /&gt;Of course I always had to keep an eye out for TIGER strangeness. Some fictional roads and that kind of thing. 99% of what I touched was just residential roads. Where I did touch tertiary/secondary ones I did some additional comparison since TIGER's road classifications don't always match up with reality.&lt;br /&gt;&lt;br /&gt;This whole process was kind of complicated when there was actually something I &amp;nbsp;&lt;b&gt;didn't&lt;/b&gt; have to replace. I always prefer local mapper contributions over TIGER data so I had to make sure I didn't steamroll over what little clean data I did manage to find.&lt;br /&gt;&lt;br /&gt;So at the end of the day (literally) the area now looks like this on badmap. Obviously I was only concentrating on the road network so the parks are still there.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-4iJTeiSmruE/T10a-AzF3SI/AAAAAAAAAnU/XXBPI7hYbyA/s1600/remapping_after.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="445" src="http://2.bp.blogspot.com/-4iJTeiSmruE/T10a-AzF3SI/AAAAAAAAAnU/XXBPI7hYbyA/s640/remapping_after.png" width="630" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;And even from a lower zoom, there is a noticeable hole in OSMI:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-FItX3Dgye70/T10ldsptkxI/AAAAAAAAAnc/vkXzNu09nZ8/s1600/osmi_after.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="530" src="http://3.bp.blogspot.com/-FItX3Dgye70/T10ldsptkxI/AAAAAAAAAnc/vkXzNu09nZ8/s640/osmi_after.png" width="630" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Definitely an improvement but there is still much, MUCH more work to do :(&lt;br /&gt;&lt;br /&gt;The changeset I did this in can be seen here:&lt;br /&gt;&lt;a href="http://www.openstreetmap.org/browse/changeset/10937678"&gt;http://www.openstreetmap.org/browse/changeset/10937678&lt;/a&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/2w1w94-99AA" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2012/03/remapping-using-tiger-2011.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://1.bp.blogspot.com/-d8qLLPU5sJQ/T10KNM0Z2wI/AAAAAAAAAnE/MFOar-B8CYM/s72-c/glendale_dirty.png" height="72" width="72" /><thr:total>1</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-4911668036047260885</guid><pubDate>Sun, 08 Jan 2012 06:04:00 +0000</pubDate><atom:updated>2012-01-11T20:31:42.135-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">license</category><category domain="http://www.blogger.com/atom/ns#">mapping</category><category domain="http://www.blogger.com/atom/ns#">ODbL</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><title>License Change Mapping</title><description>So far I have avoided saying anything about the OpenStreetMap&amp;nbsp;&lt;a href="http://www.osmfoundation.org/wiki/License/We_Are_Changing_The_License"&gt;license change&lt;/a&gt; on this blog. It is a very sensitive subject for a lot of people for a million different reasons. This is &lt;b&gt;not&lt;/b&gt; a post about how right/wrong the license change is. I personally have agreed to the new terms but then again I don't really care what license my data gets redistributed under. While I prefer (in most situations) a share-alike clause, if we were switching to public domain instead of ODbL, I would probably still agree. Whatever.&lt;br /&gt;&lt;br /&gt;But the license change is happening and we must prepare the map for it, ahead of the &lt;a href="http://blog.osmfoundation.org/2011/11/16/odbl-progress/"&gt;April 1 cutover date&lt;/a&gt; set by the license working group. Today I decided to take a look at what it would take to get I-70 in Kansas cleaned up, license-wise, and thought I'd share my findings. Note that this particular process may or may not apply to other areas. The interstate system is kind of special in that it was imported from a public domain source and has very long stretches that are identical except for tags that don't really matter. This means that there isn't really very much unique or unrecoverable information present. If there is a small section of road (say a bridge) that is dirty, it can just be removed and replaced from clean data that surrounds it plus imagery to verify that it is indeed a bridge. However whenever possible it is preferred to maintain the history of objects in OSM so I don't want to just nuke and replace large sections of I-70. Plus, deleting and recreating ways gets more complicated because of the route relations that they may be a part of.&lt;br /&gt;&lt;br /&gt;A majority of I-70 in Kansas is already clean but there is still a&amp;nbsp;noticeable&amp;nbsp;fraction of dirty objects. The goal is to end up with all objects being "clean" to re-license under ODbL. First task: find dirty objects. For this&amp;nbsp;I use the "License Change" view in the OSM inspector tool created by Geofabrik:&amp;nbsp;&lt;a href="http://tools.geofabrik.de/osmi/?view=wtfe"&gt;http://tools.geofabrik.de/osmi/?view=wtfe&lt;/a&gt;&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;Here is a section of I-70 I will be working on.&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-zF8DpseBHMg/TwjMkIA5frI/AAAAAAAAAjI/R--1Nxm3uEc/s1600/1_inspector_dirty_view.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="400" src="http://3.bp.blogspot.com/-zF8DpseBHMg/TwjMkIA5frI/AAAAAAAAAjI/R--1Nxm3uEc/s640/1_inspector_dirty_view.png" width="635" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;You can mouse over the things on the left to find out exactly what the colors and symbols mean. Basically red and orange are bad. In this screen shot I have already clicked on the right-most red dot. This brought up the information in the upper right panel which indicates that there are two items of interest near where I clicked. Node 388499694 and way 33918862 were both created by declining user moonwashed.&lt;br /&gt;&lt;br /&gt;Now I click on the clock icon of the way. This brings up Ian Dees' "Deep Diff" service which displays a nice visual history of the way.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-ZrMcaKHQhXg/TwjNvJlfu0I/AAAAAAAAAjQ/iDkthBniUxY/s1600/2_deep_history_view.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="356" src="http://2.bp.blogspot.com/-ZrMcaKHQhXg/TwjNvJlfu0I/AAAAAAAAAjQ/iDkthBniUxY/s640/2_deep_history_view.png" width="635" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;From this I can see that the declining user created the way and made one edit to it. The subsequent 11 edits have all been by users who have accepted the new terms. The only original tags that have survived unchanged from the declining user are highway=motorway and oneway=yes. In this case, I consider these both trivial tags that should not prevent this way from being re-licensed. I have yet to encountered this along I-70 but there may be non-trivial tags that were added by a declining user. In this case, such tags would have to be removed to make the way "clean."&lt;br /&gt;&lt;br /&gt;Now, what about the nodes? I have the "remote control" feature enabled in JOSM which means that I can go back to the OSM inspector and click on the icon with the little "J" on it next to the way. This will cause JOSM to automatically download a bounding box around that way and select the way after it is downloaded. (&lt;strike&gt;This doesn't seem to work from Opera though&lt;/strike&gt; see comment below) I also have the &lt;a href="http://wiki.openstreetmap.org/wiki/JOSM/Plugins/LicenseChange"&gt;License Change Plugin&lt;/a&gt;&amp;nbsp;installed which I used to do a license check on the selected way. Here is the resulting JOSM window:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-NNYIHQVQpgU/TwjSn7lCDQI/AAAAAAAAAjo/-j1nSBHmF9k/s1600/3_josm_loaded.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="396" src="http://1.bp.blogspot.com/-NNYIHQVQpgU/TwjSn7lCDQI/AAAAAAAAAjo/-j1nSBHmF9k/s640/3_josm_loaded.png" width="635" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;I have also selected all of the dirty nodes in the license problem list. Now if I click the "Select" button, it will select all the dirty nodes. Next I inspect them to make sure none of them are nodes that connect to other ways. This could cause problems in the next step. And in fact, the one at the left end of the way is a junction node with a bridge. So I will unselect it using a CTRL+click and deal with it later. Now that only dirty nodes are selected, I delete them all. This will leave me with a way that is of course horribly wrong in terms of geometry. But now I can turn on the Bing imagery and go back and re-improve the way from Bing using the &lt;a href="http://ksmapper.blogspot.com/2011/12/most-interesting-mapper.html"&gt;ImproveWayAccuracy&lt;/a&gt; plugin:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-UV8bY5TXcxQ/TwjVGf_szDI/AAAAAAAAAjw/dL3MIEeIW2U/s1600/4_reimproving_way.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="395" src="http://1.bp.blogspot.com/-UV8bY5TXcxQ/TwjVGf_szDI/AAAAAAAAAjw/dL3MIEeIW2U/s640/4_reimproving_way.png" width="635" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;And of course the screen shot software intentionally left out the mouse cursor... but it is where the green line makes a bend which is where I am about to add a node. Now I check along the rest of the way to make sure everything else is ok.&lt;br /&gt;&lt;br /&gt;Now back to that dirty node I left out. The difficulty in this case is that the node is part of a 2-node way (a short bridge) so if I delete the dirty node, the bridge will get deleted as well. Since I don't want that, I will create a new node in the middle of the bridge, then delete the dirty node. This will create a gap between the two ways which needs to be fixed by re-extending one of the ways to connect with the other. Then all that I need to do is fix the position of the junction node according to imagery. This leaves all the nodes clean.&lt;br /&gt;&lt;br /&gt;Note that for untagged nodes, simply moving might be enough to clean them, depending on what rules end up being applied during the license chage. But simply moving a node by 1 meter to make sure it is ODbL clean doesn't give me warm fuzzy feelings. I'd rather just scrap them and start from scratch to avoid uncertainty.&lt;br /&gt;&lt;br /&gt;But the way itself is still considered dirty. For the following 3 reasons, I believe that the way can still be&amp;nbsp;re-licensed:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;The way was originally created from a public domain import. All the declining user did was split it, probably to make the bridge.&lt;/li&gt;&lt;li&gt;All tags are either trivial (as mentioned above) or have been added/changed by agreeing users&lt;/li&gt;&lt;li&gt;The geometry is now completely clean as I removed all dirty nodes and recreated them as I would when mapping something from scratch: aerial imagery and GPS traces&lt;/li&gt;&lt;/ul&gt;So to indicate this, I will add the following two tags to the way:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;odbl=clean&lt;/li&gt;&lt;li&gt;odbl:note=decliner touched tags that have been overwritten or are trivial. All decliner nodes have been removed&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;The first tag is documented on the &lt;a href="http://wiki.openstreetmap.org/wiki/Remapping/License_Change_View_on_OSM_Inspector#Per-object_Overrides_with_odbl.3Dclean"&gt;Remapping/License Change View on OSM Inspector&lt;/a&gt; wiki page. I'm not sure if there has been an official nod to obeying this tag in the final license change process but I'm not the only one using it so hopefully it will. And if not, it gives me a simple tag that I can query on to retrieve all ways that need further work. The second tag is just something I decided to use to briefly explain why I think this way is clean.&lt;br /&gt;&lt;br /&gt;And that's it. After the OSM inspector updates its data, this way should show up as a green line instead of a red one.&lt;br /&gt;&lt;br /&gt;This might seem like a lot of work but once I worked out a process I was able to move along pretty quickly. Plus, there were only a few ways where I had to deal with more than a few nodes. I got all of I-70 in eastern Kansas done in a little over an hour and I should be able to finish the rest within another hour or two. And along the way I of course get sucked into improving other things too so it isn't entirely busywork.&lt;br /&gt;&lt;br /&gt;If only we lived in a perfect world where everyone got along and no licenses were required. But then what would the lawyers do?&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/B33uZ8PfrX0" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2012/01/license-change-mapping.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://3.bp.blogspot.com/-zF8DpseBHMg/TwjMkIA5frI/AAAAAAAAAjI/R--1Nxm3uEc/s72-c/1_inspector_dirty_view.png" height="72" width="72" /><thr:total>7</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-805300550809964529</guid><pubDate>Sun, 18 Dec 2011 06:09:00 +0000</pubDate><atom:updated>2012-01-25T23:17:51.487-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">java</category><category domain="http://www.blogger.com/atom/ns#">JOSM</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><title>The Most Interesting Mapper</title><description>&lt;table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td style="text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/-KYLioR5qEKI/TuU3nYQelTI/AAAAAAAAAeY/je7fOM1yqrk/s1600/most_interesting_mapper.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"&gt;&lt;img border="0" src="http://3.bp.blogspot.com/-KYLioR5qEKI/TuU3nYQelTI/AAAAAAAAAeY/je7fOM1yqrk/s1600/most_interesting_mapper.jpg" /&gt;&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td class="tr-caption" style="text-align: center;"&gt;&lt;span style="font-size: large;"&gt;The Most Interesting Mapper in the World&lt;/span&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;In case you live under a rock, this particular meme has its origins in a set of beer commercials that I find kind of amusing. Here is &lt;a href="http://www.youtube.com/watch?v=TzXRPphJouA"&gt;a decent compilation&lt;/a&gt; of them.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&amp;lt;sorry to interrupt this completely awesome post but quick note: This plugin has been turned into a core JOSM feature. See "Update 2" at the bottom of this post&amp;gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;But seriously, the ImproveWayAccuracy plugin is a wonderful tool. Perhaps the prime use case to show it off is a smoothly curving road that was originally either mapped from low resolution imagery or (around here) imported from TIGER data that may have used far too few nodes to make a smooth curve. To fix this by hand is highly annoying. You have to drag nodes around and then grab the + between two existing nodes to create a new one and drag it to where you want it, repeating that as often as needed to achieve a smooth enough curve. Each node requires several precise mouse movements.&amp;nbsp;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;It is much easier to draw a way from scratch where you just move your mouse along the curve and click as needed to form a smooth looking curve. Well, the ImproveWayAccuracy plugin makes this possible to do with existing nodes/ways!&lt;/div&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;For basic usage you can look at the plugin's wiki page:&amp;nbsp;&lt;a href="http://wiki.openstreetmap.org/wiki/JOSM/Plugins/ImproveWayAccuracy"&gt;http://wiki.openstreetmap.org/wiki/JOSM/Plugins/ImproveWayAccuracy&lt;/a&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;The first thing you will want to do after installing the plugin is to change JOSM's "highlight" color to something that has a high contrast to whatever the "selected" color is. By default they are both shades of red which makes the plugin harder to use. I changed mine to a bright green (#66FF33).&amp;nbsp;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Basically you activate the plugin by pressing k and selecting a way. Then if you move the mouse around you will see that the node that is closest to your mouse cursor is highlighted and there are temporary lines going from your cursor to the two nodes next to that one. If you click, the highlighted node will be moved to your cursor's current location. It is kind of hard to describe but pretty easy to get the hang of if you try it.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Holding down CTRL and clicking will add a new node to the way between the two nearest nodes. Holding down SHIFT will lock the node selection to the current node even if you move the mouse so that another node is now closer to the cursor. This can be helpful while trying to fix some weird geometries.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Here is a short clip of me aligning and smoothing out a curve using this plugin. You can tell when I am holding down CTRL because the cursor gets a small + symbol added to it:&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/umUhtXsPgwU?feature=player_embedded' frameborder='0' /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Now... I do have one beef with this plugin. The wiki page states that deleting nodes is not a feature and that you should re-use nodes whenever possible. While I generally agree that as much history as possible should be preserved while editing, it is not always practical. Especially with the TIGER imports that I am frequently editing. Sometimes imported ways have too many nodes along them. This makes editing them more of a pain and the correct way to handle it really is to delete nodes.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Of course the source code for this plugin is available in the OSM SVN repository (&lt;a href="http://trac.openstreetmap.org/browser/applications/editors/josm/plugins/ImproveWayAccuracy"&gt;see here&lt;/a&gt;) so I hacked this feature in myself. Just like the CTRL key adds nodes to the way, the ALT key removes them in my version. However it performs a sanity checks first. If you try to delete a node that is a member of two different ways (a junction node) then the plugin will throw up an error message. The ImproveWayAccuracy plugin is only intended to modify the geometry of a way. If you want to change the topology of the data, you should not be in the IWA mode.&amp;nbsp;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;One gotacha: Some linux window managers use ALT plus a mouse drag to move windows. So my window manager was swallowing my ALT+click event and it never got passed on to JOSM so the plugin just didn't do anything. But I don't use ALT+drag to move my windows around so I just disabled it. I looked at trying to use a different key but the way the existing code was written, it was easiest to just use another meta key and since CTRL and SHIFT were already taken... I guess I could have maybe used CTRL+SHIFT.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Update: Turns out window managers only listen for the left ALT key but JOSM will take the left or right one. So you can use the meta keys on the right side of the keyboard without problems. Oh and CTRL+SHIFT is already used by the plugin although I have never actually used this feature.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;strike&gt;If you want to use my version of the plugin, the easiest way is to first install the "official" plugin through the JOSM plugin options. Then close JOSM and copy my ImproveWayAccuracy.jar file over the one in the JOSM plugin directory (~/.josm/plugins/ on linux or C:\Documents and Settings\&amp;lt;username&amp;gt;\Application Data\JOSM\plugins\ on windows) You can &lt;/strike&gt;&lt;a href="http://ni.kwsn.net/~toby/OSM/ImproveWayAccuracy.jar"&gt;&lt;strike&gt;download the jar file here&lt;/strike&gt;&lt;/a&gt;&lt;strike&gt;. Or if you want to see the code, &lt;/strike&gt;&lt;a href="http://ni.kwsn.net/~toby/OSM/IWAMode.java.patch"&gt;&lt;strike&gt;here is the svn diff&lt;/strike&gt;&lt;/a&gt;&lt;strike&gt; for the IWAMode.java file.&amp;nbsp;(as of revision 27186) &lt;/strike&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;strike&gt;&lt;br /&gt;&lt;/strike&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;strike&gt;I submitted the patch to the original author of the plugin so at some point this might get rolled in to the real plugin. But he said he is very busy right now and probably won't get a chance to look at it any time soon. So I decided to just put it out there in this form for now. The only problem is that letting JOSM update plugins may overwrite the custom .jar file in the future.&lt;/strike&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Update: I &lt;a href="http://josm.openstreetmap.de/ticket/7253"&gt;submitted a patch&lt;/a&gt; to the JOSM trac system. There was some discussion about keys and other checks to do before deleting nodes. But at the end of the day my changes (plus a tweak or two) have been incorporated into the live code and a new version of the plugin has been released. So just update your plugins and you will be able to take advantage of this feature!&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Update 2: Turns out, this code is useful! The JOSM developers saw fit to pull this functionality into JOSM core. They even rearranged some keyboard shortcuts to make room for it on W instead of K. So no more plugin required. Just hit "W" and go! You can see the discussion in &lt;a href="http://josm.openstreetmap.de/ticket/7184"&gt;ticket #7184&lt;/a&gt; over on the JOSM trac.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Enjoy.&lt;/div&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/exdpyb-w4AU" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/12/most-interesting-mapper.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://3.bp.blogspot.com/-KYLioR5qEKI/TuU3nYQelTI/AAAAAAAAAeY/je7fOM1yqrk/s72-c/most_interesting_mapper.jpg" height="72" width="72" /><thr:total>0</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-4560975720766338525</guid><pubDate>Tue, 06 Dec 2011 03:50:00 +0000</pubDate><atom:updated>2011-12-17T14:31:19.790-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">hardware</category><category domain="http://www.blogger.com/atom/ns#">Linux</category><category domain="http://www.blogger.com/atom/ns#">osmosis</category><category domain="http://www.blogger.com/atom/ns#">postgis</category><title>Random computer upgrade notes</title><description>I recently did some reshuffling of the computers in my home. I built a brand new gaming system from parts I bought on newegg. Intel i5 Sandybridge, 16 GB of memory and an nvidia video card that not only has a 120mm fan, but has &lt;b&gt;two&lt;/b&gt; of them. The insanity. Then I used the parts from the old gaming system (first generation i5, 12 GB, ATI graphics) to give my ancient linux box a major upgrade.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Of Linux and Distros&lt;/span&gt;&lt;br /&gt;The Linux box was so old (AMD Athlon x2 4400+ with 3 GB) that it was becoming a pain to use for any kind of desktop applications. I ended up neglecting it and spending most of my time on my windows box (I know! The horror!) with multiple putty sessions open. Also, it was running Gentoo Linux. In my local Linux user group, we have determined that people who are over 30 don't use Gentoo. It is a worthy distro and has many strengths. I have been using it for over 10 years on some system or another and because of its minimal nature, I learned a lot about Linux that I would otherwise be oblivious to. But it can be a real pain, especially to maintain for desktop use. I was getting tired of every other &lt;i&gt;emerge --sync&lt;/i&gt; generating package conflicts, lengthy config file merges and the occasional system toolchain breakage that you &lt;b&gt;had&lt;/b&gt; to resolve before rebooting (or loosing power!)&amp;nbsp; otherwise the system would completely eat itself. So, being a distinguished 31 years of age now, I decided it was time to drop Gentoo. So long and thanks for all the fish.&lt;br /&gt;&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;So what to replace it with? Well I've been an Ubuntu user for a while as well. But they really worked hard to tick me off with the whole Unity business. It's not that I am completely opposed to new things. But I think Ubuntu messed up pretty good in this transition. First, they rolled out Unity to the netbook edition a year ago, before it was ready. It crashed the first time I logged in. Then it continued to be buggy. Also, I really, really don't like the mac-style combined launch/task bar that is central to Unity. I want a taskbar that displays currently running programs. Not some auto-hiding, moving-with-your-mouse contraption that shows a whole host of programs, some of which may be running and some of which may not. The running ones have a dot on them or something? Whatver. Just give me a taskbar. And now with Ubuntu 11.10 the option to go back to regular old gnome 2 is mostly gone. It's either Unity or Gnome3 with gnome-shell. On my netbook I now use gnome-shell and it works well enough for such a limited device with a small screen. But I'm not really a fan of some of the unilateral decisions they made when it comes to running it on a dual monitor desktop.&lt;br /&gt;&lt;br /&gt;Enter &lt;a href="http://blog.linuxmint.com/?p=1851"&gt;this blog post&lt;/a&gt; from Linux Mint. The condensed version: "Gnome 2 is dead so we &lt;b&gt;have&lt;/b&gt; to move to something newer. But we'll ease the transition by giving you similar functionality to what you've been used to for the last 15 years and let you choose how much new stuff you want." Perfect. So right now I'm typing this on a Linux Mint 12 release candidate install. So far I'm pretty happy. I even find myself using some of the new gnome-shell features from time to time. But I still have my taskbar, traditional alt-tab and always-visible notification area. I'm still trying to hammer out a few bugs I'm encountering. One thing that annoyed me greatly was the default workspace switching for dual monitors. Only my left monitor switched. The right one always displayed the same workspace. Turns out it is just a dumb default setting made by a gnome developer. Unfortunately they don't provide an easy checkbox to change this in the settings. Luckily I found &lt;a href="http://gregcor.com/2011/05/07/fix-dual-monitors-in-gnome-3-aka-my-workspaces-are-broken/"&gt;this blog post&lt;/a&gt; that fixed it for me in 30 seconds. &lt;br /&gt;&lt;br /&gt;Gnome-shell does seem to be a little bit crashy but I'm I'm going to blame that on the ATI video card I have in this thing. I bought this card several years ago right after ATI announced that they would open up their hardware specs to allow people to write decent open source drivers for ATI cards. It seems that hasn't happened as quickly as I had hoped. Notice I'm back to nvidia for my new gaming card. Luckily when gnome-shell crashes it doesn't take down the whole system and I can usually hit ATL+F2 to bring up a run dialog (even though it doesn't appear on the screen) and execute "r" which tells gnome-shell to restart itself. Otherwise, I can also switch to a virtual terminal run a &lt;i&gt;gnome-shell --replace &lt;/i&gt;command to make everything happy again.&lt;br /&gt;&lt;br /&gt;&lt;strong&gt;Update&lt;/strong&gt;: I think I have tracked down the gnome 3 crashiness. It is in the Mint media player plugin. &lt;a href="https://bugs.launchpad.net/linuxmint/+bug/892316"&gt;This bug on launchpad&lt;/a&gt; indicates that there is a fix which should be rolled out to the masses shortly. The workaround until then is to just disable the extension.&lt;br /&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Of Zabbix and Migrating Old Data&lt;/span&gt;&lt;br /&gt;I have been using &lt;a href="http://www.zabbix.com/"&gt;Zabbix&lt;/a&gt; to monitor and track system metrics for a while. Not that I have anything super critical running on my boxes here at home but sometimes it yields some interesting data and I'm kind of a sucker for graphs.&lt;br /&gt;&lt;br /&gt;I installed the latest zabbix on my new machine but I didn't really want to migrate my whole database from the old one because there was some cruft in it and several old hosts that haven't existed for a while. But there are a few monitored items that I wanted to migrate. In particular, the data that generates my &lt;a href="http://ni.kwsn.net/~toby/OSM/license_count.html"&gt;license acceptance graphs&lt;/a&gt;. Zabbix does have an export feature that will export a given item but it only exports the definition, not the data. Fortunately it keeps all its data in a pretty plain mysql database and I'm kind of handy with databases so I managed to transition the data as follows:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;&lt;div&gt;Create the item in the new Zabbix install. I did an export from the old system and imported it into the new one to create the item. In this case I also had to copy a shell script that zabbix calls to update the counts and then configure the new zabbix agent to poll it.&lt;/div&gt;&lt;/li&gt;&lt;li&gt;Export the data from the old mysql database. First you need the item ID. This can be found by looking at the item you are exporting in the zabbix web web UI and examining the URL. You are looking for an &lt;i&gt;itemid=????&lt;/i&gt; parameter. Then plug that in to the following query which will give you a CSV file. Note that the mysql user you connect as must have the &lt;i&gt;file&lt;/i&gt; permission - I just logged in as the mysql root user.&lt;br /&gt;&lt;pre class="source-code"&gt;select * &lt;br /&gt;into outfile '/tmp/history_uint_agreed.csv'&lt;br /&gt;fields terminated by ','&lt;br /&gt;lines terminated by '\n'&lt;br /&gt;from history_uint &lt;br /&gt;where itemid = 19913;&lt;br /&gt;&lt;/pre&gt;&lt;/li&gt;&lt;li&gt;Now, find the item ID in the target zabbix install. Then use sed or vim or something to do a string substitution. I used vim with the following command: &lt;br /&gt;&lt;pre class="source-code"&gt;:%s/19913/22304/&lt;/pre&gt;&lt;/li&gt;&lt;li&gt;Import the data into the new database:&lt;br /&gt;&lt;pre class="source-code"&gt;load data infile '/tmp/history_uint_agreed.csv'&lt;br /&gt;into table history_uint&lt;br /&gt;fields terminated by ','&lt;br /&gt;lines terminated by '\n'&lt;/pre&gt;&lt;/li&gt;&lt;li&gt;Finally, activate the item in the new zabbix so that it starts updating with new data.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;&lt;span style="font-size: large;"&gt;Of PostGIS, osmosis and the jxapi&lt;/span&gt;&lt;br /&gt;Since I was using PostgreSQL 8.4 on my old system and 9.1 is out now, I ended up just dumping my existing OSM database that I &lt;a href="http://ksmapper.blogspot.com/2011/04/parse-planet-into-pgsnapshot-database.html"&gt;blogged about in the past&lt;/a&gt; and setting up a brand new one from scratch. I set up the database right before I left for Thanksgiving. I started the planet file torrent and then drove to Nebraska where my only access to the internet was tethering through my phone which only gets EDGE data service up there. Plus, I was of course being (a little) social with family so I didn't have as much time to tinker with it. For a few tweaks dealing with   PostgreSQL 9.1 vs 8.4 I followed &lt;a href="http://www.paulnorman.ca/blog/2011/11/loading-a-pgsnapshot-schema-with-a-planet-take-2/"&gt;Paul Norman's post&lt;/a&gt; on the subject. But I didn't want to do the dump files with osmosis for a couple of reasons so I went with the &lt;i&gt;--write-pgsql&lt;/i&gt; task instead of the &lt;i&gt;--write-pgsql-dump&lt;/i&gt; one so that the data would get read out of the planet file and put straight into the database. Well almost. It still creates some temporary files along the way but you don't have to mess with loading them into the database after it finishes. Unfortunately the planet file has grown too much since I last did this and the nodes no longer fit in my 12 GB of memory so I had to use the TempFile option for node location storage. It actually didn't impact the speed nearly as much as I thought it would. I'm guessing Linux's aggressive caching probably paid off and a good chunk of the temporary node file ended up in the 12 GB of memory the system has. My final osmosis command line looked like this:&lt;br /&gt;&lt;pre class="source-code"&gt;bzcat planet-111123.osm.bz2 | &lt;br /&gt;osmosis --fast-read-xml file=- &lt;br /&gt;--log-progress interval=30 &lt;br /&gt;--write-pgsql nodeLocationStoreType=TempFile &lt;br /&gt;              host=localhost database=xapi user=xapi password=xapi &lt;/pre&gt;&lt;br /&gt;Osmosis ran for 53 hours but when it was done, the database was ready to go.&lt;br /&gt;&lt;br /&gt;For fun, I ran a few &lt;a href="http://wiki.openstreetmap.org/wiki/User:Pnorman/jxapi"&gt;benchmark queries&lt;/a&gt; that Paul came up with a while ago against my old   PostgreSQL 8.4 and then again against the 9.1 database on new hardware. Before running each query I restarted   PostgreSQL and flushed the disk cache using the command  &lt;br /&gt;&lt;pre class="source-code"&gt;sync; echo 3 &amp;gt; /proc/sys/vm/drop_caches&lt;/pre&gt;to make sure I was getting the raw hardware speed, not assisted by the various caches that the operating system and the database both have. Vital stats of the old and new systems:&lt;br /&gt;&lt;b&gt;Old:&lt;/b&gt; AMD Athlon x2 4400+ (2.2 GHz) with 3 GB of memory&lt;br /&gt;&lt;b&gt;New:&lt;/b&gt; Intel Core i5 (2.67 GHz) with 12 GB of memory&lt;br /&gt;The I/O system remained pretty much the same: software RAID 5 of four 7,200 RPM 1TB drives with LVM on top of it.&lt;br /&gt;&lt;br /&gt;&lt;table border="1"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;th&gt;Query&lt;/th&gt;&lt;th&gt;Old system&lt;/th&gt;&lt;th&gt;New system&lt;/th&gt;&lt;th&gt;speedup&lt;/th&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;map?bbox=11.54,48.14,11.543,48.145&lt;/td&gt;    &lt;td&gt;61 sec&lt;/td&gt;    &lt;td&gt;46 sec&lt;/td&gt;    &lt;td&gt;25%&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;*[bbox=11.54,48.14,11.543,48.145]&lt;/td&gt;    &lt;td&gt;56 sec&lt;/td&gt;    &lt;td&gt;42 sec&lt;/td&gt;    &lt;td&gt;25%&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;*[amenity=library][bbox=-0.57,51.24,0.31,51.75]&lt;/td&gt;    &lt;td&gt;172 sec&lt;/td&gt;    &lt;td&gt;153 sec&lt;/td&gt;    &lt;td&gt;11%&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;node[power=generator]&lt;/td&gt;    &lt;td&gt;55 sec&lt;/td&gt;    &lt;td&gt;85 sec&lt;/td&gt;    &lt;td&gt;-55%&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;*[amenity=parking][capacity:disabled=*][bbox=-0.57,51.24,0.31,51.75]&lt;/td&gt;    &lt;td&gt;192 sec&lt;/td&gt;    &lt;td&gt;177 sec&lt;/td&gt;    &lt;td&gt;8%&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;*[waterway=sluice_gate]&lt;/td&gt;    &lt;td&gt;40 sec&lt;/td&gt;    &lt;td&gt;9 sec&lt;/td&gt;    &lt;td&gt;78%&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;*[bdad6d21a=1378b02bc]&lt;/td&gt;    &lt;td&gt;6.3 sec&lt;/td&gt;    &lt;td&gt;3.3 sec&lt;/td&gt;    &lt;td&gt;47%&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;*[power=generator]&lt;/td&gt;    &lt;td&gt;405 sec&lt;/td&gt;    &lt;td&gt;367 sec&lt;/td&gt;    &lt;td&gt;9%&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;map?bbox=11.54,48.14,11.543,48.145&lt;/td&gt;    &lt;td&gt;45 sec&lt;/td&gt;    &lt;td&gt;43 sec&lt;/td&gt;    &lt;td&gt;4%&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;map?bbox=-124,49,-123,50&lt;/td&gt;    &lt;td&gt;272 sec&lt;/td&gt;    &lt;td&gt;227 sec&lt;/td&gt;    &lt;td&gt;17%&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;br /&gt;Several noticeably faster ones but nothing too earth shattering. Then there is that one that is 55% slower. No clue what's up with that. Of course this is all pretty anecdotal. So much changed between the two runs that it is hard to pinpoint whether the speedup came from the faster CPU, more memory, improvements in &amp;nbsp; PostgreSQL or changes in the data. Probably a little from all of the above.&lt;br /&gt;&lt;br /&gt;I leave you with a picture of the ridiculous video card fans that I mentioned in the first paragraph. Notice at the upper right corner of the card that it also requires two power connections. I hope it will help me achieve some epic frags at the upcoming semiannual LAN party I have with my college friends. It has already propelled my &lt;a href="http://boinc.berkeley.edu/"&gt;BOINC&lt;/a&gt; score to new heights.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-aIDpanlgrlo/TtxRzGoB0aI/AAAAAAAAAeM/0KBMS_E6Apg/s1600/SAM_1247.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="470" src="http://1.bp.blogspot.com/-aIDpanlgrlo/TtxRzGoB0aI/AAAAAAAAAeM/0KBMS_E6Apg/s640/SAM_1247.JPG" width="625" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/h5_12PlTmKw" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/12/random-computer-upgrade-notes.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://1.bp.blogspot.com/-aIDpanlgrlo/TtxRzGoB0aI/AAAAAAAAAeM/0KBMS_E6Apg/s72-c/SAM_1247.JPG" height="72" width="72" /><thr:total>0</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-4551257701545339899</guid><pubDate>Sun, 21 Aug 2011 03:44:00 +0000</pubDate><atom:updated>2011-08-22T21:30:55.149-05:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">Maperitive</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">rendering</category><category domain="http://www.blogger.com/atom/ns#">tiles</category><title>Simple OpenStreetMap tile rendering with Maperitive</title><description>Lately there have been a lot of people popping into the OSM IRC channel (&lt;a href="http://irc.openstreetmap.org/"&gt;#osm on oftc.net&lt;/a&gt;) because they have been getting the "you've been bad" tile from the OSM tile server:&lt;br /&gt;
   &lt;br /&gt;
   &lt;div class="separator" style="clear: both; text-align: center;"&gt;
   &lt;a href="http://wiki.openstreetmap.org/w/images/d/df/Tile_Abuse.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" src="http://wiki.openstreetmap.org/w/images/d/df/Tile_Abuse.png" /&gt;&lt;/a&gt;&lt;/div&gt;
   &lt;br /&gt;
   The reason is that they are using tile scraping applications that violate the OSM &lt;a href="http://wiki.openstreetmap.org/wiki/Tile_usage_policy"&gt;tile usage policy&lt;/a&gt;. Typically people just want to download tiles for their city or a vacation destination or travel route in order to pre-load them on a mobile device for offline use.&lt;br /&gt;
   &lt;br /&gt;
   This is of course a perfect use case for OSM. The problem is that OSM is a project run by volunteers and with limited server capacity. Our primary product is geographic data, not map tiles. So when these tile scraping programs start hitting high zoom levels, it can cause serious problems for the tile server. The solution is to download OSM data instead of rendered tiles. Then you can render your own tiles or even use offline vector rendering.&lt;br /&gt;
   &lt;br /&gt;
   One tool to make your own tiles is Maperitive. It is pretty easy to use and by default generates tiles that are fairly similar in style to the default mapnik style you see on osm.org although it is very customizable. It is a .NET application although it claims to work in Linux/OSX with mono. Also, while it is free to download, it is not open source which is kind of rare for the OSM world. These two things may cause some people to have an allergic reaction (I'm looking at you, Serge) but if you're still reading, here is a quick lesson on how to use it to render your own tiles. If you keep reading, you should be able to start rendering your own tiles in about 10 minutes. There is a tutorial video and some detailed documentation on the Maperitive website but I don't see a basic text based tutorial that walks you through the common "render some tiles for this area" use case.&lt;br /&gt;
   &lt;br /&gt;
   &lt;span class="Apple-style-span" style="font-size: large;"&gt;&lt;b&gt;Getting Started&lt;/b&gt;&lt;/span&gt;&lt;br /&gt;
   Installation is pretty straightforward. Just head over to&amp;nbsp;&lt;a href="http://maperitive.net/"&gt;http://maperitive.net/&lt;/a&gt; and find the "Download" link. Grab the latest .zip file (Maperitive-1228.zip right now) and download it. Once the download finishes, just unzip the Maperitive folder inside of the zip file somewhere and you're done. Open the Maperitive.exe file to fire it up.&lt;br /&gt;
   &lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;
   Here is a screen shot of my Maperitive with some OSM data already loaded in. It should all make sense by the end of this post.&lt;br /&gt;
   &lt;div class="separator" style="clear: both; text-align: center;"&gt;
   &lt;a href="http://2.bp.blogspot.com/-Eh2NEY6j3Mw/TlAarp7qc0I/AAAAAAAAAaM/BlfupgaW-eU/s1600/Maperitive_screenshot.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="640" src="http://2.bp.blogspot.com/-Eh2NEY6j3Mw/TlAarp7qc0I/AAAAAAAAAaM/BlfupgaW-eU/s640/Maperitive_screenshot.png" width="600" /&gt;&lt;/a&gt;&lt;/div&gt;
   &lt;br /&gt;
   The first time you fire up Maperitive, you will notice there is already a map on the screen! By default it just starts loading map tiles from osm.org so you can get your bearings and navigate to an area of interest. Next, look at the bottom of the window. On the left is a text area (labeled "Commander" in the tab above it) where status messages and command outputs are displayed. Below that is a command line. This is a graphical program but a lot of the advanced functionality is done via this command line. A note about typing commands: Maperitive automatically completes commands for you which is nice... but it does it in a rather annoying way. As soon as you have typed enough for it to identify a unique command, it immediately fills in the rest of the command. I am much more used to a user-driven tab completion style as seen in most linux shells so the automatic completion in the middle of typing a command means my fingers keep typing the command after Maperitive auto-completes it for me and I have to backspace to erase extra characters.&lt;br /&gt;
   &lt;br /&gt;
   On the right side is a "Sources" section that lists what map sources are currently being displayed in the main window. You can turn each one on and off by clicking the star by it. In the menus are various options. When you select one it will execute a command that you can see being executed in the Commander pane at the bottom. For example, find your home city and then go to the View menu and select "Set Home Location." You will see the text "&lt;span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"&gt;&amp;gt; set-home&lt;/span&gt;" appear in the Commander. You can do the same thing by typing "&lt;span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"&gt;set-home&lt;/span&gt;" in the command prompt and hitting enter. Now you can use the "Go Home" option to snap back to your "home" location from anywhere.&lt;br /&gt;
   &lt;br /&gt;
   &lt;b&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;Selecting an area&lt;/span&gt;&lt;/b&gt;&lt;br /&gt;
   Next, you must select an area of the world to render. You can navigate with your mouse just like on osm.org. Holding down shift while dragging with the mouse will draw a box. After you release the mouse button the view will zoom to that box. I'm not really sure what the limits of Maperitive are as far as volume of data. Note that the way it downloads data from the XAPI puts a limit of 10 square degrees on the download area. But this translates to a box of over 100 kilometers on a side and I'm guessing Maperitive will have a hard time with that much data anyway. I tried my county which is about 30x60 kilometers (18x37 miles). It contains about 26 MB of data in .osm format. I was able to render tiles on a 2.8 GHz Core i5 system down to zoom level 18 in about an hour with a memory footprint of under 300 MB. This should be a big enough example to cover many use cases so I will be using this area in my examples.&lt;br /&gt;
   &lt;br /&gt;
   By default, Maperitive renders everything you see in the map on your screen. So if you are happy with that, you can go to the next step. To customize the area to something more specific than the shape of the window, read on.&lt;br /&gt;
   &lt;br /&gt;
   You can set a more specific bounding box for all your operations by using the command prompt. One easy way to find a bounding box is to use the &lt;a href="http://openstreetmap.us/uixapi/xapi.html"&gt;XAPI frontend&lt;/a&gt;. Browse the map and then hold shift while dragging the mouse to draw a box. After drawing the box, check the "Search by Area" checkbox above the map. Now look down at the bottom of the page and copy the latitude/longitude coordinates after "bbox=" - in my case:&amp;nbsp;-96.964,39.025,-96.547,39.573 (do NOT copy the ] at the end!)&lt;br /&gt;
   &lt;br /&gt;
   Now go back to Maperitive and use the bounds-set command:&lt;br /&gt;
   &lt;pre class="source-code"&gt;bounds-set -96.964,39.025,-96.547,39.573&lt;/pre&gt;
   Maperitive will reply with "Locking the bounds to ..." and will draw a thick dotted line on the map to show the bounding box you have selected. It is clearly visible in the screenshot up above.&lt;br /&gt;
   &lt;br /&gt;
   &lt;b&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;Downloading data&lt;/span&gt;&lt;/b&gt;&lt;br /&gt;
   Now that you have your area of operations defined, it is time to download data. This should be as simple as selecting the "Download OSM Data" option in the Map menu. This will hit the java XAPI service hosted by MapQuest Open and download the data within your specified bounds. One problem with Maperitive is that its network timeout is very short. So if you get an error that says "The operation has timed out" then the XAPI server just responded too slowly. You can try again but you may need to reduce the size of your bounding box or try one of the other XAPI servers listed on the &lt;a href="http://wiki.openstreetmap.org/wiki/Xapi#Servers"&gt;XAPI wiki page&lt;/a&gt;. To specify the server you add an "xapi-url=" argument to the command. I hit my own XAPI server that I have set up here at home. To do this I issued this command:&lt;br /&gt;
   &lt;pre class="source-code"&gt;download-osm xapi-url=http://localhost:8080/xapi/api/0.6/*&lt;/pre&gt;
   &lt;br /&gt;
   Alternatively&amp;nbsp;you can download the data using another method. (in JOSM or the "Export" tab on osm.org to name two) &amp;nbsp;Then you can go to the File menu and use the "Open Map Sources" option to load the file.&lt;br /&gt;
   &lt;br /&gt;
   Once the data is loaded, turn off the Web map display by clicking the star over in the Map Sources tab. This will remove the tiles downloaded from osm.org from the display and show only what Maperitive has rendered from the downloaded data. It will be fairly similar but not identical.&lt;br /&gt;
   &lt;br /&gt;
   &lt;b&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;Generating the tiles&lt;/span&gt;&lt;/b&gt;&lt;br /&gt;
   Almost done! Now all you have to do is generate tiles. First, make sure that the web map layer is disabled or deleted in the Map Sources tab. Otherwise Maperitive uses map tiles from osm.org as a background layer. This causes it to scrape tiles which is what we are trying to avoid in the first place! There is a command in the Tools menu that will generate tiles. By default it uses how far you are zoomed in on the map window and renders 5 zoom levels of tiles based on that. If you want to generate a wider range of zoom levels, go back to the command line. This will render your bounding area from zoom level 8 down to 18:&lt;br /&gt;
   &lt;pre class="source-code"&gt;generate-tiles minzoom=8 maxzoom=18&lt;/pre&gt;
   &lt;br /&gt;
   The tiles will be generated in a directory named "Tiles" inside of the Maperitive directory, wherever you unzipped it to. If you are going all the way to zoom level 17 and 18, it could take a while to render so go outside and collect some map data or something.&lt;br /&gt;
   &lt;br /&gt;
   When the process finishes, you will have a standard tile directory tree just like you would have had by scraping tiles from osm.org with Mobile Atlass Creator or whatever other scraping software people have come up with. Load it onto a phone for offline viewing or throw it up on a web server using OpenLayers... whatever your heart desires.&lt;br /&gt;
   &lt;br /&gt;
   &lt;b&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;Easy updating&lt;/span&gt;&lt;/b&gt;&lt;br /&gt;
   Maperitive supports making simple scripts. So if you intend on updating your tiles periodically you might want to make a script so you can just fire and forget. Scripts are run from the File menu. Here is one that performs all the steps listed above for my county but only renders tiles between level 8 and 15. It takes about 90 seconds to execute including download time from the XAPI which can vary depending on current server load and size of area. You can copy this script, change the numbers in the bounds-set command and the zoom levels you want to render and then run it to generate tiles for your desired area. Maperitive expects script files to have a .mscript extension.&lt;br /&gt;
   &lt;br /&gt;
   &lt;pre class="source-code"&gt;clear-map
  &lt;br /&gt;use-ruleset alias=default
  &lt;br /&gt;bounds-set -96.964,39.025,-96.547,39.573
  &lt;br /&gt;download-osm
  &lt;br /&gt;generate-tiles minzoom=8 maxzoom=15
  &lt;br /&gt;&lt;/pre&gt;
   &lt;br /&gt;
   Maperitive can do a lot of other things like adding hillshading or relief contours (elevation information) to the map.There are also several styles that come with Maperitive besides the mapnik look-alike one. You can poke around the menus to see some of this functionality or execute the &lt;span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"&gt;help-commands&lt;/span&gt; command on the command line to get a list of all available commands. More information about a specific command can be seen by using help-commands again and adding the specific command you want more information about. Example: &lt;span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"&gt;help-commands generate-tiles&lt;/span&gt; will give you more details about the command that renders tiles.&lt;br /&gt;
   &lt;br /&gt;
   I think I am going to do another post about Maperitive that will go in to more details about customizing rendering rules for a specific purpose. Until then, happy rendering!&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/0C-VnOg6gF8" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/08/simple-openstreetmap-tile-rendering.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://2.bp.blogspot.com/-Eh2NEY6j3Mw/TlAarp7qc0I/AAAAAAAAAaM/BlfupgaW-eU/s72-c/Maperitive_screenshot.png" height="72" width="72" /><thr:total>11</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-5515794768347354757</guid><pubDate>Sat, 25 Jun 2011 22:04:00 +0000</pubDate><atom:updated>2011-09-10T18:27:22.328-05:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">data integrity</category><category domain="http://www.blogger.com/atom/ns#">imports</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><title>Cleanup on the Map Aisle</title><description>Spills happen. A sticky honey jar in the grocery store. A glass of orange juice at breakfast. Oil spills in the Gulf of Mexico. Node spills in OpenStreetMap. Some can be cleaned up with a mop and some water. Others require a more technical solution. What is a "node spill" you ask? It's when nodes get uploaded to OSM that don't have any tags on them and aren't part of a way. These tagless, unconnected nodes add no useful information to the map and are just dead weight in the database. Where do they come from? There are at least a couple of common sources of node spills. One is &lt;a href="http://trac.openstreetmap.org/ticket/2700"&gt;editor bugs&lt;/a&gt; and simple user error. These are generally pretty small spills of 10s or maybe 100s of nodes. The bigger problem is imports.&lt;br /&gt;&lt;br /&gt;The topic of imports deserves its own post. For now I'll just say that badly performed imports and insufficient checking after imports can lead to 10s of thousands of empty nodes. The basic problem is that nodes get uploaded first and don't become part of a way until the way is uploaded later. So if something goes wrong with the way upload, the nodes end up in an orphaned state.&lt;br /&gt;&lt;br /&gt;I have come across a few of these nodes before but what really caught my attention was a failed NHD (National Hydrography Dataset) import in Oklahoma. It happened to poke up into Kansas a little bit where I noticed it next to a state highway I was editing. After a &lt;a href="http://lists.openstreetmap.org/pipermail/talk-us/2011-May/thread.html#5782"&gt;lengthy thread&lt;/a&gt; on the talk-us mailing list I eventually found a good way to detect these nodes. Since then I have made my way across the US and some of Canada deleting these useless nodes from the database. As detailed in &lt;a href="http://lists.openstreetmap.org/pipermail/talk-us/2011-May/005824.html"&gt;this message&lt;/a&gt;&amp;nbsp;it goes something like this:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Use the XAPI to perform a query of the form /node[not(way)][bbox=a,b,c,d]&lt;/li&gt;&lt;li&gt;Open the result of this query in JOSM and apply a filter to hide all nodes with tags&lt;/li&gt;&lt;li&gt;Perform some checks to make sure there really is no useful data&lt;/li&gt;&lt;li&gt;Delete nodes and upload&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;b&gt;XAPI Query&lt;/b&gt;&lt;br /&gt;The [not(way)] xapi predicate is documented on the OSM wiki &lt;a href="http://wiki.openstreetmap.org/wiki/Xapi#Child_Element_Predicates"&gt;XAPI page&lt;/a&gt;. In addition you need to add a bounding box to the query. An easy way to come up with a bounding box is to use the uixapi page found at &amp;nbsp;&lt;a href="http://openstreetmap.us/uixapi/xapi.html"&gt;http://openstreetmap.us/uixapi/xapi.html&lt;/a&gt;. Hold down shift and draw a box around the area you wish to query. Then tick the "Search by Area" checkbox above the map. &amp;nbsp;The URL at the bottom of the page will change to include a [bbox=...] portion. Note that the bbox won't update automatically if you draw another box. You need to uncheck and recheck the box for it to update.&lt;br /&gt;&lt;br /&gt;If you are using one of the public XAPI services then the size of the bounding box will be limited to 10 square degrees. Since I am running these queries against my own XAPI installation, I removed that limitation so I've just been doing entire states at a time. Depending on how much data is in a given area, the queries take very different times to return. All of Alaska took under 10 minutes while some of the small states on the east coast took about an hour.&lt;br /&gt;&lt;br /&gt;I started out downloading the data straight into JOSM using the "Open Location..." feature in the File menu but for particularly dense areas this caused some strange problems so I switched to using wget to download the data to a file and then opened the file in JOSM. I learned something about wget: it has a default timeout of 15 minutes. So if the query takes longer than 15 minutes, wget will give up and try again which will result in two identical queries running simultaneously and slowing things down even more. So remember to use the --timeout=0 option to disable the timeout in wget. Here is a sample wget command for Kansas:&lt;br /&gt;&lt;pre class="source-code"&gt;wget --timeout=0 -O KS.osm "http://localhost:8080/xapi/api/0.6/node[not(way)][bbox=-102.12,36.87,-94.46,40.11]"&lt;/pre&gt;&lt;br /&gt;&lt;b&gt;Loading into JOSM&lt;/b&gt;&lt;br /&gt;The previous step gives you a .osm file that contains all nodes that are not part of a way in the area you queried. Obviously this includes all POIs that contain useful data. The file can be rather large and may require that you give JOSM a couple of GB of memory to use. Here is a screen shot of Maine:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-ye1QAVc6Y8M/TgYzFw3Kn8I/AAAAAAAAAWE/FXvLTyyudYo/s1600/Maine_all_nodes.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="490" src="http://4.bp.blogspot.com/-ye1QAVc6Y8M/TgYzFw3Kn8I/AAAAAAAAAWE/FXvLTyyudYo/s640/Maine_all_nodes.png" width="625" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Now let's filter out all nodes with useful information. In this case you can see a filter in the bottom right of the screen of the form "-untagged" which hides all nodes with tags except for nodes that &lt;b&gt;only&lt;/b&gt; have the following tags: attribution, created_by, source, fixme and note. (I won't guarantee that this is a complete list) While nodes with only these tags aren't really useful map data, you do need to be careful because nodes with a fixme or a note tag could be useful information to other mappers. The other stricter JOSM filter that I used sometimes was "tags:1-999" which filtered out all nodes with &lt;b&gt;any&lt;/b&gt;&amp;nbsp;tags on them. After filtering, things are a little more&amp;nbsp;manageable:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-iLCeTe7BKHc/TgY1Dtzo1yI/AAAAAAAAAWI/ULWkEq53iKY/s1600/Maine_all_empty.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="490" src="http://2.bp.blogspot.com/-iLCeTe7BKHc/TgY1Dtzo1yI/AAAAAAAAAWI/ULWkEq53iKY/s640/Maine_all_empty.png" width="625" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Now for a bit of data analysis. Over on the right side of the window you can see I have the "Authors" panel open which lists who the last person was to touch the currently selected objects. The most frequent user in the US was woodpeck_fixbot. After some digging I found out why. The Census Bureau TIGER data was originally imported with several tags such as source and an upload_uuid tag on every single node. After the import, it was determined that these tags on the 10s of millions of nodes in the TIGER data were&amp;nbsp;unnecessary and bloated the database and the planet file so much that it was decided to remove them. This removal was done by woodpeck_fixbot. So all these nodes are originally from the TIGER import and I deleted these without a second thought since they have been sitting in the database for several years.&lt;br /&gt;&lt;br /&gt;However for other cases I did more checking. I investigated the changesets that left a lot of empty nodes to determine what was being done in these uploads and whether the user noticed the failure. Sometimes it was obvious that the failure was noticed and the data was re-uploaded so that the empty nodes were actually duplicates of data that was successfully uploaded later. In this case deletion was also obviously&amp;nbsp;warranted. One way to check this is to just zoom in on some of the nodes with the OSM tiles in the background:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-tbn3wqVvH_I/TgY4WypegqI/AAAAAAAAAWM/ISrPPD_2tEs/s1600/Maine_NHD_Verify.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="490" src="http://4.bp.blogspot.com/-tbn3wqVvH_I/TgY4WypegqI/AAAAAAAAAWM/ISrPPD_2tEs/s640/Maine_NHD_Verify.png" width="625" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Obviously these nodes were intended to be part of the border of this lake which exists. In this case it was doubly confirmed by looking at the changeset comment where it stated that this was an upload of NHD data.&lt;br /&gt;&lt;br /&gt;Unfortunately sometimes the uploader did not notice the error so no re-uploading took place and there were no features present. In some of these cases I contacted the user to make sure they knew about the error and see if they had a way of recovering from it. A surprising number of them knew there was a problem during the upload process and thought they had fixed it but they didn't get everything. Just about everyone I contacted said to go ahead and delete the nodes.&lt;br /&gt;&lt;br /&gt;The other thing you really have to watch out for is currently running imports. I hit one of those in St Louis. Someone was actively importing NHD data while I was cleaning up empty nodes in the area. If I hadn't noticed this and blindly deleted the nodes it would have caused their upload to fail horribly and led to all kinds of headaches including even more empty nodes. One good way to do this is to select all the nodes you are thinking about deleting and then using the JOSM search feature with the string "timestamp:2011" to find nodes last touched in 2011 within your current selection. These can then be more thoroughly investigated.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;Uploading&lt;/b&gt;&lt;br /&gt;After sufficient checking had been performed, I deleted the nodes and uploaded them with a helpful changeset comment.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-Sgb2moqKD-c/TgY-FObHmWI/AAAAAAAAAWQ/uFZ4lssrbno/s1600/Maine_upload.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="440" src="http://2.bp.blogspot.com/-Sgb2moqKD-c/TgY-FObHmWI/AAAAAAAAAWQ/uFZ4lssrbno/s400/Maine_upload.png" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;There is one final kink that popped up a couple of times during upload. While all these nodes are not used in a way, a few of them were members of a relation. Most of these cases were clear errors. For example some empty node in the middle of Nebraska was a member of an administrative boundary in France. No clue how that happened. I'm assuming some editor or import bug. There were several in California that were members of turn restriction relations with a "location_hint" role. Not sure what that's supposed to mean but of course I left them alone. In these cases the API will return an error saying that the node is still in use by a relation and you can't delete it. To resolve the conflict you can take the relation ID from that error message and download the relation using JOSM's "Download Object" feature. It will complain that there is a conflict which can then be resolved in the conflict editor. Then the upload can continue.&lt;br /&gt;&lt;br /&gt;Here is a shot of &lt;a href="http://wiki.openstreetmap.org/wiki/LiveMapViewer"&gt;LiveMapViewer&lt;/a&gt;&amp;nbsp;&amp;nbsp;after a day of work near the Great Lakes. Obviously this includes the work of other mappers as well but most of that red is the result of my deletions.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-1SAEJjh6P50/TgZPdQ2OIYI/AAAAAAAAAWU/d69v62hESHY/s1600/node_cleanup.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="480" src="http://1.bp.blogspot.com/-1SAEJjh6P50/TgZPdQ2OIYI/AAAAAAAAAWU/d69v62hESHY/s640/node_cleanup.png" width="625" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;br /&gt;&lt;b&gt;Conclusion&lt;/b&gt;&lt;br /&gt;As stated above, I have covered the entire US with this cleanup. In most places I was pretty conservative in what I deleted in that I only deleted those nodes that I positively identified as part of an import or otherwise determined what had happened and why. Other nodes that I wasn't sure about I left in place. I have yet to receive a single complaint about any of these edits so I think I did ok. Despite the conservative deletion I have nuked just over 411,000 nodes. This translates to a space savings of about 50 MB in the planet.osm file assuming each empty node takes about 125 bytes of space. Guess that's not even a drop in the bucket. But still a worth while cleanup effort. It even got me pretty high up on the monthly list of most active OSM users on the &lt;a href="http://www.openstreetmap.org/stats/data_stats.html"&gt;data stats&lt;/a&gt;&amp;nbsp;page!&lt;br /&gt;&lt;br /&gt;Some of the more extreme cases of bad data I found were a failed upload of parcel data in Arkansas that was supposed to have been cleaned up by the original uploader but was not due to a bug in the editor. It took &lt;a href="http://www.openstreetmap.org/browse/changeset/8186396"&gt;three&lt;/a&gt;&amp;nbsp;&lt;a href="http://www.openstreetmap.org/browse/changeset/8186543"&gt;changesets&lt;/a&gt;&amp;nbsp;to &lt;a href="http://www.openstreetmap.org/browse/changeset/8186642"&gt;clean&lt;/a&gt; that up. Then there was the case of ridiculous over-noding. There were &lt;a href="http://www.openstreetmap.org/browse/changeset/8501065"&gt;14,000 nodes&lt;/a&gt; that were mostly contained in about a 4 block area near San Bernadino in California. That amount of bad data can actually make it difficult for casual mappers to do anything in the area since some editors may not handle that volume of data very well.&lt;br /&gt;&lt;br /&gt;Time to go get some fresh mop water and continue the cleanup.&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/XpAlEECUxt0" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/06/cleanup-on-map-aisle.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://4.bp.blogspot.com/-ye1QAVc6Y8M/TgYzFw3Kn8I/AAAAAAAAAWE/FXvLTyyudYo/s72-c/Maine_all_nodes.png" height="72" width="72" /><thr:total>0</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-650569282152703009</guid><pubDate>Sun, 15 May 2011 08:29:00 +0000</pubDate><atom:updated>2011-05-15T17:20:55.857-05:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">TIGER</category><title>The Main Attraction</title><description>I have mentioned the TIGER import several times. The initial import tagged roads with abbreviations in their name. For example "Ave" instead of "Avenue" or "St" instead of "Street." However the general rule in OSM is to &lt;b&gt;not&lt;/b&gt; abbreviate things. So a script was devised to automatically expand abbreviations. This script was run by the user &lt;a href="http://www.openstreetmap.org/user/balrog-kun"&gt;balrog-kun&lt;/a&gt;. Until tonight I assumed that it had been run over all of the TIGER data. But then I saw a few abbreviations while working on some &lt;a href="http://wiki.openstreetmap.org/wiki/Kansas_state_highways"&gt;highways in Kansas&lt;/a&gt; and decided to dig a little deeper.&amp;nbsp;I did some XAPI queries (using the database I talked about in a &lt;a href="http://ksmapper.blogspot.com/2011/04/parse-planet-into-pgsnapshot-database.html"&gt;previous post&lt;/a&gt;!) &amp;nbsp;to see if anything interesting stood out. I did one query for all streets named "Main St" and another for all roads named "Main Street" and here is what I got back.&lt;br /&gt;&lt;br /&gt;Green is "Main St" and the rest is "Main Street"&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;a href="http://4.bp.blogspot.com/-7FlvxJcl_UA/Tc9-2zprC_I/AAAAAAAAAVk/wh5_sInE53E/s1600/MainSt.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="441" src="http://4.bp.blogspot.com/-7FlvxJcl_UA/Tc9-2zprC_I/AAAAAAAAAVk/wh5_sInE53E/s640/MainSt.png" width="625" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;/div&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;First of all, I am amused at just how many "Main" streets there are in the US. But also it looks like the name expansion bot was only set loose west of the Mississippi. The bot was run long before I joined OSM so I was not around for any discussions about why this was done. I'm sure it is buried somewhere in the mailing list archives...&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;I haven't done any substantial editing east of the Mississippi which explains why I hadn't noticed abbreviated street names before. I'm still not sure why I found some in western Kansas. The most likely explanation is that a few of the bot uploads failed for some reason, leaving pockets of abbreviations around.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;I pinged balrog-kun about it on IRC so the mystery may be solved in a few hours but I thought the image was amusing enough to share anyway. And for a few more giggles, here is the whole world:&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;a href="http://3.bp.blogspot.com/-t1KPId7_Sik/Tc-FXSbFybI/AAAAAAAAAVs/V0NpWCpVCQA/s1600/MainSt_world.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="410" src="http://3.bp.blogspot.com/-t1KPId7_Sik/Tc-FXSbFybI/AAAAAAAAAVs/V0NpWCpVCQA/s640/MainSt_world.png" width="625" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;It seems the UK has a fair share of Main streets as well, plus a few scattered mostly in former British colonies. I'm sure they started it and then I guess we here in the US just &lt;a href="http://www.youtube.com/watch?v=EbVKWCpNFhY"&gt;turned it up to 11&lt;/a&gt;.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Update After talking to balrog-kun: Apparently he was moving west-to-east and at some point some people started complaining for some reason so he stopped. Fair enough I guess. He's not sure about the ones I found in western Kansas though.&amp;nbsp;&lt;/div&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/dlZK3kfUIpw" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/05/main-attraction.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://4.bp.blogspot.com/-7FlvxJcl_UA/Tc9-2zprC_I/AAAAAAAAAVk/wh5_sInE53E/s72-c/MainSt.png" height="72" width="72" /><thr:total>2</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-2963103839147233697</guid><pubDate>Sun, 10 Apr 2011 20:36:00 +0000</pubDate><atom:updated>2011-04-10T15:36:13.239-05:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">mapping</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><title>One Mapper, One Year</title><description>Today is the one year anniversary of&lt;a href="http://www.openstreetmap.org/browse/changeset/4388684"&gt; my first OpenStreetMap edit&lt;/a&gt;! I thought I would take the opportunity to reflect on how OSM and myself have had an effect on each other. My intention is not to brag about all the mapping I have done but rather to point out some different ways in which contributions can be made and some of the tools I use when mapping.&lt;br /&gt;&lt;br /&gt;My very first edit was to add a foot/bike path near my apartment so that it would show up on &lt;a href="http://opencyclemap.org/"&gt;OpenCycleMap&lt;/a&gt;. Unfortunately the imported TIGER roads were so far off that I had to realign them before I could add the path in the correct location. And of course correcting the ones right around the path led me to correct the ones connecting to those and the ones connecting to &lt;b&gt;those&lt;/b&gt; and so on... At the time I didn't even know what TIGER was or what all those tiger:* tags were on all the roads.&lt;br /&gt;&lt;br /&gt;Some highlights since then:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;GPS trackpoints &lt;a href="http://www.openstreetmap.org/user/ToeBee/traces"&gt;uploaded&lt;/a&gt;: 283,271 in 75 traces (most from my Garmin Edge 305)&lt;/li&gt;&lt;li&gt;410&amp;nbsp;&lt;a href="http://www.openstreetmap.org/user/ToeBee/edits"&gt;changesets&lt;/a&gt;&amp;nbsp;created&amp;nbsp;using Potlatch, JOSM, OsmAnd and Vespucci&lt;/li&gt;&lt;li&gt;Pictures taken: over 2,000&amp;nbsp;(4.5 GB saved)&amp;nbsp; Most were geotagged to a GPS trace to aid in mapping features (uploaded some of them to&amp;nbsp;&lt;a href="http://openstreetview.org/?lat=39.941890036188&amp;amp;lon=-100.7890033705&amp;amp;zoom=5"&gt;openstreetview.org&lt;/a&gt;)&lt;/li&gt;&lt;li&gt;Some of the locations I have edited:&amp;nbsp;&lt;/li&gt;&lt;ul&gt;&lt;li&gt;My &lt;a href="http://www.openstreetmap.org/browse/changeset/4711369"&gt;university campus&lt;/a&gt;&lt;/li&gt;&lt;li&gt;My &lt;a href="http://www.openstreetmap.org/browse/changeset/4972565"&gt;city&lt;/a&gt;&lt;/li&gt;&lt;li&gt;My &lt;a href="http://www.openstreetmap.org/browse/changeset/5642488"&gt;state&lt;/a&gt;&lt;/li&gt;&lt;li&gt;My &lt;a href="http://www.openstreetmap.org/browse/changeset/5214118"&gt;country&lt;/a&gt;&lt;/li&gt;&lt;li&gt;My &lt;a href="http://www.openstreetmap.org/browse/changeset/4682033"&gt;high school&lt;/a&gt;&lt;/li&gt;&lt;li&gt;My &lt;a href="http://www.openstreetmap.org/browse/changeset/7301352"&gt;childhood home&lt;/a&gt;&lt;/li&gt;&lt;li&gt;Some &lt;a href="http://www.openstreetmap.org/browse/changeset/7662208"&gt;random place&lt;/a&gt; where there was an earthquake&lt;/li&gt;&lt;/ul&gt;&lt;li&gt;Map objects affected (according to&amp;nbsp;&lt;a href="http://hdyc.neis-one.org/?ToeBee"&gt;hdyc.neis-one.org&lt;/a&gt;&amp;nbsp;- last updated in January)&lt;/li&gt;&lt;ul&gt;&lt;li&gt;Nodes: about 165,000&lt;/li&gt;&lt;li&gt;Ways: about 24,500&lt;/li&gt;&lt;li&gt;Relations: about 500 (I created over 200 county relations in 3 states since the last update)&lt;/li&gt;&lt;/ul&gt;&lt;li&gt;Contacted multiple government agencies in Kansas to get explicit permission to use their data to assist in mapping. Documented these data sources on the &lt;a href="http://wiki.openstreetmap.org/wiki/Kansas"&gt;Kansas &lt;/a&gt;page in the wiki&lt;/li&gt;&lt;li&gt;Started a &lt;a href="http://ksmapper.blogspot.com/"&gt;blog&lt;/a&gt;&amp;nbsp;(self referring link!)&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;Now for a pretty picture. Here are all the GPS traces in my city. I think there may be one or two from other people but most of it is me.&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;&lt;table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td style="text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-rtpNSOAPLeM/TaHL1wKKeVI/AAAAAAAAAVY/kVy8PBm5R2s/s1600/MHK_gps_traces.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"&gt;&lt;img border="0" height="537" src="http://4.bp.blogspot.com/-rtpNSOAPLeM/TaHL1wKKeVI/AAAAAAAAAVY/kVy8PBm5R2s/s640/MHK_gps_traces.png" width="630" /&gt;&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td class="tr-caption" style="text-align: center;"&gt;GPS traces in Manhattan, KS&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;br /&gt;Most of them were collected on my bicycle although there are a few roads in town that I do not bike on because I do not have a death wish so those were done in the car. But I still have work to do! On Saturday I was talking to some people in the &lt;a href="http://www.fhabclub.org/"&gt;Flint Hills Area Bike Club&lt;/a&gt; and heard laments of new streets missing from maps. They were of course talking about google maps but if these new streets are added to OSM first, I may have some converts!&lt;br /&gt;&lt;br /&gt;And speaking of converts, I have also had the pleasure of working with some other locals to enhance and use OSM data in interesting applications. For example, &lt;a href="http://ksuanth.weebly.com/wesch.html"&gt;Michael Wesch&lt;/a&gt; is a professor at Kansas State University. He used OSM data to make a &lt;a href="http://bikemanhattan.info/map.htm"&gt;bicycle safety map&lt;/a&gt;&amp;nbsp;for the city. In trying to get the map into a usable state, he added a lot of the bigger landuse areas in the city that you now see on the map. He also convinced the College of Architecture,&amp;nbsp;Planning&amp;nbsp;and Design to focus its "&lt;a href="http://www.k-state.edu/media/newsreleases/aug10/bikedesign82010.html"&gt;Design Days&lt;/a&gt;" project on bicycle friendliness and to use OSM as their map. Almost 200 students were sent out into the city with &lt;a href="http://walking-papers.org/"&gt;walking papers&lt;/a&gt; to find any bicycle oriented path or amenity that wasn't on the map and report back to people on campus who then entered the information into the OSM database. I had about 10 minutes to teach 10 people how to use JOSM. Oh the things I would now do differently.&lt;br /&gt;&lt;br /&gt;Several new paths were found that connect neighborhoods and open up the option of safely commuting on a bicycle or by foot to school and the university. Many of these paths will never make it to google/yahoo/bing/etc maps because they are not officially maintained or even known about by city officials. They are just paths that people have worn into the ground by repeated use. At the end, each group presented plans for improving one section of the city and most used OSM renderings in their presentations. I will admit that I felt a sense of satisfaction seeing mapnik renderings of the city printed out on three foot wide paper and being displayed in projected presentations.&lt;br /&gt;&lt;br /&gt;&lt;table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td style="text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-ihwSCvjgkBE/TaHVRTTPFrI/AAAAAAAAAVc/X17LC-ezZEI/s1600/2010-08-25+17.52.11.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"&gt;&lt;img border="0" height="475" src="http://4.bp.blogspot.com/-ihwSCvjgkBE/TaHVRTTPFrI/AAAAAAAAAVc/X17LC-ezZEI/s640/2010-08-25+17.52.11.jpg" width="630" /&gt;&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td class="tr-caption" style="text-align: center;"&gt;A giant printed map for people to mark on at the Design Days public presentation&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;br /&gt;And speaking of the university, my involvement in OSM got me interested in geographic data in general and I decided to take advantage of the university employee tuition discount. I took a cartography course last fall and am now in the middle of the GIS I course. It is interesting to see differences in the way OSM and "professional" GIS applications (mostly ESRI ArcGIS for these courses) handle data and allow you to interact with it. Both have different strengths and weaknesses.&lt;br /&gt;&lt;br /&gt;After my dramatic and internationally anticipated* entrance into the blogosphere with my &lt;a href="http://ksmapper.blogspot.com/2010/10/tiger-ways-in-kansas.html"&gt;TIGER ways in Kansas&lt;/a&gt; post, I was made aware of and encouraged to attend &lt;a href="http://www.wherecamp5280.org/"&gt;WhereCamp5280&lt;/a&gt; in Denver. Here I met &lt;a href="http://www.stevecoast.com/"&gt;Steve&lt;/a&gt;&amp;nbsp;and&amp;nbsp;&lt;a href="http://www.hurricanemcewen.com/"&gt;Hurricane&lt;/a&gt;&amp;nbsp;Coast, &lt;a href="http://weait.com/"&gt;Richard Weait&lt;/a&gt; and &lt;a href="http://blog.samat.org/"&gt;Samat Jain&lt;/a&gt; among others. I learned several things that day and had some fun while I was at it. Ski mapping in &lt;a href="http://www.openstreetmap.org/?lat=39.87943&amp;amp;lon=-105.77013&amp;amp;zoom=15&amp;amp;layers=M"&gt;Winter Park&lt;/a&gt; the next day didn't hurt the "fun" aspect either!&lt;br /&gt;&lt;span class="Apple-style-span" style="font-size: xx-small;"&gt;*for some definition of "dramatic" and "internationally anticipated"&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;A lot of my mapping has obviously been local to &lt;a href="http://www.openstreetmap.org/?lat=39.1981&amp;amp;lon=-96.5812&amp;amp;zoom=13&amp;amp;layers=M"&gt;Manhattan, KS&lt;/a&gt;. (Note: I have heard rumors that some other "Manhattan" exists up in the northeast of the country. I have been unable to verify this rumor personally and assume it is false based on my own&amp;nbsp;empirical&amp;nbsp;evidence) This is OSM's strength. Get people who know their own neighborhood to add detail to the map and then update it as it changes. As evidenced by the GPS traces above, I have explored much of the city. Doing so on a bicycle has been really fun. I have discovered interesting neighborhoods and parts of the city that I had never seen before. It surprised me how much I did &lt;b&gt;not&lt;/b&gt;&amp;nbsp;know about this relatively small city even after having lived here for over 10 years.&lt;br /&gt;&lt;br /&gt;&lt;table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td style="text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/-JrqBEw1MfPM/TaIBnm05lFI/AAAAAAAAAVg/xlAlO8vz11Q/s1600/map.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"&gt;&lt;img border="0" height="590" src="http://2.bp.blogspot.com/-JrqBEw1MfPM/TaIBnm05lFI/AAAAAAAAAVg/xlAlO8vz11Q/s640/map.png" width="630" /&gt;&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td class="tr-caption" style="text-align: center;"&gt;Current map of Manhattan, KS&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;br /&gt;Other edits have touched places that I have visited on trips. After the trip to Denver, I edited all along I-70. Mostly just improving alignment plus adding the occasional gas station, hotel or other business along the interstate that I managed to snap a picture of as I went past. I also ended up fixing a major cloverleaf intersection in Denver that the local OSMers hadn't gotten to yet. Tsk. Tsk.&lt;br /&gt;&lt;br /&gt;Some of the more regional editing started with tracing the &lt;a href="http://www.openstreetmap.org/browse/relation/1061544"&gt;Kansas River&lt;/a&gt; from USGS imagery. Someone else happened to be doing the same thing, starting from the east. I started from the west and we met up in the middle. More recently I have started doing some edits that touch much larger areas. For example, while browsing the &lt;a href="http://open.mapquest.com/"&gt;MapQuest Open&lt;/a&gt; map, I noticed that county name labels were being placed towards the southeast of the county instead of the middle where you would expect them to be. After some investigation I realized that they are rendering the county name from the place=county nodes and that when these were originally imported, they were all off in the extreme southeast of their counties for some reason. So I used an XAPI query to get all the county nodes in Kansas and another one to get all the county borders. Then I moved all the nodes to be approximately in the center of their respective counties. Now the MapQuest site shows county names in the correct location.&lt;br /&gt;&lt;br /&gt;While local mappers are key to OSM's value, people with some data analysis skills and above average technical abilities are also needed to tend to some of the "bigger" data that local mappers may not notice or care about. In the US, a lot of this data that needs tending to comes from data imports. Administrative boundaries, TIGER data, etc. Imports is an entirely separate topic that I won't get in to here though.&lt;br /&gt;&lt;br /&gt;Another issue that requires some data analysis is tagging. Luckily we have some great tools such as &lt;a href="http://tagstat.hypercube.telascience.org/"&gt;tagstat &lt;/a&gt;and &lt;a href="http://taginfo.openstreetmap.de/"&gt;taginfo &lt;/a&gt;to help explore tag usage. One problem with tags is typos. Recently I got a message from a mapper who discovered through one of these tools that I had mistyped the word "petroleum" when mapping some of the oil wells south of the city. Thanks to JOSM's auto complete feature when entering tags, the typo had been applied to all of the wells in the area. Thanks to this other mapper discovering the outlying values in this tag, the typo was discovered and corrected.&lt;br /&gt;&lt;br /&gt;Another problem with tags is inconsistencies. Unfortunately, as with many open source projects, documentation seems to be a low priority. When I first tried to figure out how to tag a doctor's office, I found 3 different pages on the OSM wiki describing how to tag them. One of them was just a stub page. The other two were fairly well fleshed out. But by looking at the tagstat I found that the one that was being used the most by far was the tag&amp;nbsp;corresponding&amp;nbsp;to the stub page. It looks like this has been cleaned up somewhat now. There are still &lt;a href="http://wiki.openstreetmap.org/wiki/Proposed_features/Doctor"&gt;two&lt;/a&gt;&amp;nbsp;different &lt;a href="http://wiki.openstreetmap.org/wiki/Tag:amenity%3Ddoctors"&gt;pages&lt;/a&gt;&amp;nbsp;with a proposal to merge them.&lt;br /&gt;&lt;br /&gt;Initially I was really frustrated by the disorganized tagging rules. In my day job, inconsistent data is sometimes worse than missing data. I still haven't entirely gotten over it but at the end of the day, I consider the various sources and tag things however it makes the most sense to me. The bad thing about such an approach is that it makes the OSM data harder for our consumers to use because they have to consider all the tagging variations for a given type of object. The good news is that since I had these problems, a lot of commonly mapped objects have been added to presets that various editors have so most users don't need to know about the intricacies of tag values or the impressive debates that surround them on various &lt;a href="http://lists.openstreetmap.org/listinfo"&gt;mailing lists&lt;/a&gt;. Just use the preset and you're done - and the data is consistent.&lt;br /&gt;&lt;br /&gt;In summary: It has been a fun year learning how to map, meeting other mappers, getting involved in the mapping community and experimenting with map data. I plan to continue! In fact, I just recently decided to make the relationship official and became a card-carrying member of the&amp;nbsp;&lt;a href="http://www.openstreetmap.us/"&gt;OSM-US&lt;/a&gt; foundation. Although I'm still waiting on the actual card to arrive. In the meantime...&lt;br /&gt;&lt;br /&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;Map the planet!&lt;/span&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/ogunI9ajjh8" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/04/one-mapper-one-year.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://4.bp.blogspot.com/-rtpNSOAPLeM/TaHL1wKKeVI/AAAAAAAAAVY/kVy8PBm5R2s/s72-c/MHK_gps_traces.png" height="72" width="72" /><thr:total>2</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-4792440131163113296</guid><pubDate>Sat, 09 Apr 2011 04:34:00 +0000</pubDate><atom:updated>2011-08-31T01:12:56.387-05:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">osmosis</category><category domain="http://www.blogger.com/atom/ns#">pgsnapshot</category><category domain="http://www.blogger.com/atom/ns#">postgis</category><title>Keeping a database up to date with osmosis</title><description>This is a follow-up to my &lt;a href="http://ksmapper.blogspot.com/2011/04/parse-planet-into-pgsnapshot-database.html"&gt;previous post&lt;/a&gt; about importing a planet file into a pgsnapshot database schema. However the same basic process applies to other schemas that osmosis can write to.&lt;br /&gt;
   &lt;br /&gt;
   To briefly set the stage: After downloading the weekly planet file and processing/importing it into a database, the data is now probably 3 or 4 days old. OpenStreetMap provides both minutely and hourly change files that can be applied to the database to bring it up to date with what is live right now in the master OSM database. If you think about it, that is actually a pretty sweet deal. Someone on the other side of the planet can spot a new restaurant while on their lunch break, use an application on their phone to upload it to OSM and you will get the change applied to your local database in less than &amp;nbsp;120 seconds. Does this blow anyone else's mind just a little? Open data is awesome like that.&lt;br /&gt;
   &lt;br /&gt;
   &lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;
   &lt;span class="Apple-style-span" style="font-size: x-large;"&gt;Initializing the working directory&lt;/span&gt;&lt;br /&gt;
   &lt;br /&gt;
   So what does it take to pull of this feat of mad wizardry? The "&lt;a href="http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage#Replication_Tasks"&gt;Replication Tasks&lt;/a&gt;" section of the osmosis usage page is of interest. Again, not the easiest read in the world. Let's break it down.&amp;nbsp;First, you need to set up a working directory for osmosis to keep some information about the current state of the data. To do this, create a directory. I chose to put this in a hidden directory in my home dir: ~/.osmosis-minutely/ Next, enter this directory and execute this command (again, I am assuming you have osmosis installed in your home directory):&lt;br /&gt;
   &lt;br /&gt;
   &lt;pre class="source-code"&gt;~/osmosis/bin/osmosis --rrii
  &lt;br /&gt;&lt;/pre&gt;
   &lt;br /&gt;
   This will create two files: download.lock and configuration.txt. You can ignore download.lock. It is used by osmosis to ensure that only one copy is running at a time. The interesting bits are in configuration.txt. First of all, this is where you have to decide if you want to do hourly or minutely updates. Minutely allows you to stay in sync with live changes to the map as people upload them as described above. However&amp;nbsp;this means that your computer will always be spending anywhere from maybe 10 seconds to a full minute every minute importing OSM data. This won't make it unusable for other things but will certainly have an impact on the system. In theory using the hourly files might be slightly more efficient. I haven't played with this to determine if this is true or not but Paul Norman indicated that it didn't seem to be for him. Once you have decided which to use, put either&amp;nbsp;&lt;i&gt;http://planet.openstreetmap.org/minute-replicate&lt;/i&gt; or&amp;nbsp;&lt;i&gt;http://planet.openstreetmap.org/hour-replicate&lt;/i&gt; in the &lt;i&gt;baseUrl &lt;/i&gt;line of the configuration.txt file.&lt;br /&gt;
   &lt;br /&gt;
   Now you need to determine the time stamp of your planet file. It is in the 2nd line of the planet file. You can get it like this:&lt;br /&gt;
   &lt;br /&gt;
   &lt;pre class="source-code"&gt;bunzip2 -c planet-110309.osm.bz2 | head&lt;/pre&gt;
   &lt;br /&gt;
   In my case it reads &lt;i&gt;timestamp="2011-03-09T01:11:06Z"&lt;/i&gt;&lt;br /&gt;
   &lt;br /&gt;
   Now you can do it the easy way or the hard way. The easy way is to use the&amp;nbsp;&lt;a href="http://toolserver.org/~mazder/replicate-sequences/"&gt;replicate-sequences tool&lt;/a&gt;. Note: this tool only supports minutely updates. As mentioned in the osmosis documentation, I would suggest taking the time stamp from above and subtracting an hour and using that time just to make sure you don't miss an update. After you enter the time and click the button, take the text it returns and copy/paste it into a file named state.txt in the working directory (~/.osmosis-minutely/)&lt;br /&gt;
   &lt;br /&gt;
   If you want to know what is going on behind the scenes, read on here. Otherwise feel free to skip to the next section. Every minute (and every hour) files are created on&amp;nbsp;&lt;a href="http://planet.openstreetmap.org/"&gt;http://planet.openstreetmap.org&lt;/a&gt;&amp;nbsp;that contain the changes that have been uploaded to the database since the last minute/hourly file was generated. These files contain only the new attributes of objects that have been changed. So applying a change twice is fine. It just updates the data to the same values it had before. This is why it is safe to go back an hour from your time stamp. However if you miss an update, all the objects affected by that update will be out of sync with the master database until (if ever) they are updated again.&lt;br /&gt;
   &lt;br /&gt;
   Along with the change file itself, there is also file containing metadata about the change. For example, &lt;a href="http://planet.osm.org/minute-replicate/000/751/000.state.txt"&gt;this one&lt;/a&gt;. It has a sequence number that increments for each change file plus a time stamp of when exactly the last change in the associated change file was applied. If you want to find the correct state file without using the tool listed above (or if you want to use hourly updates) you will need to browse through the directories to find an appropriate state file to start off from, again going back in time at least a few minutes or an hour. The time stamps listed next to the files are one hour off from the time stamp inside of the file (the time stamp in the file is in UTC and the server is one time zone away from UTC) so you can use it as a rough guide for locating the correct file but be sure to check the time stamp inside of the file before using it! As above, paste the contents of the appropriate state file into your ~/.osmosis-minutely/ directory as &lt;i&gt;state.txt&lt;/i&gt;.&lt;br /&gt;
   &lt;br /&gt;
   &lt;span class="Apple-style-span" style="font-size: x-large;"&gt;Catching up&lt;/span&gt;&lt;br /&gt;
   &lt;br /&gt;
   You should have everything in place now! Time to fire up osmosis! Paul reached this point long before I did (because I was playing around with things) so I swiped this command from him:&lt;br /&gt;
   &lt;br /&gt;
   &lt;pre class="source-code"&gt;~/osmosis/bin/osmosis --rri workingDirectory=~/.osmosis-minutely/ 
  &lt;br /&gt;                      --sc 
  &lt;br /&gt;                      --wpc user="xapi" database="xapi" password="xapi"&lt;/pre&gt;
   &lt;br /&gt;
   Let's look at the command:&lt;br /&gt;
   &lt;ul&gt;
   &lt;li&gt;--rri or "&lt;a href="http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.39#--read-replication-interval_.28--rri.29"&gt;read replication interval&lt;/a&gt;:" Not much to add to the wiki description. This makes osmosis download a set of change files and tells it where to look for the state.txt file to know which files to download.&lt;/li&gt;
   &lt;li&gt;--sc or "&lt;a href="http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.39#--sort-change_.28--sc.29"&gt;sort change&lt;/a&gt;:" I'm not 100% sure about the function here. I'm assuming it puts the changes in chronological order so that you don't end up applying a newer change before an older change which would lead to an outdated object in your database.&lt;/li&gt;
   &lt;li&gt;--wpc or "write pgsql change:" writes the change to the database using the given credentials. (modify to match your database/username/password) This task can be replaced by other "write change" tasks like &lt;a href="http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.39#--write-apidb-change_.28--wdc.29"&gt;--wdc&lt;/a&gt; or &lt;a href="http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.39#--write-xml-change_.28--wxc.29"&gt;--wxc&lt;/a&gt;&amp;nbsp;to write changes to other destinations.&lt;/li&gt;
   &lt;/ul&gt;
   &lt;br /&gt;
   By default osmosis will download one hours worth of changes (even if you are using minutely updates - it will just download 60 at a time) and then sit there applying them. This will take anywhere from a few minutes to a full hour. It just depends on how many changes are in the files it is processing and how complex the changes are. Applying changes to huge multipolygons is known to be slow and weekends are always a lot busier than week days.&lt;br /&gt;
   &lt;br /&gt;
   If you want quicker feedback to make sure it is working, open the ~/.osmosis-minutely/configuration.txt file and change the &lt;i&gt;maxInterval &lt;/i&gt;value to something lower. This controls the maximum number of changes that osmosis will download in one go. As mentioned above, the default is one hour (3600 seconds) so you can set it to 60 seconds to just download one minute at a time which should only take a few seconds to apply. Once the process finishes you can check the state.txt file. The time stamp and sequence number should have changed to reflect the update that was applied.&lt;br /&gt;
   &lt;br /&gt;
   Due to my tinkering around and multiple runs I did with osmosis, I had to apply 488 hours worth of updates to my database. I did it by hand in 10 and 48 hour chunks by changing maxInterval to 36000 and 172800, respectively. After it was all said and done, it took about 163 hours to apply 488 hours of updates. That averages out to about 20 minutes of processing time per hour of updates. But I saw variation, especially in the 10 hour runs. Anywhere from 1.8 to 4.5 hours to apply 10 hours of updates. As before, disk I/O tends to be the bottleneck and I'm using a software RAID5 composed of three 1TB 7,200RPM drives so not exactly tuned for speed.&lt;br /&gt;
   &lt;br /&gt;
   Now all you need to do is put the osmosis command into a minutely or hourly cron job and you're done! And really you can do this before the database is all caught up. I just did the catching up by hand to monitor and record progress. As I mentioned above, it is safe to call osmosis every minute. Osmosis puts a lock on the download.lock file and if the previous run is still executing, the next one will exit immediately without doing anything. I chose to put the command in a script and then put a call to the script into cron. You might want to redirect stdout and stderr to either a log file or /dev/null to avoid cron sending out emails every time the script runs. My cron job looks like this, minus the newline put in for readability here:&lt;br /&gt;
   &lt;br /&gt;
   &lt;pre class="source-code"&gt;* * * * * /home/toby/bin/osmosis-update.bash &amp;gt;&amp;gt; 
  &lt;br /&gt;          /home/toby/.osmosis-minutely/osmosis.log 2&amp;gt;&amp;amp;1&lt;/pre&gt;
   &lt;br /&gt;
   And that's all, folks! You now have an automatically updating OSM database to query as you please. Have fun!&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/Zyu1ZrQc6EA" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/04/keeping-database-up-to-date-with.html</link><author>noreply@blogger.com (Unknown)</author><thr:total>6</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-4708535036302350732</guid><pubDate>Sun, 03 Apr 2011 22:35:00 +0000</pubDate><atom:updated>2011-11-22T23:46:50.615-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">osmosis</category><category domain="http://www.blogger.com/atom/ns#">pgsnapshot</category><category domain="http://www.blogger.com/atom/ns#">postgis</category><title>Parse the Planet! (into a pgsnapshot database)</title><description>What makes OpenStreetMap... well... &lt;b&gt;Open&lt;/b&gt;? In a link: &lt;a href="http://planet.openstreetmap.org/"&gt;http://planet.openstreetmap.org/&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;Yes, of course there are the legal issues of having an open license, the transparent community processes to govern the project and a whole host of open source software to create, manipulate, consume and display map data. But at the end of the day, none of that would matter if the raw data wasn't available.&lt;br /&gt;&lt;br /&gt;As is obvious from my previous posts, I have dabbled in manipulating this raw data in the past, with varying degrees of success. After learning from my previous attempts and hanging around on the mailing lists and IRC channel, picking up information here and there I decided to take another crack at processing a planet file. I ended up finding a partner in crime: &lt;a href="http://www.paulnorman.ca/blog/"&gt;Paul Norman&lt;/a&gt;. He was wanting to import a planet file for use in a private XAPI server using the new &lt;a href="https://github.com/iandees/xapi-servlet"&gt;java based XAPI implementation&lt;/a&gt; written by &lt;a href="http://www.openstreetmap.org/user/iandees/diary"&gt;Ian Dees&lt;/a&gt;. This sounded like a good idea so I decided to follow along. We bounced commands and results off of each other for a while. I ended up going a slightly different route in some of the processing because I had some different hardware available to me.&lt;br /&gt;&lt;br /&gt;The first thing to know about processing planet data is that there are several different database schemas that can be used to hold the data. This topic probably deserves its own post at some point. Here is a brief list of the ones I am at least minimally familiar with. I know there are some others (like the gazetteer database used by nominatim):&lt;br /&gt;&lt;ul&gt;&lt;li&gt;API database: Creates a duplicate of the master OSM database. Does not use any GIS features. Nodes are stored with lat/lon values as simple numbers. Contains tables for: ways, tags, users, changesets. Can track history of objects once you start applying updates.&lt;/li&gt;&lt;li&gt;"pgsnapshot" schema: creates a schema that utilizes the GIS features of postgis that allows you to run spatial queries against OSM data. Still maintains all tag information but they are in a single "hstore" column, not in their own relational tables. Changesets are not stored nor is any information about users except the ID and username. History is not maintained as updates are applied. There is also a related "pgsimple" schema that seems to be an older version of the pgsnapshot schema that may have some advantages. Differences are discussed in the "pgsnapshot_and_pgsimple.txt" file in the osmosis &lt;i&gt;script &lt;/i&gt;directory.&lt;/li&gt;&lt;li&gt;osm2pgsql rendering schema: This schema is created by the osm2pgsql tool and used by mapnik to render map tiles. It uses GIS features as well but it is lossy and very de-normalized. Only objects and tags that you care about rendering are imported. This means it is also smaller and faster to get set up.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;The XAPI uses the pgsnapshot schema so this is what I set up. The instructions on how to get going are provided in the &lt;a href="https://github.com/iandees/xapi-servlet/blob/master/readme.md"&gt;xapi readme file&lt;/a&gt; but I will go ahead and copy it here with some additional notes.&lt;br /&gt;&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;&lt;b&gt;Setting up the database&lt;/b&gt;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;First of all, you will need a postgresql server with some spare drive space. If you are processing the entire planet file, you will end up creating a database that is about 430 GB in size. However during processing, you may end up needing anywhere between 200 and 400 GB of additional space for temporary files. Some of this depends on exactly how you execute the process. And some of the storage doesn't need to be fast. I used an external USB drive for some of the temporary files and it worked great.&lt;br /&gt;&lt;br /&gt;Next, you might want to take a quick look at the "Configure the PostGIS database" section in &lt;a href="http://weait.com/content/build-your-own-openstreetmap-server"&gt;this post by Richard Weait&lt;/a&gt;. I am pretty new to postgresql so I have no tuning recommendations of my own. I used the values he suggested and they seem to work.&lt;br /&gt;&lt;br /&gt;As the postgres user on your database system execute the following commands from a shell: &lt;br /&gt;&lt;pre class="source-code"&gt;createdb xapi&lt;br /&gt;createlang plpgsql xapi&lt;br /&gt;createuser xapi&lt;br /&gt;&lt;/pre&gt;Note: this may not be the best way to do things, but answer "yes" to the superuser question. Otherwise you have to modify postgres ownership which wasn't quite straightforward. Mostly because I'm new to postgres.&lt;br /&gt;&lt;br /&gt;Now to set up the GIS part of postgis and the hstore for tags. This step sets up the database to hold spatial data instead of just being a bland relational database. Note that your exact path may vary depending on what Linux distro you are using. You can always try a quick &lt;i&gt;locate&lt;/i&gt; command. On Ubuntu you have to install the &lt;i&gt;postgresql-8.4-hstore-new&lt;/i&gt; package first. On Gentoo I actually had to compile the hstore-new plugin myself.&lt;br /&gt;&lt;pre class="source-code"&gt;psql -d xapi -f /usr/share/postgresql/8.4/contrib/postgis.sql&lt;br /&gt;psql -d xapi -f /usr/share/postgresql/8.4/contrib/spatial_ref_sys.sql&lt;br /&gt;psql -d xapi -f /usr/share/postgresql/8.4/contrib/hstore-new.sql&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;Now we set up the actual OSM tables. You will need to download &lt;a href="http://wiki.openstreetmap.org/wiki/Osmosis"&gt;osmosis&lt;/a&gt;. The pgsnapshot schema was just recently introduced. I had to use a development snapshot but as of April 2, it is included in the release of osmosis 0.39. In this post I will assume that you have osmosis unpacked in your home directory.&lt;br /&gt;&lt;pre class="source-code"&gt;psql -d xapi -f ~/osmosis/script/pgsnapshot_schema_0.6.sql&lt;br /&gt;psql -d xapi -f ~/osmosis/script/pgsnapshot_schema_0.6_linestring.sql&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;&lt;br /&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;&lt;b&gt;Getting the data&lt;/b&gt;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Now that the database is ready, let's get some data to work on. For the boring people, just go to &lt;a href="http://planet.openstreetmap.org/"&gt;http://planet.openstreetmap.org/&lt;/a&gt; and download the file &lt;i&gt;planet-latest.osm.bz2.&lt;/i&gt; It will likely take several hours to get the 15 GB file. For the cool kids, check out &lt;a href="http://osm-torrent.torres.voyager.hr/"&gt;http://osm-torrent.torres.voyager.hr/&lt;/a&gt; where you can find a torrent file with the latest planet dump in it. Planet files are typically produced every Wednesday. The torrent is updated a day or two later. I downloaded several planet files via the torrent and each time I truly did get a &lt;i&gt;torrent &lt;/i&gt;of data. During most of the transfer, my downstream bandwidth was maxed out at 2 megabytes/sec.&lt;br /&gt;&lt;br /&gt;If this is your first time working with OSM data I would strongly recommend testing the waters with a country or state extract. Extract providers are listed on the &lt;a href="http://wiki.openstreetmap.org/wiki/Planet"&gt;Planet page&lt;/a&gt; on the wiki. I would look at geofabrik and cloudmade. Geofabrik has the data available in a format called PBF which is more compact and quicker to work with than the bzip2 compressed XML file you get from planet.osm.&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;&lt;b&gt;Processing with Osmosis&lt;/b&gt;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;You might want to go refill your drink. This is going to be long.&lt;br /&gt;&lt;br /&gt;Osmosis is an amazing tool. Also, amazingly confusing. Just take a look at the &lt;a href="http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage"&gt;"Detailed Usage"&lt;/a&gt; page on the wiki. The page &lt;b&gt;summary&lt;/b&gt; is two full screens tall in my browser. And once you start to actually read things, it just gets worse. It talks about tasks, pipes and streams as casually as most people would discuss the weather. These are all useful things if you're an osmosis expert wanting to look up how to do a specific task like splitting a planet file into 16 polygon defined extracts, all at the same time - while filtering out relations. But it is a little daunting to new users.&lt;br /&gt;&lt;br /&gt;One thing to keep in mind: argument order &lt;b&gt;matters&lt;/b&gt;. For example, If you want to use the --log-progress option to output progress information, it must go between the read and write tasks - you can't simply add it to the end of the command line. There are also multiple ways to do some things. This can be a good thing as it lets you optimize the process based on what kind of hardware you have available to you.&lt;br /&gt;&lt;br /&gt;Osmosis is a java program. As such, you may need to specify certain options to the JVM to make it do what you want. These JVM level options are controlled by a file. On linux this file is named &lt;i&gt;.osmosis&lt;/i&gt; and needs to be in your home directory. On windows 7 it needs to be at &lt;i&gt;C:\Users\&amp;lt;user name&amp;gt;\osmosis.bat&lt;/i&gt;. The most common option you may need to use is to give java more memory using the -Xmx option. The other one I needed for one of my attempts was -Djava.io.tmpdir=/some/other/path/ to set the location of the temporary directory to something other than /tmp/&lt;br /&gt;&lt;br /&gt;So my ~/.osmosis file in linux looks like this to give it 2 GB of memory and redirect the temp directory&lt;br /&gt;&lt;pre class="source-code"&gt;export JAVACMD_OPTIONS="-Xmx2G -Djava.io.tmpdir=/mnt/database/tmp/"&lt;/pre&gt;&lt;br /&gt;The format is identical in windows except you replace "export" with "set"&lt;br /&gt;&lt;br /&gt;One more important note: Osmosis claims to be able to read bzip2 compressed files natively and will happily set about doing this for a few minutes. But &lt;b&gt;do not attempt this!&lt;/b&gt; It will end in tears and very odd "invalid XML" errors being thrown. The reason is that java's native bzip2 implementation can not read files with multiple streams in them. So it reads the first stream and thinks it is done, truncating any XML elements it might be in the middle of at the time. Instead, use an external utility to unzip the file to standard out and then tell osmosis to read the uncompressed data from standard in. There is a note about this at the bottom of the osmosis page on the wiki but it is easy to miss (I did at first) so it bears repeating.&lt;br /&gt;&lt;br /&gt;Now that we have some basics down (I told you this was going to be long) let's look at what it takes to actually process the data. The first thing we need to do is read the XML using the &lt;a href="http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage#--read-xml_.28--rx.29"&gt;--read-xml&lt;/a&gt; task.&lt;br /&gt;&lt;br /&gt;Next, there is a decision to make. There are two basic ways to get the data into the database. One is to let osmosis do it all automatically using the &lt;a href="http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage#--write-pgsql_.28--wp.29"&gt;--write-pgsql&lt;/a&gt; task. This will cause osmosis to read the XML into some temporary files in your tmp directory and then shove it into the database. The first step may take up to several days (depending on if you are using memory node storage or not - keep reading) and create close to 200 GB of temporary files before it even touches the database. If something fails during the database load, the temporary files are deleted and you start over from scratch. Why the temporary files? Osmosis has to do some processing of the data to correctly construct ways out of nodes and relations out of ways and nodes before it can put them in the database.&lt;br /&gt;&lt;br /&gt;The other option is to use the &lt;a href="http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage#--write-pgsql-dump_.28--wpd.29"&gt;--write-pgsql-dump&lt;/a&gt; task to process the XML into a series of files that can be loaded into postgres using the \copy command which is much faster than issuing INSERT queries. It also gives you a fall back point if something goes wrong during the database import. The primary reason I chose to use it is that it lets you easily run this process on another computer and then take the resulting files and import them into postgres on a different computer. This is also the only reason I have been mentioning Windows. My file server that has the disk space and is running postgres is an older machine with only 3 GB of memory and a slower CPU. My windows desktop is a much newer machine since I use it for gaming. So it has a quad core CPU and 12 GB of memory. I found a good use for the memory.&lt;br /&gt;&lt;br /&gt;I will focus on the --write-pgsql-dump task that I used but some of the same things apply to the --write-pgsql task. I will start out with the full command line I used and then explain the parts. As I mentioned above, I ran this on my windows box because it has the memory. So please excuse the ugly file paths :)&lt;br /&gt;&lt;pre class="source-code"&gt;c:\bzcat.exe o:\planet-110309.osm.bz2 | bin\osmosis.bat &lt;br /&gt;  --read-xml file=- --log-progress interval=30 &lt;br /&gt;  --write-pgsql-dump directory=g:\osmosis &lt;br /&gt;    enableLinestringBuilder=yes &lt;br /&gt;    nodeLocationStoreType=InMemory&lt;br /&gt;&lt;/pre&gt;Now for the parts:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;&lt;b&gt;c:\bzcat.exe o:\planet-110309.osm.bz2 | bin\osmosis.bat&lt;/b&gt;: As I mentioned, it is a bad idea to let osmosis unzip the file so we must do this using an external utility and pipe the result into osmosis&lt;/li&gt;&lt;li&gt;&lt;b&gt;--read-xml file=-&lt;/b&gt;: This tells osmosis to read an XML document from standard in. It is easy to miss, but don't forget the - at the end!&lt;/li&gt;&lt;li&gt;&lt;b&gt;--log-progress interval=30&lt;/b&gt;: This will cause osmosis to report its progress every 30 seconds. The output simply tells you which node/way/relation it is working on and how many per second it is processing so you can make sure it is actually doing something and possibly estimate completion time.&lt;/li&gt;&lt;li&gt;&lt;b&gt;--write-pgsql-dump directory=g:\osmosis&lt;/b&gt;: Tells osmosis to format its output as a series of postgres dump files in the given directory. The following three sub-options affect the behavior of this task.&lt;/li&gt;&lt;li&gt;&lt;b&gt;enableLinestringBuilder=yes&lt;/b&gt;: This builds a geometric representation of the way which is then stored in a "geometry" type column in postgres that can be used for spatial queries. Building the linestring can be done in the database but you may not live to see it finish if you are processing the entire planet. Seriously.&amp;nbsp;&lt;/li&gt;&lt;li&gt;&lt;b&gt;nodeLocationStoreType=InMemory&lt;/b&gt;: This tells osmosis to hold geometric information about every node in memory. By default it writes this to a temporary file on disk. If you have enough memory you really, really, &lt;b&gt;really&lt;/b&gt; want to do this as it cuts processing time from about a week down to, in my case, about 8 hours.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;I mentioned "enough" memory. I did this on a computer with 12 GB of memory. The first time I tried this command I gave java 12 GB (-Xmx12G) and java threw an out of memory exception even though my system monitoring (I have &lt;a href="http://www.zabbix.com/"&gt;zabbix &lt;/a&gt;set up) showed that there was still 1GB free. So the second time I gave java 16GB of memory. Yes, that is more memory than the system has and I would not recommend doing this on a regular basis... but it worked in this case. In fact, zabbix still showed that there was almost 1 GB free except for a brief dip to 200 MB. I'm guessing there is some interplay between the JVM and the operating system going on that complicated things a bit. Also, keep in mind that my 12 GB was able to process the planet file today with about 1 billion, 100 million nodes in it but the map is constantly growing (at about &lt;a href="http://wiki.openstreetmap.org/wiki/Statistics#Database_Statistics_-_Graphical"&gt;a million nodes per day&lt;/a&gt;!) so hardware requirements will go up over time. If you don't have enough memory, you can still use osmosis but you will need to use a different node location store type and this process will take several days - maybe up to a week - instead of 8 hours. Anyway, here is a picture of memory usage as this process ran:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/-DYT4FLeWj5M/TZPeFNeb7CI/AAAAAAAAAVQ/6chrLh30qNQ/s1600/osmosis_total_memory.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="184" src="http://1.bp.blogspot.com/-DYT4FLeWj5M/TZPeFNeb7CI/AAAAAAAAAVQ/6chrLh30qNQ/s640/osmosis_total_memory.png" width="610" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;The initial downhill slide is where osmosis reads all the nodes into memory. After that, memory usage is relatively constant as it processes ways and relations using the nodes it has in memory. The first upwards bump in the graph is when I closed Starcraft 2 and went to bed :)&lt;br /&gt;&lt;br /&gt;One tip if you are doing this across the network like I was: have your output directory be a local disk or even an external USB drive. My windows system didn't have enough disk space so at first I tried putting the output on a network mounted drive that pointed to my file server which is running a samba share. Bandwidth was not a problem but network latency killed performance. Switching to an external USB drive almost tripled the speed at which osmosis processed objects. The planet file can reside on the network. But the output directory needs to be something local.&lt;br /&gt;&lt;br /&gt;Here are the resulting files after this 8 hour long memory sucking excursion:&lt;br /&gt;&lt;br /&gt;&lt;pre class="source-code"&gt;nodes.txt            105 GB&lt;br /&gt;relation_members.txt 231 MB&lt;br /&gt;relations.txt        165 MB&lt;br /&gt;users.txt              2 MB&lt;br /&gt;way_nodes.txt         26 GB&lt;br /&gt;ways.txt              79 GB&lt;br /&gt;---------------------------&lt;br /&gt;total:               210 GB&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;&lt;br /&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;&lt;b&gt;Loading into postgres &lt;/b&gt;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;Now for the long part. These 6 files need to be loaded into a database. Luckily there is another script in the osmosis distribution that will assist you. However it will need some modifications. The file is ~/osmosis/&lt;i&gt;script/pgsnapshot_load_0.6.sql&lt;/i&gt;. If you open this file you will see that it first drops all constraints and indexes to make the mass inserts go faster. Then it actually imports the data, rebuilds indexes and constraints and finishes up with some database maintenance. The first thing I did was enable query timing just so I could see how long things took. To do this, insert the following line. All of the index/constraint drops will be instantaneous so you can put it after those:&lt;br /&gt;&lt;pre class="source-code"&gt;\timing&lt;/pre&gt;&lt;br /&gt;Next, comment out the next two lines that start with &lt;i&gt;SELECT DropGeometryColumn&lt;/i&gt;. The bbox one isn't needed for the xapi and, assuming you followed my advice above about building the linestring in osmosis, that should already be there. You will also need to comment out the two &lt;i&gt;SELECT AddGeometryColumn&lt;/i&gt; statements and the two bigger UPDATE queries below that. These sections are all preceded by comments indicating that they may need to be commented out. To comment them out, just add two dashes at the beginning of the line.&lt;br /&gt;&lt;br /&gt;I am also unconvinced as to the necessity of the CLUSTER queries. I let the one for the nodes table run and it took almost 3 days. Then I lost power during a spring thunderstorm while the one on the ways table was running. I didn't go back and re run it and my database still seems to be working fine. From what I learned in the &lt;a href="http://www.postgresql.org/docs/8.4/interactive/sql-cluster.html"&gt;postgres documentation on the subject&lt;/a&gt;, in theory it makes things faster by copying the entire table in an ordered fashion so that nodes that are geographically near each other are also close to each other in the database. This can increase the chances that a single disk read operation will load many of the nodes that you need into memory at once. In practice my use of the database via the XAPI tends to touch a wide area at once so I doubt I'm seeing much benefit from this. You will also see that the postgres documentation states that this needs to be run on a regular basis to maintain the theoretical benefit. As objects are added/deleted/changed, the clustering is slowly lost. And the CLUSTER query puts an exclusive lock on the table so your database will effectively be offline for at least a day any time you do this. So feel free to comment these out as well if you want to save some time. You can always run them by hand later if you determine that they are actually needed.&lt;br /&gt;&lt;br /&gt;I also did not run the VACUUM command. &lt;a href="http://www.postgresql.org/docs/8.4/interactive/sql-vacuum.html"&gt;VACUUM &lt;/a&gt;is a database maintenance task that compacts your tables to remove "holes" left by deleted and changed records. In my case, the CLUSTER query completely copied my nodes table, so I don't think that a VACUUM immediately following this would have done anything. Even without the CLUSTER, I'm not sure it would have done much to a brand new database. I may be wrong about that but I have yet to see any adverse effects from not running it. Again, you can always run it by hand later - and may need to do so periodically anyway.&lt;br /&gt;&lt;br /&gt;However it is very important to run the ANALYZE query! This will cause postgres to do a quick (10 minutes or so) analysis of all the tables in the database and will store some basic information that postgres can use later to optimize your joins. I initially didn't do this either and some simple queries I tried looked like they were going to take days to finish. After running the ANALYZE query, they finished in seconds. It is &lt;b&gt;THAT &lt;/b&gt;important!&lt;br /&gt;&lt;br /&gt;But back to the import process itself. Now that you have tweaked the &lt;i&gt;pgsnapshot_load_0.6.sql&lt;/i&gt; file, it is time to wield it. Change to the directory where the .txt files from osmosis are. Then execute:&lt;br /&gt;&lt;pre class="source-code"&gt;psql -U xapi -d xapi -f ~/osmosis/script/pgsnapshot_load_0.6.sql&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;At this point you should go to sleep or spend time with your family or something. Here are the timings from my load:&lt;br /&gt;&lt;pre class="source-code"&gt;users.txt:            20 seconds&lt;br /&gt;nodes.txt:            13 hours&lt;br /&gt;ways.txt:             20 hours&lt;br /&gt;way_nodes.txt:         2 hours&lt;br /&gt;relations.txt:        40 seconds&lt;br /&gt;relation_members.txt: 80 seconds&lt;br /&gt;&lt;br /&gt;Creating indexes:&lt;br /&gt;node primary key:              1.5 hours&lt;br /&gt;way primary key:               30 minutes&lt;br /&gt;way_nodes primary key:         1.5 hours&lt;br /&gt;relations primary key:         40 seconds&lt;br /&gt;relation_members primary key:  50 seconds&lt;br /&gt;nodes geometry index:          25 hours&lt;br /&gt;way_nodes node ID index:       2 hours&lt;br /&gt;realtion_members member index: 45 seconds&lt;br /&gt;ways bounding box index:       1.7 hours&lt;br /&gt;way linestring index:          1.6 hours&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;This ads up to about 70 hours. And that's without the CLUSTER query. If something goes wrong, you can truncate the tables (only the 6 affected by the load... leave the other GIS tables alone!) and then just rerun the script. It will spit out errors when it tries to drop the indexes at the beginning because they don't exist but that is fine.&lt;br /&gt;&lt;br /&gt;You can monitor the progress of the \copy commands in linux by using  &lt;br /&gt;&lt;pre class="source-code"&gt;lsof -o -p &amp;lt;pid&amp;gt; &lt;/pre&gt;where &amp;lt;pid&amp;gt; is the process ID of the pgsql process. It will give you a readout the position of the current read within the file in bytes - in hex. Convert to decimal and divide by 1073741824 to get it in gigabytes.&lt;br /&gt;&lt;br /&gt;If you are tight on space, it is safe to delete or compress the input files after pgsql has finished the \copy command on that file.&lt;br /&gt;&lt;br /&gt;Here is a pretty picture of my disk usage during the import process:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/-fQ-ZbxcoLog/TZju3ENW5MI/AAAAAAAAAVU/7CsDA3JQscU/s1600/pgsnapshot_disk.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="216" src="http://4.bp.blogspot.com/-fQ-ZbxcoLog/TZju3ENW5MI/AAAAAAAAAVU/7CsDA3JQscU/s640/pgsnapshot_disk.png" width="610" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;As you can see, this covers a 6 day period. 3 days (from some time on the 16th to pretty much the end) is the CLUSTER query that I'm not sure is needed. The occasional drops in space usage towards the end is me deleting old files from previous osmosis test runs to make sure it didn't fill up the partition. The big drop at the end is where it finally finished the CLUSTER query and was able to drop the temporary nodes table. Total partition size is 670 GB.&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;&lt;b&gt;Conclusion&lt;/b&gt;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;And that's it! Once that process finishes, you will have a complete pgsnapshot database containing the OpenStreetMap database! As I mentioned up towards the top, one use for this is to run your own XAPI server to do specific map queries however you could also use this database to do other things with the OSM data. Depending on what you are doing, you may need to set osmosis up to keep your database in sync with new edits. Obviously by the time this process finishes, the data will already be several days out of date. I will cover the syncing process in a separate post soon.&lt;br /&gt;&lt;br /&gt;The total processing time for me was about 80 hours. To make that at least somewhat meaningful I should list the specs of the computer doing all the importing. I already mentioned that the computer I ran osmosis on was a quad core with 12 GB of memory. The computer that is running the database and that actually ran the import process is a little older. It has an AMD 4400+ (first generation dual core) with 3 GB of memory. But the most important part for the database processes is disk. Mine aren't really spectacular: 3 1TB Samsung Spinpoint F1 drives (7,200 RPM) in a software RAID-5 (so total of 2TB usable storage). I am also running LVM on top of the RAID which splits the space into two partitions. Pretty much all of the database related processes are slowed down by disk access times. Faster disks or a better RAID setup (0 or 10) would drop these times. Or SSDs. Donations welcome :)&lt;br /&gt;&lt;br /&gt;As I said above, the total size of the database after it is all said and done is 430 GB. However if you execute the CLUSTER query on the nodes table, you will need an additional 180 GB or so because it makes a complete copy of the nodes table. Plus remember the temporary files created by osmosis need another 210 GB although they can be deleted/moved/compressed as the load script finishes using them.&lt;br /&gt;&lt;br /&gt;&lt;strike&gt;Check back soon (or subscribe!) to catch&lt;/strike&gt; &lt;a href="http://ksmapper.blogspot.com/2011/04/keeping-database-up-to-date-with.html"&gt;Check out my next post&lt;/a&gt; about keeping the database up to date.&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/XKRMsvpLI-A" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/04/parse-planet-into-pgsnapshot-database.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://1.bp.blogspot.com/-DYT4FLeWj5M/TZPeFNeb7CI/AAAAAAAAAVQ/6chrLh30qNQ/s72-c/osmosis_total_memory.png" height="72" width="72" /><thr:total>6</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-3305393165724139696</guid><pubDate>Wed, 02 Mar 2011 06:11:00 +0000</pubDate><atom:updated>2011-03-02T00:11:18.972-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">mapping</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">TIGER</category><title>Dual Carriageways</title><description>When I first started editing OSM, I was mostly doing residential roads and bike trails. Eventually I worked my way to major roads. Some of them had dual carriageways. In my case these were also US/State highways. Somehow I got the two mixed up in my head and thought that mapping any dual carriageway road required relations. Since I didn't know much about relations at the time I was kind of scared to touch them. Turns out relations ARE used to describe highways however they are not required for dual carriageway roads.&lt;br /&gt;&lt;br /&gt;Since I've recently been working on "dualizing" roads in the Kansas City area, I thought I would throw up a quick post about the process. It has probably already been done (I seem to recall a "TIGER fixup" tutorial that described the process) but here is my version.&lt;br /&gt;&lt;br /&gt;To Start off with, let's define a dual carriageway road. There may be some difference of opinion (there usually is in OSM) but in general it is any road where the opposing streams of traffic are separated by a &lt;b&gt;physical barrier.&lt;/b&gt;&amp;nbsp;A grass median (think interstates) or a strip of trees in the middle of a boulevard are prime examples. Sometimes the barrier is just a small strip of raised pavement or bricks. Regardless, it is something that would make your car go "bump" or even cause damage if you were to try and drive through/over it.&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;The Census TIGER data that was imported for all of the US just has a single centerline for any given road. The Census Bureau doesn't care about routing topology. They just want basic road centerlines that they can use to carve the country up into their tracts for statistical purposes.&amp;nbsp;There is a &lt;i&gt;tiger:separated&lt;/i&gt; tag that I presume is supposed to indicate dual carriageway roads but I have only seen this set to &lt;i&gt;yes&lt;/i&gt; a handful of times so as far as I am concerned, that tag can be ignored.&lt;br /&gt;&lt;br /&gt;So why bother splitting things up? Well for interstates it is pretty obvious. The ways are far enough apart from each other that you really need to be able to distinguish between the two ways for navigation purposes. They may also be part of different routes (where highways overlap interstates) or have other differentiating characteristics. But even on smaller roads, it can be important detail. For example, see this intersection in &lt;a href="http://www.openstreetmap.org/?lat=38.9668&amp;amp;lon=-94.6764&amp;amp;zoom=12&amp;amp;layers=M"&gt;Overland Park&lt;/a&gt; (part of the Kansas City metro area):&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="https://lh4.googleusercontent.com/-CVL1sXHOOh4/TW22gfLhZzI/AAAAAAAAAUg/zZ6TlBMXDuE/s1600/dual_before.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="263" src="https://lh4.googleusercontent.com/-CVL1sXHOOh4/TW22gfLhZzI/AAAAAAAAAUg/zZ6TlBMXDuE/s320/dual_before.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;This is obviously going to cause problems for routing and is likely to set off a flurry of annoying "recalculating" messages in navigation devices if you are trying to go east from that residential road.&lt;br /&gt;&lt;br /&gt;Medians in the middle of roads are also a visual navigation aid for drivers so having the map show where the road is split is a good thing.&lt;br /&gt;&lt;br /&gt;Here is another short section of road that I will convert to a dual carriageway. In this case it won't really affect routing very heavily but if we are going to have the best map in the world then we need details!&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="https://lh6.googleusercontent.com/-JG4hN1sNhqA/TW24l7sZJvI/AAAAAAAAAUk/QIk_mzFp9kI/s1600/dc_before.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="272" src="https://lh6.googleusercontent.com/-JG4hN1sNhqA/TW24l7sZJvI/AAAAAAAAAUk/QIk_mzFp9kI/s400/dc_before.png" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;The first task is to split the way at the beginning and end of the dual carriageway section and mark the section as a one-way street using the oneway=yes tag. Now adjust the geometry to fit into one lane or the other.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="https://lh3.googleusercontent.com/-n2gjo3MExZ4/TW24n_bODsI/AAAAAAAAAUo/kT8pgLAVyhc/s1600/dc_1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="272" src="https://lh3.googleusercontent.com/-n2gjo3MExZ4/TW24n_bODsI/AAAAAAAAAUo/kT8pgLAVyhc/s400/dc_1.png" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;You will see that JOSM (as of the style update last week!) puts black arrows along the way to indicate its one-way-ness. Potlatch does something similar.&amp;nbsp;You may need to reverse the direction of the way to make sure the arrows are pointing in the right direction.&lt;br /&gt;&lt;br /&gt;The next step is to draw in the other side of the road. Then copy the highway=* classification as well as the name and the oneway=yes tag (and possibly other tags, as appropriate) from the old way to the new one. To reduce typo errors, I usually just copy the old way and then paste the tags onto the new one using CTRL+SHIFT+V in JOSM. Then I remove any tags I don't want to duplicate, like all the &lt;i&gt;tiger:&lt;/i&gt; tags since my new way is not TIGER data.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="https://lh3.googleusercontent.com/-rpJjdYFgQbE/TW24vzF6niI/AAAAAAAAAUw/IIPXV28H5_M/s1600/dc_3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="272" src="https://lh3.googleusercontent.com/-rpJjdYFgQbE/TW24vzF6niI/AAAAAAAAAUw/IIPXV28H5_M/s400/dc_3.png" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;You will notice that they look slightly different. The old way still is still tagged with &lt;i&gt;tiger:reviewed=no&lt;/i&gt;&amp;nbsp;but the new one doesn't have a tiger:reviewed tag. JOSM renders an extra casing around unreviewed TIGER roads to highlight them. I usually only remove this tag if I have done a thorough survey on the ground. In this case I am just using aerial imagery so I will leave it alone. I also usually try to tag the new way I created with a source&amp;nbsp;tag (&lt;i&gt;source=bing&lt;/i&gt; in this case) to indicate where I got this information from. That way if someone has a source that they know is better or more up to date than bing, they can update my work in the future.&lt;br /&gt;&lt;br /&gt;Finally, I need to fix the connecting road so that the topology is still correct. In this case it actually involves dualizing that road as well.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="https://lh6.googleusercontent.com/-4bdd-AIBGT0/TW24zywFVvI/AAAAAAAAAU0/F8Q5xaUcR3U/s1600/dc_after.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="272" src="https://lh6.googleusercontent.com/-4bdd-AIBGT0/TW24zywFVvI/AAAAAAAAAU0/F8Q5xaUcR3U/s400/dc_after.png" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;In this case I brought the intersecting road to a point on the north way. Depending on the situation I sometimes do this or leave them straight and have two intersection points instead of one. Here I felt like using a single node more closely matched the path a car would actually take to turn from one street onto the other. There is always a balance to be struck between topology, geometry and level of detail.&lt;br /&gt;&lt;br /&gt;And that's it. Nothing too complicated really. TL;DNR:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;Split way, tag with oneway=yes&lt;/li&gt;&lt;li&gt;Draw new way&lt;/li&gt;&lt;li&gt;Copy name, highway and oneway tags to new way&lt;/li&gt;&lt;li&gt;Fix intersecting roads&lt;/li&gt;&lt;li&gt;PROFIT! (optional)&lt;/li&gt;&lt;/ul&gt;On the miles long roads I've been working on in Kansas City I often make several passes back and forth, mapping additional things like traffic lights (highway=traffic_signals) which can be identified from pavement markings and shadows on the ground (obviously only if imagery is good enough).&lt;br /&gt;&lt;br /&gt;Some intersections take a bit of work. Here is a pretty complex one involving US 69, College Boulevard, on/off ramps and a couple of bridges for good measure.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="https://lh5.googleusercontent.com/-NMveoBHr28c/TW3EWtr9ELI/AAAAAAAAAU4/1Jlk4YXiMyE/s1600/dc_motorway.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="331" src="https://lh5.googleusercontent.com/-NMveoBHr28c/TW3EWtr9ELI/AAAAAAAAAU4/1Jlk4YXiMyE/s640/dc_motorway.png" width="640" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Here is a simple one but it involves a smaller road that splits right before it intersects the big road:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="https://lh5.googleusercontent.com/-ysntuuMD_PU/TW3GIb06UvI/AAAAAAAAAU8/HyaN_R_EeUo/s1600/dc_residential.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="320" src="https://lh5.googleusercontent.com/-ysntuuMD_PU/TW3GIb06UvI/AAAAAAAAAU8/HyaN_R_EeUo/s320/dc_residential.png" width="276" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;And as the last image for this post, I give you the rendered version of the area I used as an editing example:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="https://lh6.googleusercontent.com/-cch9oXtiknQ/TW3YWIQACCI/AAAAAAAAAVA/aWZGIJVi6lg/s1600/dc_rendered.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="275" src="https://lh6.googleusercontent.com/-cch9oXtiknQ/TW3YWIQACCI/AAAAAAAAAVA/aWZGIJVi6lg/s320/dc_rendered.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;That's all!&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/fz1dXYYCsoI" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/03/dual-carriageways.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://lh4.googleusercontent.com/-CVL1sXHOOh4/TW22gfLhZzI/AAAAAAAAAUg/zZ6TlBMXDuE/s72-c/dual_before.png" height="72" width="72" /><thr:total>0</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-4645768395932351645</guid><pubDate>Tue, 01 Feb 2011 03:43:00 +0000</pubDate><atom:updated>2011-01-31T21:43:16.689-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">airport</category><category domain="http://www.blogger.com/atom/ns#">mapping</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><title>Airports, Part 2</title><description>In my &lt;a href="http://ksmapper.blogspot.com/2011/01/airports-part-1.html"&gt;first installment&lt;/a&gt; of the airport series I presented some basic mapping techniques I used while mapping rural airports in Kansas. One comment I got on IRC suggested using a slightly different tagging scheme for displaced thresholds. The suggestion (I believe it was from pnorman) was to tag it with two tags. One being aeroway=runway and the other being runway=displaced_threshold. While I haven't gone back and changed my previous work, this may be a better way of doing it. First of all, they would get rendered on the default map. I know, I know... "&lt;a href="http://wiki.openstreetmap.org/wiki/Tagging_for_the_renderer"&gt;don't tag for the renderer&lt;/a&gt;!" But here's the thing: This is technically not incorrect. Displaced thresholds are considered part of the runway and are included when the runway length is specified. So tagging it as a runway but then specifying that it is a specific type of runway is not incorrect tagging. On standard maps it would be rendered as part of the runway but someone making an aeronautical themed map could use the runway=displaced_threshold tag to differentiate it and produce a customized rendering.&lt;br /&gt;&lt;br /&gt;Before I go on, I am just going to repost a link to the OSM wiki page for &lt;a href="http://wiki.openstreetmap.org/wiki/Airport"&gt;Airports &lt;/a&gt;as a reference.&lt;br /&gt;&lt;br /&gt;Now on to some new material. I was moving west-to-east through Kansas so the first "big deal" airport I came to was the Salina Municipal Airport. Its 17/35 runway is a massive 12,300 feet (3.8 kilometers) long. Any aircraft currently in service could land here with room to spare. Why does a municipal airport in a city of under 50,000 people in the middle of Kansas have such a massive runway? The military, of course! This facility was built at the beginning of World War II and was used to train heavy bomber crews. Then during the cold war it became part of Strategic Air Command operations until it was closed and turned over to the city of Salina in 1965. (source: &lt;a href="http://en.wikipedia.org/wiki/Schilling_Air_National_Guard_Base"&gt;wikipedia&lt;/a&gt;) Now it serves mostly general aviation but is also used by Kansas State University's aviation program which is based at the Salina campus. A few years ago it was used by Steve Fossett as the start and end point of his solo, nonstop flight around the globe. And I got to bill them for the telecommunications services they used while doing it! (My current day job deals a lot with the telecom billing system at KSU)&lt;br /&gt;&lt;br /&gt;I promise, I'll get to the mapping in this post, eventually. But I should point out that this is one thing I love about contributing to OSM. Seeing something unusual in aerial imagery can often lead to an hour spent on wikipedia or a fun discussion on IRC involving people from around the world or, occasionally, a field trip to go see what the heck is happening "on the ground" to cause it to show up a certain way in aerial imagery.&lt;br /&gt;&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;Back to Salina: I had two sets of aerial imagery that conflicted slightly about the length of one of the runways. (USGS and Bing) Obviously the images were taken at different times. But which was newer? Bing imagery has metadata that is sent with each tile that can sometimes tell you when it was taken. In JOSM you can display this metadata by right clicking without moving the mouse. A menu pops up with a "Show Tile Info" option. Unfortunately in most of the areas I have worked in, it usually says something like it did in this case: "capture-date: 2/1/1995-3/15/2007." So that's useless. While reading the &lt;a href="http://en.wikipedia.org/wiki/Salina_Municipal_Airport"&gt;wikipedia page&lt;/a&gt; about the Salina airport,&amp;nbsp;I noticed a link down in the "External links" section pointing to an FAA "&lt;a href="http://naco.faa.gov/d-tpp/1101/00362AD.PDF"&gt;airport diagram&lt;/a&gt;" in a PDF. I clicked on it... and just like that I had found a new source of data to use while mapping! Since this is federal government data that is intended for public distribution, I believe it is public domain so license isn't an issue. And yes, this is only for airports in the US. My apologies to the rest of the world.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;/div&gt;These diagrams are constantly updated by the FAA. They have the basic layout of the airport. Turns out the Salina airport shortened the 12/30 runway for some reason so that answered my aerial imagery question. But the diagram also has information about the taxiways. Unused ones are X'd out and the ones that are still in use have their names specified. That can go into the ref=* tag of the taxiways just like runway numbers. One thing to note however is that these are "diagrams" not "maps." So they can not be relied on for accurate cartography. Buildings are usually just squares and do not represent the true proportions or shape of the building. So use with caution - just like any other external data source.&lt;br /&gt;&lt;br /&gt;Here is the &lt;a href="http://www.openstreetmap.org/?lat=38.79443&amp;amp;lon=-97.64998&amp;amp;zoom=15&amp;amp;layers=M"&gt;Salina airport&lt;/a&gt; with its enormous runway:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/_RA5iXwXMupA/TUdUkRV9aSI/AAAAAAAAATs/mPdZLbHAC-8/s1600/salina_map.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="320" src="http://3.bp.blogspot.com/_RA5iXwXMupA/TUdUkRV9aSI/AAAAAAAAATs/mPdZLbHAC-8/s320/salina_map.png" width="214" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Other information that can be taken from the FAA PDF files include location of the control tower and runway width which can go in a width=* tag. Looking at the contents of existing width tags, it doesn't look like there is currently any support for specifying units so I guess everything should probably be in meters. (Everything should be in meters anyway. Everywhere. But that's another debate for another blog)&lt;br /&gt;&lt;br /&gt;It seems there are two ways to map the control tower. One being aeroway=control_tower and the other (documented on the wiki) being man_made=tower/service=aircraft_control. Currently they seem to have about the same usage (both&amp;nbsp;minuscule&amp;nbsp;- about 50 instances) but I think I personally prefer aeroway=control_tower. That keeps airport features together in the same tag. Also, man_made=tower seems to be more for things like cell phone or radio towers. An aircraft control tower is typically a building that happens to be tall and sometimes skinny.&lt;br /&gt;&lt;br /&gt;As I said above, I found this particular FAA diagram via Wikipedia. When I went back to try and find diagrams for other airports I ran into all kinds of old information, broken links and just plain bad design on faa.gov. So to save you the pain, here is a link directly to where you can start a search for airports in your state:&amp;nbsp;&lt;a href="http://avn.faa.gov/index.asp?xml=aeronav/applications/d_tpp"&gt;http://avn.faa.gov/index.asp?xml=aeronav/applications/d_tpp&lt;/a&gt;&amp;nbsp;First you have to select the date range, then click on a state. Then you can search by airport code, city name or airport name. Not every airport has a diagram but certainly all major ones do. Go forth and tag!&lt;br /&gt;&lt;br /&gt;Another interesting document from the FAA is&lt;a href="http://www.faa.gov/air_traffic/publications/atpubs/aim/Chap2/aim0203.html"&gt;&amp;nbsp;Airport Marking Aids and Signs&lt;/a&gt;. This is just a document showing exactly what all the little lines/blocks/arrows/signs/etc at an airport mean.&lt;br /&gt;&lt;br /&gt;The next major airport I hit was Mid-Continent Airport in Wichita. It already had some pretty good coverage but I added ref=* tags for all the runways and taxiways using the FAA diagrams. The Wichita airport has a few more taxiways than usual because it has both Learjet and Cessna facilities adjacent to the airport and they are all connected by taxiways. &amp;nbsp;Because the previous mapper also used the source=* tag, I knew it was based on Yahoo imagery so I also improved the level of detail according to the high resolution Bing imagery available here.&lt;br /&gt;&lt;br /&gt;Two new features I identified at this airport: parallel runways and terminal buildings. If runways are named based on magnetic heading then what happens for parallel runways? Turns out this is handled pretty simply. Just tack "R" and "L" to the end of it based on which one is to the right and which one is to the left as you are approaching the runway to land. So one runway is named 1L/19R and the other 1R/19L.&lt;br /&gt;&lt;br /&gt;Terminal buildings aren't too hard either. In this case, there is a main airport building and two wings that extend to the east and west. These wings are where the gates are. Since I have been to this airport before, I know they are simply referred to "East Concourse" and "West Concourse." The previous mapper had done it as a single huge polygon with a building=yes tag. I decided to split it up int 3 parts so that I could name the concourses. I added an aeroway=terminal tag to the two concourses. Technically I believe there is a difference between a "terminal" and a "concourse" but the distinction seems a little hazy in common usage and depending on airport design. OSM doesn't seem to have anything mapped as a concourse so I just stuck with terminal. Anyway, here is the finished product:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/_RA5iXwXMupA/TUYQ1IPQYUI/AAAAAAAAATU/29-VpA4rST8/s1600/ICT.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="284" src="http://1.bp.blogspot.com/_RA5iXwXMupA/TUYQ1IPQYUI/AAAAAAAAATU/29-VpA4rST8/s320/ICT.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;The couple of times I have embarked on bigger projects like this on OSM, I tend to start out mapping basics. As I go I find more details to map and by the end I'm doing some serious micro-mapping and worrying about sub-meter offsets in aerial imagery. In this case, these tendencies were worsened by the increasingly good resolution of aerial imagery as I moved into the urban areas of eastern Kansas. The last airport I touched was Kansas City International airport. Yes, it is in Missouri. But it is in the part that was robbed from Kansas by the pesky meandering of the Missouri river. Also, it was within the simple square bounding box I used to download airport nodes from the XAPI.&lt;br /&gt;&lt;br /&gt;Again, a lot of features were already basically there but were traced from old Yahoo imagery. Having flown out of this airport multiple times, I was able to use my own local knowledge to augment the aerial imagery. For example, the original imagery had a good bit of radial distortion and it was hard to distinguish the terminal buildings from the large overhang that covers the sidewalk in front of them so the buildings were lopsided and not uniform like I know them to be in reality. Here is a "before" shot I got from OpenCycleMap before it had a chance to re-render:&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/_RA5iXwXMupA/TUZJbidfL8I/AAAAAAAAATY/_0R3jrwGI9I/s1600/KCI_before.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="252" src="http://4.bp.blogspot.com/_RA5iXwXMupA/TUZJbidfL8I/AAAAAAAAATY/_0R3jrwGI9I/s320/KCI_before.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Not horrible at this zoom level but when viewing it in JOSM with z21 aerial imagery, it looked a lot worse. I found this airport to be kind of fun to work on and ended up adding more and more detail over a period of a couple of days. With large operations like this and high resolution imagery, you can pretty much add as much detail as you want, especially if you have some local knowledge to add to the imagery. So here is what I did with the terminals:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/_RA5iXwXMupA/TUb0dvs4EpI/AAAAAAAAATc/Gihi5wqzsJI/s1600/KCI_terminal_closeup.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="249" src="http://2.bp.blogspot.com/_RA5iXwXMupA/TUb0dvs4EpI/AAAAAAAAATc/Gihi5wqzsJI/s320/KCI_terminal_closeup.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;I used the architecture to my advantage and placed nodes in the&amp;nbsp;columns&amp;nbsp;of the building where straight segments connect at an angle to make a curved building. Here you can clearly see the large overhang in front of the building that somewhat confused the previous mapper.&lt;br /&gt;&lt;br /&gt;One area that wasn't mapped very well were the long term parking lots north of the airport as well as the buses that take you from long term parking to the terminals. Again, my local knowledge combined with the high resolution aerial imagery let me pick out a lot of features. For example, bus stops, tagged highway=bus_stop and shelter=yes since they are small enclosed huts to protect travelers from the wind that usually blows from some direction around here.&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/_RA5iXwXMupA/TUb2SDlBffI/AAAAAAAAATg/0kn_i_b4nW0/s1600/KCI_bus_stop.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="317" src="http://2.bp.blogspot.com/_RA5iXwXMupA/TUb2SDlBffI/AAAAAAAAATg/0kn_i_b4nW0/s320/KCI_bus_stop.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Then of course there are the parking aisles (highway=service/service=parking_aisle) which in this case can also be tagged as oneway=yes since you can clearly see the angled parking stalls. Also, there are dedicated bus service roads leading from the parking lots to the main road to the airport. Access to these roads is restricted to buses only and enforced by lift gates. The roads are tagged &amp;nbsp;tagged&amp;nbsp;highway=service/access=private and the lift gates,&amp;nbsp;barrier=lift_gate/access=private. This is where a restricted bus route enters a parking area through a lift gate:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/_RA5iXwXMupA/TUb41K9eI2I/AAAAAAAAATk/zqC6KC5NnGE/s1600/KCI_liftgate.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="310" src="http://4.bp.blogspot.com/_RA5iXwXMupA/TUb41K9eI2I/AAAAAAAAATk/zqC6KC5NnGE/s320/KCI_liftgate.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Then of course there are the parking areas themselves. These were already present but I refined them and added some more tags. Currently they are tagged with this: amenity=parking/parking=surface/fee=yes to indicate that this is a surface (as opposed to underground or garage) parking lot and that a fee is charged for parking here. The result is this map of the long-term parking area:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/_RA5iXwXMupA/TUb6JGo2CDI/AAAAAAAAATo/7pmNnLOuBpM/s1600/KCI_parking_map.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="233" src="http://3.bp.blogspot.com/_RA5iXwXMupA/TUb6JGo2CDI/AAAAAAAAATo/7pmNnLOuBpM/s400/KCI_parking_map.png" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;One useful thing I need to add is either a name or ref tag for the parking areas. I know they are named to&amp;nbsp;correspond&amp;nbsp;with the terminals (A, B, C) and you are supposed to park in the lot that is named the same as the terminal you will be flying out of (for bus routing reasons) but I don't recall which is which off the top of my head. Guess it's time for a trip to some place exotic so I can survey the airport parking lot on the way there.&lt;br /&gt;&lt;br /&gt;The short term parking in the center of the terminal buildings can be tagged similarly but with "parking=multi-storey" instead of "surface" to indicate that these are multi-level parking garages. They aren't rendered any differently though. There are also walkways that cross a small bridge from the top of the parking garage to the streets in front of the terminal buildings. Here is a view of one of the terminals:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/_RA5iXwXMupA/TUd1cNnvvWI/AAAAAAAAAT4/jHaFSYaq6TM/s1600/kci_terminal_B.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="320" src="http://1.bp.blogspot.com/_RA5iXwXMupA/TUd1cNnvvWI/AAAAAAAAAT4/jHaFSYaq6TM/s320/kci_terminal_B.png" width="299" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Notice the "B" in the middle of the parking area. That is the name of this terminal building. This is kind of a bug in the Mapnik rendering software in my opinion. I believe it uses a "center of gravity" algorithm for determining where to place polygon labels. In this case, the center of gravity of the building is outside of the building outline. Oops.&lt;br /&gt;&lt;br /&gt;The last bit of micro-mapping involved the grassy areas between the runways/taxiways. The wiki page mentions that they can be mapped as landuse=grass so I thought I'd give it a shot and see how it turned out. The result seems pleasing to the eye however it is a fair a mount of effort. Although I personally find mapping just about anything at zoom level 21 is kind of fun. Like this:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/_RA5iXwXMupA/TUdp--Z8jfI/AAAAAAAAATw/w-d2WSWeQCw/s1600/kci_windsock.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="185" src="http://3.bp.blogspot.com/_RA5iXwXMupA/TUdp--Z8jfI/AAAAAAAAATw/w-d2WSWeQCw/s200/kci_windsock.png" width="200" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Yes, I found a windsock in aerial imagery. And of course there is a tag for it! aeroway=windsock - and there are currently 280 of them in the database. JOSM even renders a little red/white striped cone on a little stick! However it is not rendered in the default map styles on osm.org.&lt;br /&gt;&lt;br /&gt;After adding a few more features like fences where I could clearly make them out (barrier=fence) and gates (barrier=gate) as well as the drainage ditches, (waterway=drain) here is the &lt;a href="http://www.openstreetmap.org/?lat=39.30273&amp;amp;lon=-94.71051&amp;amp;zoom=15&amp;amp;layers=M"&gt;Kansas City International Airport&lt;/a&gt;:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/_RA5iXwXMupA/TUdrR1LqSYI/AAAAAAAAAT0/6oKSVqKoC2I/s1600/kci_map.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="320" src="http://1.bp.blogspot.com/_RA5iXwXMupA/TUdrR1LqSYI/AAAAAAAAAT0/6oKSVqKoC2I/s320/kci_map.png" width="233" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;The main thing I'm not really happy about is the road classification. When I started, everything in the area was tagged as highway=residential, as imported from TIGER. I upgraded the major road coming in to the airport to trunk since it is basically built like an interstate. Even some of the roads inside of the security perimeter of the airport had been imported from TIGER. These I changed to highway=service and added access=private. The main circle that goes to all three terminals is marked as tertiary. This all seems fine. The terminal roads themselves are very low speed (I think the sign says 15 mph) but also very wide. There are two 3-lane roads separated by an island in the middle for taxis and hotel buses and such. They seem too big to be called a service road. I left them classified as residential for now but this doesn't seem quite right either. Same with the roads leading to the long term parking lots. Suggestions are of course welcome. :)&lt;br /&gt;&lt;br /&gt;One thing I haven't touched on is gates. They can be mapped by tagging a node with aeroway=gate and ref=* to indicate the gate number. I have no source of gate names/locations so for now I ignored them. I was also unsure about how to map them. One airport I looked at had lone gate nodes floating out in the apron, essentially where the airplane would sit to dock at the gate. While this kind of makes sense from the pilot's perspective, I might argue that gates should probably be mapped at the edge of the terminal building or even inside the building to make them of use to passengers wanting to find out where they need to go in the airport to find their gate.&lt;br /&gt;&lt;br /&gt;And one last note: there are a good number of adjacent features (mostly apron/terminals/landuse) that share nodes with each other right now in my work at the Kansas City airport. These should probably be converted to use multipolygons as node sharing is often frowned upon for this kind of thing and is indeed somewhat harder to deal with in some instances. That's the way it was mapped when I got there. My edits made the map better. Not perfect. Yet.&lt;br /&gt;&lt;br /&gt;And so concludes my great Kansas airport mapping adventure. Thanks for flying!&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/Z7Vyhmpwg0M" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/01/airports-part-2.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://3.bp.blogspot.com/_RA5iXwXMupA/TUdUkRV9aSI/AAAAAAAAATs/mPdZLbHAC-8/s72-c/salina_map.png" height="72" width="72" /><thr:total>9</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-2236879733022690497</guid><pubDate>Fri, 07 Jan 2011 07:47:00 +0000</pubDate><atom:updated>2011-02-01T09:58:44.149-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">airport</category><category domain="http://www.blogger.com/atom/ns#">mapping</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><title>Airports, Part 1</title><description>For the past few days I've gone on an airport binge in&amp;nbsp;&lt;a href="http://www.openstreetmap.org/"&gt;OpenStreetMap&lt;/a&gt;. I ended up learning a lot about airports. It was fun to start mapping, then come across something different or notice a new detail and realizing that this too can be mapped! Here are my musings on the subject. Feel free to tell me I did it all wrong in the comments :)&lt;br /&gt;&lt;br /&gt;The recent import of airport nodes from&amp;nbsp;&lt;a href="http://www.ourairports.com/"&gt;www.ourairports.com&lt;/a&gt;&amp;nbsp;and the subsequent discussion sparked the idea to map&amp;nbsp;easily identifiable runways from aerial imagery. So I downloaded all "aeroway=aerodrome" nodes within Kansas from the XAPI. In addition to the above mentioned import, a lot of airports were imported from USGS GNIS (Geographic Names Information System) back in early 2009. In total there were just under 600.&lt;br /&gt;&lt;br /&gt;This being Kansas, a vast majority of those 600 "airports" are nothing more than a flat spot in a field where farmer Joe lands his crop duster. They have such names as "Lundgren Hereford Ranch Airport" or "Flying N Ranch Airport." They should probably be downgraded from "aerodrome" to something else and not rendered at zoom level 10 like they are now. Right now looking at &lt;a href="http://www.openstreetmap.org/?lat=38.676&amp;amp;lon=-101.259&amp;amp;zoom=10&amp;amp;layers=M"&gt;western Kansas at z10&lt;/a&gt; reveals a ridiculous number of airports being rendered. I have yet to see any consensus on a tagging scheme for smaller airports so I mostly just ignored them. Additionally, the aerial imagery I was using (USGS NAIP) wasn't really good enough to reliably pinpoint these grass strips in the middle green fields. It also didn't help that the GNIS information is often off by quite a bit so I wasn't sure if the runway was just blending in or if there really wasn't one in the field I was looking at but rather 2 fields over.&lt;br /&gt;&lt;br /&gt;The more interesting airports are the municipal air fields. Most of these have a paved runway, sometimes in addition to a grass one. Even small airports stick out when they have a paved runway:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/_RA5iXwXMupA/TSU5rRGp71I/AAAAAAAAAS0/gGV_ldvt0QA/s1600/small_airport.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="300" src="http://2.bp.blogspot.com/_RA5iXwXMupA/TSU5rRGp71I/AAAAAAAAAS0/gGV_ldvt0QA/s320/small_airport.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;This is pretty easy to map. One runway, 3 taxiways and an apron. A quick look at the&amp;nbsp;&lt;a href="http://wiki.openstreetmap.org/wiki/Airport"&gt;Airport page&lt;/a&gt;&amp;nbsp;on the OSM wiki will tell you to use the aeroway=* tag with values of runway, taxiway and apron. However upon closer inspection, there is a little bit more information that can be extracted:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/_RA5iXwXMupA/TSU8ZsBwVTI/AAAAAAAAAS4/MKNIqsvUv4M/s1600/small_runway.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="249" src="http://1.bp.blogspot.com/_RA5iXwXMupA/TSU8ZsBwVTI/AAAAAAAAAS4/MKNIqsvUv4M/s320/small_runway.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;That big number 17 on the pavement must mean something, right? I have known that runways have names for a long time. But I never knew what they meant.&amp;nbsp;Maybe this is already common knowledge to some people but it turns out they are based on nothing more than which direction the runway points. The runway above has a heading of 173 degrees. Round that to the nearest 10 and chop off the zero and you get 17. The exception to this naming rule is that there is no zero. Runways that point within 5 degrees of north are named 36. But since each runway has two ends, it also has two names obviously separated from each other by 180 degrees. So this is actually runway 17/35. This goes in a ref=* tag on the way that represents the runway. One catch: This is based on the &lt;b&gt;magnetic&lt;/b&gt; north pole, not the geographic one. So in theory these can change over time as the magnetic pole moves. Luckily down south here magnetic and geographic poles aren't too far apart and the magnetic pole only moves about 0.1 degrees per year. However up in northern Canada they actually do use the geographic pole for runway numbering because the declination is ridiculously large. I'm sure you will thank me for this information next time you are flying in northern Canada.&lt;br /&gt;&lt;br /&gt;Update: One of the first readers of this post&amp;nbsp;(petschge on IRC)&amp;nbsp;&amp;nbsp;pointed out that the magnetic pole shift has actually affected at least one major airport:&amp;nbsp;&lt;a href="http://www2.tbo.com/content/2011/jan/05/060831/shift-of-earths-magnetic-north-pole-impacts-tampa-/news-money/"&gt;Tampa International&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;Next, notice the lines extending across the taxiways a couple hundred feet from the edge of the runway. Kind of hard to make out in this picture but they are there. These are the holding positions where airplanes must stop and wait for permission to enter the runway. In OSM these are tagged with aeroway=holding_position. They are not rendered on the default map style but it is still useful information.&lt;br /&gt;&lt;br /&gt;Well that's done. The other thing that could be mapped is the buildings and the outline of the airport land. I did map buildings on some airports but not this one. The land area I found difficult to determine from aerial imagery. It was hard to tell if I was looking at an area of mowed grass that belonged to the airport or a field of soybeans so I left the airports as single nodes for now. The final product: (the airport name isn't actually rendered at this zoom level. That "Russell" is from the city boundary around the airport. And I just noticed now that the city boundary probably also defines the outline of the airport. Hindsight!)&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/_RA5iXwXMupA/TSaS_JCShZI/AAAAAAAAAS8/vU8XNFNTzH4/s1600/russell_airport.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="320" src="http://4.bp.blogspot.com/_RA5iXwXMupA/TSaS_JCShZI/AAAAAAAAAS8/vU8XNFNTzH4/s320/russell_airport.png" width="213" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Side note: I was surprised and somewhat disturbed by how many cemeteries I saw immediately next to airport runways in rural Kansas. It kind of makes sense from a landuse perspective. You can't put tall buildings next to a runway. Plus, the local residents won't repeatedly call city hall to complain about the noise. But still, it seems kind of disconcerting in a slightly&amp;nbsp;humorous&amp;nbsp;way.&lt;br /&gt;&lt;br /&gt;Next up:&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/_RA5iXwXMupA/TSaY2oSnmGI/AAAAAAAAATE/LSGpJi1B4uo/s1600/displaced_threshold.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="320" src="http://4.bp.blogspot.com/_RA5iXwXMupA/TSaY2oSnmGI/AAAAAAAAATE/LSGpJi1B4uo/s320/displaced_threshold.png" width="313" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Ok, we've got the runway which will be named 17/35 again. The taxiway with the holding position, check. But what's this business with the big arrows and the runway start line not being... well... at the &lt;b&gt;start&amp;nbsp;&lt;/b&gt;of the runway?! That's what I asked on IRC. User&amp;nbsp;stevensn had the answer. It is called a "displaced threshold." What's that? To&amp;nbsp;&lt;a href="http://en.wikipedia.org/wiki/Runway"&gt;Wikipedia&lt;/a&gt;! Turns out a displaced threshold is a section of the runway which can be used for planes to start their takeoff roll but it can &lt;b&gt;not&lt;/b&gt;&amp;nbsp;be used to land on. In this case, there is a prison just south of the runway. Presumably it has tall fences or walls which means the glide slope for approach to this runway is too high to safely hit the beginning of the runway.&lt;br /&gt;&lt;br /&gt;This setup also makes me&amp;nbsp;conjure&amp;nbsp;up images of a low flying airplane dropping a bungee cord into the prison yard to yank someone out. Probably impractical due to airspeed and g-force limitations but I'm sure Hollywood could stuff it into a movie somehow. If they do, I want royalties! There &lt;b&gt;have&lt;/b&gt;&amp;nbsp;however been several successful (and some failed) helicopter jailbreaks. Google it for fun some time.&lt;br /&gt;&lt;br /&gt;Anyway, back to mapping. There is no guidance on how to map displaced thresholds on the OSM wiki. Using the&amp;nbsp;&lt;a href="http://taginfo.openstreetmap.de/keys/aeroway"&gt;TagInfo&lt;/a&gt;&amp;nbsp;tool I looked around and noticed that there were about 15 existing uses of &amp;nbsp;"aeroway=displaced_threshold" tags. This seemed pretty logical to me so that's what I used. Now there are almost 40 uses of this tag. Unfortunately the renderers don't know about it so it doesn't show up on the map but as with holding positions, it is still useful information and if enough of them get mapped we might be able to convince the renderers to add support for it. The only real problem is when a taxiway connects to a displaced threshold. On the map it looks like the taxiway just ends in the middle of nowhere. But this is a rendering problem and remember, in OSM we tag for what's on the ground,&amp;nbsp;&lt;a href="http://wiki.openstreetmap.org/wiki/Tagging_for_the_renderer"&gt;not the renderer&lt;/a&gt;!&lt;br /&gt;&lt;br /&gt;And finally (for now) two more new things I learned about:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/_RA5iXwXMupA/TSa0UdW563I/AAAAAAAAATI/T80dMVMzSeM/s1600/overrun.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="299" src="http://3.bp.blogspot.com/_RA5iXwXMupA/TSa0UdW563I/AAAAAAAAATI/T80dMVMzSeM/s320/overrun.png" width="320" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;The yellow chevrons might appear to be similar to the displaced threshold but of course aviation tends to be a very precise business so each splash of paint is precisely regulated to mean something distinct. Going back to the wikipedia page on &lt;a href="http://en.wikipedia.org/wiki/Runway"&gt;runways&lt;/a&gt;, we see that these are "Blast pads, also known as overrun areas or stopways" and are constructed to take the brunt of the hot air blast when jet engines throttle up for takeoff so that the ground doesn't get eroded. Their construction may not be as robust as the rest of the runway so aircraft are prohibited from using this area for any purpose except in emergencies.&lt;br /&gt;&lt;br /&gt;Blast pads/overruns/stopways are also not documented in OSM. Once again, I looked at&amp;nbsp;&lt;a href="http://taginfo.openstreetmap.de/keys/aeroway"&gt;TagInfo&lt;/a&gt;. There were a few that are tagged as aeroway=stopway and a few more that were tagged as aeroway=overrun. So I picked the more popular one. If someone has a compelling argument that it should be stopway or blast_pad I would not object to changing it. The same issue exists with not being rendered as described above for displaced thresholds although it isn't as critical since active taxiways will not connect to this section of the runway.&lt;br /&gt;&lt;br /&gt;Lastly, note the yellow X marks to the left of the runway. This isn't too hard to figure out. It means the runway/taxiway is not in use. It could be temporary, due to construction, or (as is the case here) permanent closure with the possibility of obstacles/damage present which would make landing here dangerous. I tended to not map these. There were a few that were already mapped as active runways/taxiways so I retagged them aeroway=runway_disused/taxiway_disused which I also found used in TagInfo.&lt;br /&gt;&lt;br /&gt;I am going to cut this post off here before it gets so long that no one will ever read it. Tune in again soon to see part 2. It will focus on some of the bigger airports and some data sources that I found to help map them in more detail.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;&amp;lt;ding&amp;gt;&amp;nbsp;&lt;/b&gt;This is your Kansas mapper speaking. You are now free to move about the planet but we ask that you leave your GPS device turned on at all times. We hope you enjoy the resulting map. Thank you for choosing OpenStreetMap. Have a nice day.&lt;br /&gt;&lt;br /&gt;Update: &lt;a href="http://ksmapper.blogspot.com/2011/01/airports-part-2.html"&gt;Here is part 2&lt;/a&gt;!&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/EFFh7U695gk" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2011/01/airports-part-1.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://2.bp.blogspot.com/_RA5iXwXMupA/TSU5rRGp71I/AAAAAAAAAS0/gGV_ldvt0QA/s72-c/small_airport.png" height="72" width="72" /><thr:total>5</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-4762728961726772238</guid><pubDate>Thu, 16 Dec 2010 04:45:00 +0000</pubDate><atom:updated>2010-12-15T23:04:18.453-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">TIGER</category><category domain="http://www.blogger.com/atom/ns#">update</category><title>Updated TIGER map</title><description>&lt;i&gt;For a full history of this post you may want to read these three posts first:&amp;nbsp;&lt;a href="http://ksmapper.blogspot.com/2010/10/tiger-ways-in-kansas.html"&gt;1&lt;/a&gt;,&amp;nbsp;&lt;a href="http://ksmapper.blogspot.com/2010/10/couple-more-tiger-maps.html"&gt;2&lt;/a&gt;&amp;nbsp;and&amp;nbsp;&lt;a href="http://ksmapper.blogspot.com/2010/11/nationwide-tiger-map.html"&gt;3&lt;/a&gt;.&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;When I first presented my nationwide TIGER map&amp;nbsp;at wherecamp5280,&amp;nbsp;&lt;a href="http://blog.samat.org/"&gt;Samat&lt;/a&gt;&amp;nbsp;mentioned that he was surprised that his county wasn't more heavily edited since he has done a lot of work there. A few days ago we were chatting on IRC (#osm on oftc.net) and he asked a question that made me realize that the map did indeed have a serious flaw in it. Earlier today a &lt;a href="http://lists.openstreetmap.org/pipermail/talk-us/2010-December/004973.html"&gt;discussion &lt;/a&gt;came up on the talk-us mailing list that made me notice that the same error exists in the &lt;a href="http://open.mapquestapi.com/tigerviewer/"&gt;TIGER edited map&lt;/a&gt; that MapQuest (re)set up. So I decided to look at correcting it.&lt;br /&gt;&lt;br /&gt;The error is that I was only looking at the latest version of the ways. This means that any edits between the original import and the mass edit that expanded street names were not taken into account. Both the import and the abbreviation expansion happened well before I started mapping so I didn't even consider this fact. As Antony Pegg says later in the mailing list thread, it is too expensive to go back and look at all previous versions. Especially since you would have to use a FULL planet file since the regular ones only contain the current version of objects, not a full history.&lt;br /&gt;&lt;br /&gt;But we don't really care about the contents of the edits or who performed them... we just care that they happened. The initial import obviously created version 1 of the ways and if nothing else changed, then version 2 was created by balrog-kun in the name expansion edit. If any edits happened between the import and the name edit, then the ways will be on version 3 or higher. Thus any TIGER way with a version higher than 2 must have been edited by someone other than these users, even if one of them was the last to touch a way.&lt;br /&gt;&lt;br /&gt;So here is the updated map which takes this into consideration:&lt;br /&gt;&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;&lt;table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td style="text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/_RA5iXwXMupA/TQmGQOqPZUI/AAAAAAAAASc/0RdKH1ACiss/s1600/USA_2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"&gt;&lt;img border="0" height="308" src="http://2.bp.blogspot.com/_RA5iXwXMupA/TQmGQOqPZUI/AAAAAAAAASc/0RdKH1ACiss/s400/USA_2.png" width="400" /&gt;&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td class="tr-caption" style="text-align: center;"&gt;Click for bigger version&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;br /&gt;There are indeed several noticeable differences. For one, Samat's county (Dona Ana, NM) turned two shades lighter! Also, a large chunk of Wyoming came out of the "virtually untouched" classification. The southern tip of Texas as well as the Dallas area lightened up a bit as well. But at the end of the day, there is still a &lt;b&gt;LOT&lt;/b&gt; of untouched data out there.&lt;br /&gt;&lt;br /&gt;This map still isn't perfect. For example, if someone split an imported way, creating a new version 1 way as well as version 2 of the existing one and then the name change edit happened, the new way would be counted as not having been touched by a human. Also, people deleting the tiger:county tag on ways will obviously throw numbers off a little. But this should be much better at least. I'm hoping this change can be incorporated into the MapQuest version as well.&lt;br /&gt;&lt;br /&gt;For giggles I threw the data up on&amp;nbsp;&lt;a href="http://www.openheatmap.com/view.html?map=InsinuationMaritzasAurelius"&gt;OpenHeatMap&lt;/a&gt;&amp;nbsp;as well. I'm not sure what happened to Alaska and Louisiana. For some reason OpenHeatMap didn't recognize the county names. It also did its own data classification so it won't match my map exactly. But you can zoom in and mouse over counties and get a pop-up with the name and percentage of unedited ways so that is kind of neat.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;&lt;span class="Apple-style-span" style="font-size: large;"&gt;The Data&lt;/span&gt;&lt;/b&gt;&lt;br /&gt;For your further enjoyment, I am providing the CSV file that was produced as the result of my SQL query. The map is nice and all but some people might want cold hard numbers. I'm thinking some statistical analysis might be interesting. I originally tried to make a graph of the distribution of values but OpenOffice Calc wasn't cooperating that night and I had to pack for a trip so I dropped it.&lt;br /&gt;&lt;br /&gt;&lt;a href="http://ni.kwsn.net/~toby/OSM/tiger_edited.csv"&gt;Click here to download the file.&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;The first column is the value from the tiger:county tag. Following that is the percentage of unmodified ways in that county followed by the raw numbers of edited, unedited and total ways. The last three columns were used by me to manipulate the data into an acceptable form to join the data to the shapefile I had. I had to split out state code and county name and then expand the state to its full name using a table I generated from the source of a wikipedia page that lists all states and their abbreviations.&lt;br /&gt;&lt;br /&gt;I didn't clean it up at all so there are some nonsensical values in there. ArcMap just ignored the values it couldn't match to the shapefile. The first line has a county name of "30 days late" - I have no clue what that is doing in a tiger:county tag. Feel free to do an XAPI request and clean it up! &amp;nbsp;There is also data there for Puerto Rico and D.C. that didn't get matched up. Puerto Rico wasn't in my shapefile and I don't think D.C was correctly matched to the abbreviation.&lt;br /&gt;&lt;br /&gt;Enjoy!&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/9C7irsdxYaI" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2010/12/updated-tiger-map.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://2.bp.blogspot.com/_RA5iXwXMupA/TQmGQOqPZUI/AAAAAAAAASc/0RdKH1ACiss/s72-c/USA_2.png" height="72" width="72" /><thr:total>4</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-3199770012135799682</guid><pubDate>Sun, 21 Nov 2010 07:57:00 +0000</pubDate><atom:updated>2010-12-15T22:49:22.409-06:00</atom:updated><title>Nationwide TIGER Map</title><description>&lt;b&gt;Update: &lt;/b&gt;&lt;i&gt;You should of course still read this post because it is is awesome but there is a new post with an updated version of the map&amp;nbsp;&lt;a href="http://ksmapper.blogspot.com/2010/12/updated-tiger-map.html"&gt;here&lt;/a&gt;.&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;I presented most of this material this weekend at &lt;a href="http://www.wherecamp5280.org/"&gt;WhereCamp5280&lt;/a&gt;&amp;nbsp;and promised to put this online soon. I have a few more details and maybe some raw data that I will put in another post after I get back home after Thanksgiving but for now I'll throw up what I have.&lt;br /&gt;&lt;br /&gt;Well after some sweat, blood and tears (mostly on the part of my hard drives) I was eventually successful in extracting the data I needed to make a map of the entire US! If you just want the map feel free to skip down but I am going to document my trials and tribulations a little.&lt;br /&gt;&lt;br /&gt;First, some things that did &lt;b&gt;not&lt;/b&gt;&amp;nbsp;work:&lt;br /&gt;&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;Importing the whole planet file into an apidb using osmosis. I actually did this before I made the original Kansas map but it is part of the workflow I went through to get to the final map. The initial response I got on IRC was that it would take maybe 200 GB and a few days, maybe a week. I thought it would be fun to poke around in the database a bit beyond just this map so I decided to try it. I believe the time/space estimate may be accurate for importing to a pgsql database for rendering however I wanted a full apidb. The result:&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/_RA5iXwXMupA/TOOLd9N4wkI/AAAAAAAAARk/PYf0YiEfqmk/s1600/planet_import_disk.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="172" src="http://3.bp.blogspot.com/_RA5iXwXMupA/TOOLd9N4wkI/AAAAAAAAARk/PYf0YiEfqmk/s400/planet_import_disk.png" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;Yes, that is 3 weeks. I kept adding disk space to the volume hoping each time that it was almost done. The good news is that ext4's online resizing works great! But after having my heart broken one too many times I went hunting for more information which I probably should have done first. On the mailing list it was pointed out that the size of the main OSM apidb (The Source) is 1.4 TB and while that does include historic data that isn't in the planet file, I realized that I may only half way done so I decided that I needed a different approach.&lt;br /&gt;&lt;br /&gt;I tried trimming out a rough bounding box for the US which worked. Then I tried filtering out only nodes used in ways since I am only interested in the imported TIGER ways. I was hoping that a lot of the overhead in creating the apidb had to do with having to hit the node table every time a way was inserted due to foreign key constraints and such. I don't have a pretty graph for this attempt but apparently osmosis builds a data structure in memory for every node before filtering out the unused ones. Or something... The end result is that a computer with 16 GB of memory in it was still not enough to finish the job and failed within minutes with an "Out of Memory" exception from osmosis.&lt;br /&gt;&lt;br /&gt;My last ditch effort was just to import the whole US. I didn't use a fancy polygon to filter, just a straight bounding box that included Alaska and Hawaii so I'm sure I got some extra data (like a good chunk of Canada) but this ended up working... after 80 hours and about to 200 GB.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;Success!&lt;/b&gt;&lt;br /&gt;&lt;br /&gt;&lt;table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td style="text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/_RA5iXwXMupA/TOjC1UD6oXI/AAAAAAAAARo/V02l_xVXK4k/s1600/USA.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"&gt;&lt;img border="0" height="307" src="http://3.bp.blogspot.com/_RA5iXwXMupA/TOjC1UD6oXI/AAAAAAAAARo/V02l_xVXK4k/s400/USA.png" width="400" /&gt;&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td class="tr-caption" style="text-align: center;"&gt;Click for a bigger version to see those small counties!&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;The first thing you might notice is that there are some counties with no data. This is because the TIGER ways in those counties, for one reason or another, do not have tiger:county tags. Some were imported by a user other than DaveHansenTiger and they chose not to import all the TIGER attributes. Some were imported by him but just didn't have the tags. It was suggested at WhereCamp that these may have been the first counties imported and that he may have still been fine-tuning the import process. This seems like a plausible theory.&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Other than the missing data, the map makes sense for the most part. The west and northeast coasts are more edited than the center of the country. Large cities are easy to pick out. But there are a few individual counties that do stand out here and there. I'm guessing there are more people like me who have taken an interest in their local area out in the middle of nowhere and have made noticeable contributions to the map. The level of activity in Iowa does seem a little odd. I should single out Iowa and take a look at the data to see if I missed a bot or something. But who knows - it might just be a really motivated Iowan.&amp;nbsp;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Anyway, there you have it. As I said above, there will be another post with some details and data at a later time. Right now I am tired and need to get to bed after spending the day gathering GPS traces on the slopes of Winter Park with Steve, Hurricane, Josh and Richard. It was all for the map - I can assure you that no "fun" was had by anyone!&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;&lt;br /&gt;&lt;/div&gt;&lt;div class="separator" style="clear: both; text-align: left;"&gt;Well maybe a little.&lt;/div&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/2XCbl1zIN7Q" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2010/11/nationwide-tiger-map.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://3.bp.blogspot.com/_RA5iXwXMupA/TOOLd9N4wkI/AAAAAAAAARk/PYf0YiEfqmk/s72-c/planet_import_disk.png" height="72" width="72" /><thr:total>1</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-7151597907926502129</guid><pubDate>Sat, 30 Oct 2010 15:36:00 +0000</pubDate><atom:updated>2010-10-30T10:36:28.288-05:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">cartography</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">TIGER</category><title>Two more TIGER maps</title><description>After my last post there were some requests for me to do additional states. So I imported Colorado and New Mexico since they are kind of one big block. However doing 50 states a couple at a time will be rather cumbersome so I think after this I will try to coax osmosis into doing a more fine-tuned import so that I can import only the things I need but do it for the entire US at once.&lt;br /&gt;&lt;br /&gt;I also learned some more quirks about ArcMap. The program is amazing in some ways but user interface is not one of them. I was also unable to figure out how to use two different fields to join my data to the counties shapefile. So that was slightly frustrating but I was able to work around it.&lt;br /&gt;&lt;br /&gt;Anyway, until I get around to poking osmosis again,&lt;br /&gt;&lt;b&gt;&lt;span class="Apple-style-span" style="font-size: x-large;"&gt;I give you Colorado:&lt;/span&gt;&lt;/b&gt;&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://3.bp.blogspot.com/_RA5iXwXMupA/TMkVWaguiqI/AAAAAAAAARQ/pu5IQmNU6WU/s1600/CO_TIGER.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="308" src="http://3.bp.blogspot.com/_RA5iXwXMupA/TMkVWaguiqI/AAAAAAAAARQ/pu5IQmNU6WU/s400/CO_TIGER.png" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Colorado has another strong outlier: Teller county with only 35.8% of ways&amp;nbsp;untouched&amp;nbsp;. I didn't look at the data but I believe I know the person responsible for this, in real life even! Hi&amp;nbsp;&lt;a href="http://www.openstreetmap.org/user/SpenSmith"&gt;SpenSmith&lt;/a&gt;! Also, as you can see I replaced the "total number of TIGER ways proportional symbol with a few city names for general reference.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;&lt;span class="Apple-style-span" style="font-size: x-large;"&gt;On to New Mexico:&lt;/span&gt;&lt;/b&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/_RA5iXwXMupA/TMkNVykQV9I/AAAAAAAAARM/amwmDeMn_kA/s1600/NM_TIGER.png" imageanchor="1"&gt;&lt;img border="0" height="400" src="http://1.bp.blogspot.com/_RA5iXwXMupA/TMkNVykQV9I/AAAAAAAAARM/amwmDeMn_kA/s400/NM_TIGER.png" width="307" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;Notice the breakdown here. Looking at the data it would seem that either the TIGER data in New Mexico is nearly perfect and requires no editing or New Mexico is a barren wasteland, incapable of sustaining the fragile life force of mappers. There are no outlying counties and the most heavily edited one (Socorro county) is still sitting at almost 90% untouched. And looking at that county closer, I'm guessing it just doesn't have very many roads and all the editing that &lt;b&gt;has&lt;/b&gt;&amp;nbsp;been done there is just related to I-25 that runs right through the middle of the county. Of course this is percentages. As far as total number of TIGER ways touched, McKinley and&amp;nbsp;Bernalillo counties win with 877 and 762 respectively.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;&lt;span class="Apple-style-span" style="font-size: x-large;"&gt;OpenHeatMap&lt;/span&gt;&lt;/b&gt;&lt;br /&gt;&lt;b&gt;&lt;br /&gt;&lt;/b&gt;&lt;br /&gt;Lastly, SeveC pointed me at an interesting website that may be of use once I get my nationwide data lined up:&amp;nbsp;&lt;a href="http://www.openheatmap.com/"&gt;http://www.openheatmap.com/&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;I went ahead and imported the Colorado data and you can view it here:&lt;br /&gt;&lt;a href="http://www.openheatmap.com/view.html?map=FoundlingsFiascoesUngodliest"&gt;http://www.openheatmap.com/view.html?map=FoundlingsFiascoesUngodliest&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;I found a slight bug in that Broomfield county is missing from OpenHeatMap's database. I chatted with Pete (the admin) about it for a few minutes, until he had to go to participate in an interview for&amp;nbsp;&lt;a href="http://thisweekinmaps.com/"&gt;thisweekinmaps&lt;/a&gt;.&amp;nbsp;It turns out that he is using census data from 2000 for the county boundaries and Broomfield county &lt;a href="http://en.wikipedia.org/wiki/Broomfield_County#History"&gt;didn't yet exist in 2000&lt;/a&gt;!&amp;nbsp;I also mentioned to him that it would be nice if state borders were drawn on county based maps and he seemed to agree. There is always &lt;b&gt;something&lt;/b&gt;&amp;nbsp;to nitpick, right?&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/t5SgsQfOiLg" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2010/10/couple-more-tiger-maps.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://3.bp.blogspot.com/_RA5iXwXMupA/TMkVWaguiqI/AAAAAAAAARQ/pu5IQmNU6WU/s72-c/CO_TIGER.png" height="72" width="72" /><thr:total>1</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-7141410822343255383</guid><pubDate>Sun, 24 Oct 2010 21:41:00 +0000</pubDate><atom:updated>2010-12-15T22:52:35.224-06:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">cartography</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">TIGER</category><title>TIGER ways in Kansas</title><description>&lt;b&gt;Update:&lt;/b&gt;&amp;nbsp;&lt;i&gt;I have a second post with two more state maps up&amp;nbsp;&lt;a href="http://ksmapper.blogspot.com/2010/10/couple-more-tiger-maps.html"&gt;here&lt;/a&gt;. More to come!&lt;/i&gt;&lt;br /&gt;&lt;b&gt;Update 2:&lt;/b&gt; &lt;i&gt;Here are&amp;nbsp;&lt;a href="http://ksmapper.blogspot.com/2010/11/nationwide-tiger-map.html"&gt;two&lt;/a&gt;&amp;nbsp;&lt;a href="http://ksmapper.blogspot.com/2010/12/updated-tiger-map.html"&gt;more&lt;/a&gt;&amp;nbsp;posts with nationwide maps.&lt;/i&gt;&lt;br /&gt;&lt;br /&gt;If you are at all familiar with&amp;nbsp;&lt;a href="http://openstreetmap.org/"&gt;OpenStreetMap&lt;/a&gt;, especially in the US then you will know that most of the roads here are imported from the US Census Bureau TIGER data. This data was imported from the current data in 2007. It is fairly complete but the accuracy leaves a lot to be desired in some areas. For example look at this screen shot comparing the TIGER data to reality (USGS aerial imagery in this case). Some roads are off by over 200 meters. And there are even worse spots - this was just one I was able to find quickly.&lt;br /&gt;&lt;br /&gt;&lt;table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"&gt;&lt;tbody&gt;&lt;tr&gt;&lt;td style="text-align: center;"&gt;&lt;a href="http://1.bp.blogspot.com/_RA5iXwXMupA/TMPU8aGq4lI/AAAAAAAAARA/k3QAIUeHl1c/s1600/TIGER_error.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"&gt;&lt;img border="0" height="293" src="http://1.bp.blogspot.com/_RA5iXwXMupA/TMPU8aGq4lI/AAAAAAAAARA/k3QAIUeHl1c/s400/TIGER_error.png" width="400" /&gt;&lt;/a&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td class="tr-caption" style="text-align: center;"&gt;Example of bad TIGER data&lt;/td&gt;&lt;/tr&gt;&lt;/tbody&gt;&lt;/table&gt;&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;/div&gt;While the data was pretty complete when it was made, it is now several years old and missing many new housing developments and does not reflect instances where roads were moved. The good news is that, this being a census year, there will be new TIGER data available from the Census Bureau starting at the end of the year. It will be more complete and probably more accurate considering that GPS technology has made some advances over the last 10 years.&lt;br /&gt;&lt;br /&gt;Well that's great but the question is how do we use the new data to improve OSM? In the software world this would be an ideal situation to use a diff utility to determine what had changed between two versions and apply patches to update the old version. But I am not aware of any geo-spatial diff utilities.&lt;br /&gt;&lt;br /&gt;So how about just deleting the old data and re-importing the new stuff? This would obviously be a disaster in areas with active mappers. All the work they have put into correcting existing TIGER ways would be destroyed and replaced with data that is probably better than the old TIGER import but also probably worse than what local mappers have done.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;BUT...&lt;/b&gt;&lt;br /&gt;&lt;br /&gt;&lt;a name='more'&gt;&lt;/a&gt;&lt;br /&gt;&lt;br /&gt;I'm in Kansas. There aren't a lot of mappers here, especially in the western part of the state. Mostly because there just aren't a lot of people out there. Some counties have a population density of 6 people per square mile. Oh and the TIGER data was imported in county sized chunks. So how about handling this on a county-by-county basis? Counties that haven't been touched could just be blown away and refreshed with the new data. If only someone could determine how much TIGER data has changed since the initial import, on a per-county basis... Oh wait, I did!&lt;br /&gt;&lt;br /&gt;I am taking a cartography class at Kansas State University this semester. After starting to contribute to OSM I decided to take advantage of the employee tuition assistance and learn more about the subject. For $150/semester, why not! 20% of the grade in the class is for a project where he turned us loose to go get our own data and make some kind of interesting map. So I got my data by importing a Kansas extract from the OSM planet file and made this map:&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://2.bp.blogspot.com/_RA5iXwXMupA/TMPfanliQRI/AAAAAAAAARE/DcczmV5ffNI/s1600/KS_TIGER_2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="308" src="http://2.bp.blogspot.com/_RA5iXwXMupA/TMPfanliQRI/AAAAAAAAARE/DcczmV5ffNI/s400/KS_TIGER_2.png" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;br /&gt;A couple of notes about the map:&lt;br /&gt;&lt;br /&gt;&lt;ul&gt;&lt;li&gt;I let ArcMap classify the data using natural breaks. There are 105 counties in Kansas. 104 of them still have have 78% of their TIGER data in its original state. One has had 75% of it modified. Yes, this is the county I live in. Yes, most of it was done by me. Yes, this inflates my ego.&lt;/li&gt;&lt;li&gt;The total count doesn't really do much for the map in my opinion but the class project called for a bivariate map, so a bivariate map I made! Obviously counties with bigger cities have more TIGER ways. This is your dose of non-surprise for the day.&lt;/li&gt;&lt;li&gt;My method of detecting changes by "local" mappers is by no means fool-proof. Basically I looked at the user who last changed a given way. If the user was one of 4 users I identified that were definitely not local mappers, I counted that road as unchanged. (technical details below)&lt;/li&gt;&lt;li&gt;Labels would add a lot of clutter to the map. If you want to see which counties are which, I suggest the &lt;a href="http://en.wikipedia.org/wiki/Counties_of_Kansas"&gt;Counties of Kansas&lt;/a&gt; wikipedia page.&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;So I guess the question is how do we use this information? This will have to be a discussion amongst the OSM-US community. Due to time limitations I only did Kansas but this could certainly be done for other states (particularly the sparsely populated ones) to help local mappers decide what, if anything, to do with the new TIGER data. My suggestion would be to do a fresh import of any counties that have above a certain threshold of unchanged data. Say 95%? In Kansas that would be 59 counties. If a threshold were decided upon the map classifications could be altered to reflect that number.&lt;br /&gt;&lt;br /&gt;For counties that HAVE had local activity, perhaps some process could be set up using tiles from the&lt;a href="http://open.mapquestapi.com/tigerviewer/index.html"&gt; TIGER edited map&lt;/a&gt; that MapQuest has provided as a background layer in JOSM or P2. Then pull up the new TIGER data on top of it and compare. Import missing roads or ones that are more accurate in the new data and haven't been touched by local mappers. That is just a thought that popped into my head as I was writing this.&lt;br /&gt;&lt;br /&gt;&lt;b&gt;Technical Details&lt;/b&gt;&lt;br /&gt;&lt;br /&gt;I imported the October 20th planet file into an apidb using osmosis. I originally tried to do the whole planet and intended to do a larger analysis of bot activity on a worldwide basis but after 4 weeks and 350 GB &amp;nbsp;it showed no signs of being anywhere close to finished so I fell back to only importing Kansas. Luckily Kansas is pretty much a big rectangle so I just used a simple bounding box. This finished in about 2 hours.&lt;br /&gt;&lt;br /&gt;Here is the final SQL query I came up with to get me all the data I needed in one result set. Keep in mind that I only imported a bounding box around Kansas:&lt;br /&gt;&lt;br /&gt;&lt;pre class="source-code"&gt;select v as county,&lt;br /&gt;sum(CASE WHEN user_id in (147510,7168,20587,293105)&lt;br /&gt;    THEN 1 ELSE 0 END) as bot_count,&lt;br /&gt;sum(CASE WHEN user_id in (147510,7168,20587,293105)&lt;br /&gt;    THEN 0 ELSE 1 END) as user_count,&lt;br /&gt;count(*) as total_count&lt;br /&gt;from ways, way_tags, changesets&lt;br /&gt;where ways.id = way_tags.id&lt;br /&gt;and ways.changeset_id = changesets.id&lt;br /&gt;and k = 'tiger:county'&lt;br /&gt;and v like '%KS%'&lt;br /&gt;and v not like '%;%'&lt;br /&gt;group by v&lt;br /&gt;order by v&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;In less SQLish terms: I am primarily looking at the tiger:county tag to group the query by county. It contains values like "Riley, KS" so basically I am looking for any way with a 'tiger:county=*KS*' tag. This excludes the few ways around the Kansas border that are from other states since my bounding box was just a little bigger than the state borders. However I exclude ways that have multiple values in the tag, separated by a semicolon. Typically this would come from two ways in adjacent counties being joined together. This is pretty rare so I ignored them. Once I find those ways I examine the user who last touched the way by joining through the changesets table. If the user ID is one of 4 values then I count the way as not having been modified by a local mapper. Those 4 user IDs belong to the following users:&lt;br /&gt;&lt;ul&gt;&lt;li&gt;147510 = woodpeck_fixbot (this is a bot that has performed various automated edits as documented on the OSM wiki)&lt;/li&gt;&lt;li&gt;7168 =&amp;nbsp;DaveHansenTiger (this is the user who did the initial TIGER import)&lt;/li&gt;&lt;li&gt;20587 = balrog-kun (this user did some mass editing of TIGER ways by expanding abbreviations in street names)&lt;/li&gt;&lt;li&gt;293105 = NHD edits&amp;nbsp;(NHD = National Hydrography Dataset. I'm assuming this user probably imported some rivers and ended up splitting some TIGER ways to make bridges or something&lt;/li&gt;&lt;/ul&gt;&lt;br /&gt;&lt;br /&gt;Looking at the data now, I should have maybe also excluded NE2. I believe he has done a lot of work on national highway/interstate routes. I don't think there is any reason to re-import ways that are part of those systems since they have since been added to route relations and such. So those should probably be excluded from the data. Hindsight and all that.&lt;br /&gt;&lt;br /&gt;Anyway, this query gives me a result like this:&lt;br /&gt;&lt;pre class="source-code"&gt;county            | bot_count | user_count | total_count&lt;br /&gt;------------------+-----------+------------+-------------&lt;br /&gt; Allen, KS        |      3062 |         33 |        3095&lt;br /&gt; Anderson, KS     |      2879 |         64 |        2943&lt;br /&gt; Atchison, KS     |      1235 |         37 |        1272&lt;br /&gt; Barber, KS       |      2173 |         26 |        2199&lt;br /&gt; Barton, KS       |      1764 |        254 |        2018&lt;br /&gt; Bourbon, KS      |      3815 |        137 |        3952&lt;br /&gt; Brown, KS        |       907 |         46 |         953&lt;br /&gt;.&lt;br /&gt;.&lt;br /&gt;.&lt;br /&gt;&lt;/pre&gt;&lt;br /&gt;From there I regex'd it into a CSV file and imported it into ArcMap and joined it to the shapefile of Kansas counties that was provided to us. Easy, right?&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/MwBd0msaX_o" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2010/10/tiger-ways-in-kansas.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://1.bp.blogspot.com/_RA5iXwXMupA/TMPU8aGq4lI/AAAAAAAAARA/k3QAIUeHl1c/s72-c/TIGER_error.png" height="72" width="72" /><thr:total>11</thr:total></item><item><guid isPermaLink="false">tag:blogger.com,1999:blog-7044416767097884399.post-7494061836987470846</guid><pubDate>Sat, 23 Oct 2010 05:53:00 +0000</pubDate><atom:updated>2010-10-23T00:53:59.050-05:00</atom:updated><category domain="http://www.blogger.com/atom/ns#">FOSS</category><category domain="http://www.blogger.com/atom/ns#">hello world</category><category domain="http://www.blogger.com/atom/ns#">Linux</category><category domain="http://www.blogger.com/atom/ns#">OSM</category><category domain="http://www.blogger.com/atom/ns#">picture</category><title>The Shiny New Blog</title><description>I have avoided creating a blog for years now. I never felt like I had enough important ideas to force upon the world to justify it. That and I assumed I would abandon it at some point and it would become yet another piece of litter along the information superhighway. I'm not convinced that these things have really changed but there have been a few times recently where I did want to post an idea to a more permanent medium than IRC, especially concerning my latest hobby: OpenStreetMap. So here I am, creating a blog on a Friday night.&lt;br /&gt;&lt;br /&gt;Who knows where it will go but I do hope to post occasional observations concerning OpenStreetMap as well as some open source topics in general (yes, I use Linux) and maybe an occasional note about my job. I work as a developer for Kansas State University. Java with a heavy dose of SQL (Oracle) if you are curious.&lt;br /&gt;&lt;br /&gt;Now, here is a random picture of me at the end of No-Shave November in 2008. If anyone reading this doesn't know me, it is a horrible first impression because I usually shave. But there you have it.&lt;br /&gt;&lt;br /&gt;&lt;br /&gt;&lt;div class="separator" style="clear: both; text-align: center;"&gt;&lt;a href="http://4.bp.blogspot.com/_RA5iXwXMupA/TMJ1YYMHOrI/AAAAAAAAAQM/X9WKfkx3NIE/s1600/PC110635.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"&gt;&lt;img border="0" height="300" src="http://4.bp.blogspot.com/_RA5iXwXMupA/TMJ1YYMHOrI/AAAAAAAAAQM/X9WKfkx3NIE/s400/PC110635.JPG" width="400" /&gt;&lt;/a&gt;&lt;/div&gt;&lt;img src="http://feeds.feedburner.com/~r/ksmapper/~4/ylTnj8hJMuk" height="1" width="1" alt=""/&gt;</description><link>http://ksmapper.blogspot.com/2010/10/shiny-new-blog.html</link><author>noreply@blogger.com (Unknown)</author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="http://4.bp.blogspot.com/_RA5iXwXMupA/TMJ1YYMHOrI/AAAAAAAAAQM/X9WKfkx3NIE/s72-c/PC110635.JPG" height="72" width="72" /><thr:total>3</thr:total></item></channel></rss>


