Author: scruss

Categories
GIS

osm vs maps.me

(Hey! I’m not a lawyer. Read a few paragraphs of this, and it’ll be clear I shouldn’t even try to give any kind of opinion. The screenshots I include with maps.me, booking.com and Google data are presented merely for review and critique of the situation.)

An interesting discussion showed up on the OSM Legal mailing list a couple of weeks back: MAPS.ME combining OSM data and non-OSM data? OSM has some restrictions on how its data can be used. At a grossly simple level, if you mix your proprietary data in with OSM data and present a single layer map, you must either:

  1. provide the proprietary data you mixed in under the same licence as OSM, or;
  2. stop doing that.

The OSMF Licence/Community Guidelines/Horizontal Map Layers – Guideline has some example scenarios that I will quote here:

Examples of where you DO NOT need to share your non-OpenStreetMap data

  1. …
  2. You use OpenStreetMap as a base topographical map and make your best reasonable efforts to exclude ALL restaurants. You then add a layer of your own restaurant data.

Examples of where you DO need to share your non-OpenStreetMap data

  1. …
  2. You add restaurants in one area from non-OpenStreetMap data based on comparison with OpenStreetMap data in other layers.

So for this article, please bear in mind that where the above example says restaurants, I’m looking for an analogous case with hotels.

MAPS.ME (which they want you to pronounce as “maps with me”, even though it says “maps dot me”; I guess “maps ∘ me” wouldn’t make such a nice URI) is an offline map app that shows a mix of POIs over a custom-styled OSM basemap. Its publisher mixes OSM POIs with proprietary ones, and publishes a list (local copy: skipped_nodes-20160731.zip) of OSM nodes that it has skipped from its POI list. To satisfy my curiosity, I wanted to see if the required separation of data was in place with maps.me.

First, I used overpass turbo to return all the hotel and motel nodes in and around Toronto (the query I used was “type:node and (tourism=hotel or tourism=motel) in bbox”, saved as TorontoHotelNodes.zip). This list contains 50 hotels and motels.

Next, I used maps.me’s skipped nodes list to find the OSM hotels and motels that maps.me had omitted from their app. This list should closely match the one from booking.com, where the app company said they got their hotel data. For completeness, I queried the OSM hotels extract to see what hotels maps.me had possibly not skipped.

Finally, I overlaid those OSM nodes on a georeferenced screendump from booking.com’s Google map. Here’s what it looks like (click to enlarge):

to_hotels-OSM_vs_maps_me

So OSM nodes not skipped (which is not necessarily the same as used) by maps.me are marked  ★, while those excluded by maps.me are marked ⯅. At first glance, there’s clearly not a 100% correlation between booking.com hotels, but a few clues emerge that hint strongly that maps.me are mixing in some OSM nodes with some proprietary nodes. It’s therefore likely that maps.me has not abided by the OSMF community guideline.

Scrolling around and zooming in on the booking.com hotel map shows that they don’t show you all of the listed properties. There’s always some filter based on location, price or availability. This makes sense if you’re managing a valuable proprietary data set, as there will always be someone trying to scrape your data and take your business with slightly cheaper referrals. One example of a hotel available on booking.com is The Old Mill (booking.com link: Old Mill Toronto) even though it’s not shown with a booking.com pin on my screenshot above. The Old Mill is also shown as not skipped on maps.me, so let’s take a look at it in the app:

2016-08-01_12.09.00Ah; there’s the booking.com property, with the OSM node off to the east under a slightly different name, and the OSM restaurant node somewhere in between. (When you search maps.me for restaurants, in Toronto at least, there are no referral links: missed opportunity there, folks!)

Since OSM is community-generated data, sometimes our spelling isn’t so great. One hotel node that leaps out from the maps.me node set is the (now fixed) “Sharaton”:

2016-07-31_23.35.19aka Sheraton Toronto Airport Hotel to booking.com, or Sheraton Toronto Airport Hotel & Conference Centre to its friends:

2016-07-31_23.36.11I wonder if maps.me will exclude the OSM node for this Sheraton from now on? Now that its name has been edited, if they do skip it, it means that they’re comparing OSM against a proprietary list, so would be breaking the guidelines. On the other hand, if they don’t exclude it, they really need to get a better fuzzy match algorithm, and post haste.

We can’t know the algorithm that maps.me uses to choose hotels to include referral links and skip the OSM data. The details of that are their trade secret, and no open licence can compel them to disclose it. But the cynic in me has found a possible clue in the budget motels that line Kingston Road on the eastern Toronto lake shore. Both the Days Inn – Toronto East and the Park Motel are listed on booking.com. Only the slightly more expensive Days Inn gets a referral link on maps.me. Could it be that the value of the cheaper motel’s referral isn’t worth including? (Update: this is wrong; I found another reason, which I will write about later.)

In addition, I wonder if maps.me’s skipped nodes list could be considered a “derived work” of booking.com? As the choice of nodes to skip is informed by booking.com’s data, it does give us a small insight into their database.

Categories
Uncategorized

map of the world (actual size)

Time Comprehensive Atlas of the World, Millennium Edition (banana for scale)

Back in 2000, when I worked for Collins Dictionaries, all staff were given a copy of The Times Comprehensive Atlas of the World (ISBN 0-7230-0792-6). It’s colossal: nearly half a metre tall, and around six kilos in its slip case. Devised by Bartholomew’s and gloriously printed in Germany, it’s unlikely I’ll get this home from Scotland. So here are some photos, lest I forget:

Time Comprehensive Atlas of the World, Millennium Edition (banana for scale)
Times Comprehensive Atlas of the World, Millennium Edition
(banana for scale)

atlas2

atlas3

atlas4

 

Categories
GIS

VËRY MËTÄL MÄPPÏNG

Screenshot from 2015-11-21 08-55-49“King City of Rock” was a nice touch.

Categories
GIS

Maptime Toronto talk — tomorrow!

I’m talking about data and tools at Maptime Toronto tomorrow. Here are my slides: maptime201511-tools_and_data.

Categories
Uncategorized

GTALUG OSM Talk last week

Title OpenStreetMap: Building a great map while everyone tells you you’re doing it wrong.

frontpageNotes/Links:

  1. ‘The Map’ — http://www.openstreetmap.org . It supports routing now, too.
  2. QGIS, an open GIS manager. It’s rather good — http://qgis.org/
  3. The OSM Wiki; ridiculously complete documentation: https://wiki.openstreetmap.org/wiki/
  4. OSM Help Stack Exchange-style question/answer: https://help.openstreetmap.org/
  5. All of the OSM stats! — https://wiki.openstreetmap.org/wiki/Stats
  6. Toronto map growth animation — http://www.geofabrik.de/gallery/history/index.html#toronto
  7. Crowdsourced geocoding (+ lawsuit from Canada Post) — http://geocoder.ca/
  8. Open Data Commons Open Database License (ODbL) — http://opendatacommons.org/licenses/odbl/
  9. Canada’s new Open Government portal — http://open.canada.ca related: Toronto Open Data — http://toronto.ca/open
  10. CIPPIC Open Licensing Project (CLIP) — http://clipol.org/
  11. Humanitarian OpenStreetMap Team [HOT] — http://hotosm.org/
  12. OpenCycleMap — http://www.openstreetmap.org/#map=13/43.6666/-79.3785&layers=C
  13. The rather wonderful /uMap/ — https://umap.openstreetmap.fr/en/

Original GTALUG note: OpenStreetMap links from the other night.

Categories
Uncategorized

Bitcoin Map spams OpenStreetMap

It looks like the Bitcoin Map website adds points illicitly to OSM through a Google Maps interface. This is rather bad.

Update: they’re fixing this …

Here’s a test run I took to see if the data was really being added to OSM:

Screenshot from 2015-03-04 21:52:38
Bitcoin Map website showing a location near me in Google Maps
Screenshot from 2015-03-04 21:56:34
Adding fake POI info (including massively fake city address, for lulz)
Screenshot from 2015-03-04 21:57:05
Confirmation that Bitcoin Map has accepted my edit
Screenshot from 2015-03-04 21:58:26
Checking the details on the website

Here’s the POI data, in raw OSM XML:

<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="CGImap 0.3.3 (28262 thorn-02.openstreetmap.org)" copyright="OpenStreetMap and contributors" attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/">
 <node id="3383877893" visible="true" version="1" changeset="29257856" timestamp="2015-03-05T02:56:47Z" user="BitcoinMaps" uid="2135320" lat="43.7298277" lon="-79.2721787">
  <tag k="addr:city" v="Bemidji"/>
  <tag k="addr:housenumber" v="134"/>
  <tag k="addr:street" v="Woodfern Drive"/>
  <tag k="contact:email" v="name@example.com"/>
  <tag k="contact:phone" v="+1 416 555 1234"/>
  <tag k="contact:website" v="http://example.com/"/>
  <tag k="description" v="i have totally made this up to see if it will be added to OSM even though the location was derived from Google Maps"/>
  <tag k="name" v="Entirely FictitiousName"/>
  <tag k="payment:litecoin" v="yes"/>
  <tag k="shop" v="books"/>
 </node>
</osm>
Screenshot from 2015-03-04 22:24:41
The node is still (temporarily) on the map, despite deleting it. It’ll be gone soon.

 

Categories
Uncategorized

Kirkby Fleetham Proposed Pit (MJP60)

See full screen

for more details, please see:
Minerals and waste joint plan consultation – North Yorkshire County Council http://northyorks.gov.uk/article/23999/Minerals-and-waste-joint-plan-consultation
Outline traced from http://northyorks.gov.uk/media/30250/Supplementary-sites-consultation—January-2015/pdf/Supplementary_sites_Consultation-_Web_Version.pdf

Update: After reading Anita’s article about Publishing interactive web maps using QGIS, I had to try the QGIS2leaf plugin. And lo! It works:

though, to be fair, this is just a static screen dump; click on the image for the live map …
though, to be fair, this is just a static screen dump; click on the image for the live map …

Joshua Frazier’s great tutorial Let’s make some web maps using Leaflet.js! for Maptime Toronto was a huge help in getting me started with Leaflet. Thanks, Joshua!

Categories
GIS

Some notes on the wartime “Modified British” map coordinate systems

Growing up in a small country, I assumed the whole world used metric grid map coordinates. I mean, why would anyone bother with those tedious latitudes and longitudes when you could have your location defined by something as neat as NS539555?

The tidy, militaristic Ordnance Survey came up with the National Grid reference system, where large grid squares were given letters, and the rest of the reference was given numerically. Wikipedia gives a better graphical explanation than I could:

Illustration of the Ordnance Survey National Grid coordinate system, with Trafalgar Square as an example [CC BY-SA 3.0]

During WW2, the UK war office extended this system across most of Europe. Since most European countries didn’t use exactly the same Transverse Mercator projection as the UK did, a number of existing mapping systems were pressed into use, but using the same interleaved alphanumeric format as the OS grid reference.

The reference site for these systems is Thierry Arsicaud’s excellent Notes on the “Modified British System” used on the European Theatre of Operations during the WWII. Thierry, however, wasn’t trying to use these historical map references in a GIS, so his work needs a little massage to get to be used with QGIS.

In this example, I’m going to concentrate on the South Italy zone, as that’s where I was asked to look at some war diaries from 1943. The system is similar to the OS grid, but the main difference is that the major grid reference is often given in lower case. So RN in OSGB would most often be denoted rN in the Modified British system. Both would refer to a 100 km square at (700, 700) km from the origin. (The exceedingly nitpicky might point out that RN is never used in the UK as it’s somewhere in the Atlantic, west of Ireland. To them, I say: Well, bless your heart …)

The key to both the OSGB grid and the Modified British system is a 2500 × 2500 km square, split into 25 500 km squares, and given a letter, a … z with i excluded:

mod_brit-major

Each letter encodes both an easting and a northing; so r is (500000, 500000). About the easiest way to unpack this encoding is through a simple string lookup:

result = SEARCH(letter, "VWXYZQRSTULMNOPFGHJKABCDE")-1
easting = result MOD 5
northing = INT(result / 5)

where SEARCH is a function which returns the position of a letter in a string (so SEARCH(“V”, “VWXYZ…”) returns 1).

When applied as per the GSGS South Italy system, you get something like this:

italy_south-major-r

These major grid squares are in turn split into 25 minor squares of 100 km side:

mod_brit-minor

Or overlaid on a map:

italy_south-minor-r

For grid references of higher precision, a series of numbers is appended. There should always be an even count of these numbers, for reasons which should become clear soon. Here is the rC square, split into 10 km references:

mod_brit-rC

These two digit references are about the shortest/least precise you might ever see. Overlaid on an appropriate sheet from the McMaster archive WWII Topographic Map Series (which are CC BY-NC; for which, many thanks), you get:

chieti-rC_grid

The actual projection details are given on each map:

south_italy

We can turn this into a PROJ.4 definition:

Projection - Lambert Conical Orthomorphic  →    +proj=lcc
Ellipsoid: Bessel 1841                     →    +ellps=bessel
False Easting : 700000                     →    +x_0=700000
False Northing : 600000                    →    +y_0=600000
Central Meridian : 14.0°                   →    +lon_0=14 
Central Parallel : 39.5°                   →    +lat_0=39.5 +lat_1=39.5
Scale Factor : 0.99906                     →    +k_0=0.99906
                       (other proj.4 terms)     +units=m +no_defs

or in one line,

+proj=lcc +lat_0=39.5 +lat_1=39.5 +lon_0=14 +k_0=0.99906 +x_0=700000 +y_0=600000 +ellps=bessel +units=m +no_defs

In QGIS, you can plug those values into the Custom CRS manager, and you will be able to work in these antiquated coordinate systems with ease:

Screenshot from 2015-01-09 07:28:53

I haven’t yet quite managed to work out some of the other GSGS coordinate systems. My work on North Italy is a stubborn 100 km off true, for no well-defined reason. I haven’t managed to work out unpicking alphanumeric grid references into geometries automatically, either. These will come later.

Finally, some of the coordinates you might see might not meet these specifications. In the limited survey I’ve done, I’ve noted:

  1. references with the major grid missing, so rCxy… was written as Cxy….
  2. references to ‘MR’ (map reference), with no alphanumeric part, such as MR 322142 (from here), which would be more correctly given as rC322142.

Huge thanks to Thierry Arsicaud, both for the great reference website, and also for the e-mail correspondence helping explain the parameters for the GSGS Italy South system. Props too to the Geographic Information Systems Stack Exchange folks for help with working out the proj.4 settings.

Categories
GIS

So maybe I am getting better at handling WW2 Modified British System coordinates after all …

Screenshot from 2014-11-27 22:20:57

Categories
GIS

QGIS on Raspberry Pi

Hey! This is really old! The current version of Raspbian has QGIS 2.4 included in the repository. Just install that. It won’t run very fast on single-core Raspberry Pis, though.

qgis-on-pi

  1. Install Raspbian:
  2. Update Raspbian from its Debian wheezy base to Debian jessie:
    • sudo vi /etc/apt/sources.list # or use your favourite editor
    • change all references of wheezy to jessie
    • sudo apt-get update
    • sudo apt-get upgrade # this will take a long time, with occasional user prompts
    • sudo apt-get dist-upgrade # this will take a very long time …
  3. Install qgis: sudo apt-get install gdal-bin qgis

This will install QGIS 2.2. It’s a bit slow for general use, but it does work …

(modified from my gis.stackexchange answer: linux – GDAL and QGIS Raspberry Pi. Mainly just so I would have a place to put the image.)