Category Archives: Uncategorized

too damn many open data licences

Oh Canada …

  1. Open Government Licence – Ontario
  2. City of Guelph Open Government Licence
  3. Open Government Licence – Vancouver
  4. Open Government Licence – County of Grande Prairie
  5. Open Government Licence – Alberta
  6. Open Government Licence for City of Nanaimo
  7. Open Government Licence – Strathcona County
  8. Open Data Licence for Town of Banff
  9. Open Government Licence for District of Squamish
  10. Open Government Licence — Town of Oakville
  11. Open Government Licence – Kamloops
  12. Open Government License for the City of Surrey
  13. Open Government Licence – Toronto
  14. OPEN GOVERNMENT LICENCE – TORONTO PUBLIC LIBRARYand yes, they’re not covered by the City’s licence
  15. Open Government License for Government of British Columbia
  16. City of Edmonton Open Data Terms of Usea truly odd one that seems to have little in common with any of the others
  17. Region of Waterloo – Open Data Licence
  18. Open Government Licence for the City of Regina
  19. Open Government Licence – The Corporation of the City of Kitchener
  20. York Region’s Open Data Licencea catchy little URL at only 427 characters short
  21. Open Data Licence for The Regional Municipality of Peel
  22. Open Government Licence – Canada
  23. Open Government Licence – City of Ottawa

— and they’re all slightly different, even in how they spell the word “Licence”  …

map markers: don’t have to stick to the defaults

Some of the languages of Irian Jaya

Map background: Natural Earth (public domain)
Language point data: Glottolog (licence: Creative Commons — Attribution-ShareAlike 3.0 Unported — CC BY-SA 3.0)
Map markers: font: Overpass (licence: SIL OFL/LGPL), character: 💬 — U+1F4AC SPEECH BALLOON.
Label font: U001Con Italic (foundry: URW++, licence: AFPL)

map of the world (actual size)

Back in 2000, when I worked for Collins Dictionaries, all staff were given a copy of The Times Comprehensive Atlas of the World (ISBN 0-7230-0792-6). It’s colossal: nearly half a metre tall, and around six kilos in its slip case. Devised by Bartholomew’s and gloriously printed in Germany, it’s unlikely I’ll get this home from Scotland. So here are some photos, lest I forget:

Time Comprehensive Atlas of the World, Millennium Edition (banana for scale)
Times Comprehensive Atlas of the World, Millennium Edition
(banana for scale)

atlas2

atlas3

atlas4

 

GTALUG OSM Talk last week

Title OpenStreetMap: Building a great map while everyone tells you you’re doing it wrong.

frontpageNotes/Links:

  1. ‘The Map’ — http://www.openstreetmap.org . It supports routing now, too.
  2. QGIS, an open GIS manager. It’s rather good — http://qgis.org/
  3. The OSM Wiki; ridiculously complete documentation: https://wiki.openstreetmap.org/wiki/
  4. OSM Help Stack Exchange-style question/answer: https://help.openstreetmap.org/
  5. All of the OSM stats! — https://wiki.openstreetmap.org/wiki/Stats
  6. Toronto map growth animation — http://www.geofabrik.de/gallery/history/index.html#toronto
  7. Crowdsourced geocoding (+ lawsuit from Canada Post) — http://geocoder.ca/
  8. Open Data Commons Open Database License (ODbL) — http://opendatacommons.org/licenses/odbl/
  9. Canada’s new Open Government portal — http://open.canada.ca related: Toronto Open Data — http://toronto.ca/open
  10. CIPPIC Open Licensing Project (CLIP) — http://clipol.org/
  11. Humanitarian OpenStreetMap Team [HOT] — http://hotosm.org/
  12. OpenCycleMap — http://www.openstreetmap.org/#map=13/43.6666/-79.3785&layers=C
  13. The rather wonderful /uMap/https://umap.openstreetmap.fr/en/

Original GTALUG note: OpenStreetMap links from the other night.

Bitcoin Map spams OpenStreetMap

It looks like the Bitcoin Map website adds points illicitly to OSM through a Google Maps interface. This is rather bad.

Update: they’re fixing this …

Here’s a test run I took to see if the data was really being added to OSM:

Screenshot from 2015-03-04 21:52:38
Bitcoin Map website showing a location near me in Google Maps
Screenshot from 2015-03-04 21:56:34
Adding fake POI info (including massively fake city address, for lulz)
Screenshot from 2015-03-04 21:57:05
Confirmation that Bitcoin Map has accepted my edit
Screenshot from 2015-03-04 21:58:26
Checking the details on the website

Here’s the POI data, in raw OSM XML:

<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="CGImap 0.3.3 (28262 thorn-02.openstreetmap.org)" copyright="OpenStreetMap and contributors" attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/">
 <node id="3383877893" visible="true" version="1" changeset="29257856" timestamp="2015-03-05T02:56:47Z" user="BitcoinMaps" uid="2135320" lat="43.7298277" lon="-79.2721787">
  <tag k="addr:city" v="Bemidji"/>
  <tag k="addr:housenumber" v="134"/>
  <tag k="addr:street" v="Woodfern Drive"/>
  <tag k="contact:email" v="name@example.com"/>
  <tag k="contact:phone" v="+1 416 555 1234"/>
  <tag k="contact:website" v="http://example.com/"/>
  <tag k="description" v="i have totally made this up to see if it will be added to OSM even though the location was derived from Google Maps"/>
  <tag k="name" v="Entirely FictitiousName"/>
  <tag k="payment:litecoin" v="yes"/>
  <tag k="shop" v="books"/>
 </node>
</osm>
Screenshot from 2015-03-04 22:24:41
The node is still (temporarily) on the map, despite deleting it. It’ll be gone soon.

 

Kirkby Fleetham Proposed Pit (MJP60)

See full screen

for more details, please see:
Minerals and waste joint plan consultation – North Yorkshire County Council http://northyorks.gov.uk/article/23999/Minerals-and-waste-joint-plan-consultation
Outline traced from http://northyorks.gov.uk/media/30250/Supplementary-sites-consultation—January-2015/pdf/Supplementary_sites_Consultation-_Web_Version.pdf

Update: After reading Anita’s article about Publishing interactive web maps using QGIS, I had to try the QGIS2leaf plugin. And lo! It works:

though, to be fair, this is just a static screen dump; click on the image for the live map …
though, to be fair, this is just a static screen dump; click on the image for the live map …

Joshua Frazier’s great tutorial Let’s make some web maps using Leaflet.js! for Maptime Toronto was a huge help in getting me started with Leaflet. Thanks, Joshua!

Summary of my off-the-cuff Maptime Presentation: Canadian Microwave Links

Screenshot from 2014-07-26 10:17:15@MaptimeTO asked me to summarize the brief talk I gave last week at Maptime Toronto on making maps from the Technical and Administrative Frequency List (TAFL) radio database. It was mostly taken from posts on this blog, but here goes:

  1. One of the many constraints in building wind farms is allowing for radio links. Both the radio and the wind industries have agreed on a process of buffering and consultation. Here’s how I handled it in Python: Making weird composite shapes with Shapely.
  2. The TAFL databases — which contains locations and technical data for all licensed transmitters — are now open data. You can find them here: Technical and Administrative Frequency List (TAFL).
  3. The format is a real delight for all legacy-data nerds: aka a horrible mess of conditional field widths and arcane numeric codes. I wrote a SpatiaLite SQL script to make sense of it all: scruss/taflmunge. This (kind of) explains what it does: TAFL — as a proper geodatabase.
  4. Here’s a raw dump (very little metadata, sorry) from 2013 in the wonderful uMap: Ontario Microwave Links.
  5. In a fabulous piece of #opendatafail, Industry Canada have migrated all the microwave data (so, all links ≥ 960 MHz) to a new system which doesn’t work yet, and also stripped out all of the microwave data from recent TAFL files. They claim to be fixing it, but don’t hold your breath. If you want data to play with, here’s Ontario’s data from October 2013 (nb: huge) — ltaf_ont_tafl-20131001.

TAFL — as a proper geodatabase

Update, 2017: TAFL now seems to be completely dead, and Spectrum Management System has replaced it. None of the records appear to be open data, and the search environment seems — if this is actually possible — slower and less feature-filled than in 2013.

Update, 2013-08-13: Looks like most of the summary pages for these data sets have been pulled from data.gc.ca; they’re 404ing. The data, current at the beginning of this month, can still be found at these URLs:

I build wind farms. You knew that, right? One of the things you have to take into account in planning a wind farm is existing radio infrastructure: cell towers, microwave links, the (now-increasingly-rare) terrestrial television reception.

I’ve previously written on how to make the oddly blobby shape files to avoid microwave links.  But finding the locations of radio transmitters in Canada is tricky, despite there being two ways of doing it:

  1. Wrestle with the Spectrum Direct website, which can’t handle the large search radii needed for comprehensive wind farm design. At best, it spits out weird fixed-width text data, which takes some effort to parse.
  2. Download the Technical and Administrative Frequency Lists (TAFL; see update above for URLs), and try to parse those (layout, fields). Unless you’re really patient, or have mad OpenRefine skillz, this is going to be unrewarding, as the files occasionally drop format bombs like
    never do this, okay?
    Yes, you just saw conditional different fixed-width fields in a fixed-width text file. In my best Malcolm Tucker (caution, swearies) voice I exhort you to never do this.

So searching for links is far from obvious, and it’s not like wireless operators do anything conventional like register their links on the title of the properties they cross … so these databases are it, and we must work with them.

The good things is that TAFL is now Open Data, defined by a reasonable Open Government Licence, and available on the data.gc.ca website. Unfortunately, the official Industry Canada tool to process and query these files, is a little, uh, behind the times:tafl2dbfYes, it’s an MS-DOS exe. It spits out DBase III Files. It won’t run on Windows 7 or 8. It will run on DOSBox, but it’s rather slow, and fails on bigger files.

That’s why I wrote taflmunge. It currently does one thing properly, and another kinda-sorta:

  1. For all TAFL records fed to it, generates a SpatiaLite database containing these points and all their data; certainly all the fields that the old EXE produced. This process seems to work for all the data I’ve fed to it.
  2. Tries to calculate point-to-point links for microwave communications. This it does less well, but I can see where the SQL is going wrong, and will fix it soon.

taflmunge runs anywhere SpatiaLite does. I’ve tested it on Linux and Windows 7. It’s just a SQL script, so no additional glue language required. The database can be queried on anything that supports SQLite, but for real spatial cleverness, needs SpatiaLite loaded. Full instructions are in the taflmunge / README.md.

TAFL is clearly maintained by licensees, as the data can be a bit “vernacular”. Take, for example, a tower near me:

pharm_eg

The tower is near the top of the image, but the database entries are spread out by several hundred meters. It’s the best we’ve got to work with.

Ultimately, I’d like to keep this maintained (the Open Data TAFL files are updated monthly), and host it in a nice WebGIS that would allow querying by location, frequency, call sign, operator, … But that’s for later. For now, I’ll stick with refining it locally, and I hope that someone will find it useful.