geocoder.ca provides a crowdsourced postal code geocoder under the ODbL. You can download the database as CSV directly. Here’s a bash script to convert that text file into a (very large) point shapefile:

#!/bin/bash
# geocoder2shp.sh - convert geocoder.ca CSV to a shape file
# NB: input CSV is UTF-8; it is passed through unchanged
# Needs >= v1.7 of GDAL
# scruss - 2012/04/15

if [ ! -f Canada.csv.gz ]
then
    echo ""
    echo " " Download \"Canada.csv.gz\" into the current directory from
    echo "  " http://geocoder.ca/onetimedownload/Canada.csv.gz
    echo " " and try again.
    echo ""
    exit 1
fi

# make input file with header
echo PostalCode,Latitude,Longitude,City,Province > Canada2.csv
gunzip -c Canada.csv.gz >> Canada2.csv

# create GDAL VRT file
cat > Canada2.vrt <<EOF
<OGRVRTDataSource>
  <!-- note that OGRVRTLayer name must be basename of source file -->
  <OGRVRTLayer name="Canada2">
    <SrcDataSource>Canada2.csv</SrcDataSource>
    <GeometryType>wkbPoint</GeometryType>
    <LayerSRS>EPSG:4326</LayerSRS>
    <GeometryField encoding="PointFromColumns" x="Longitude" y="Latitude"/>
    <Field name="PostalCode" type="String" width="6" />
    <Field name="Latitude" type="Real" />
    <Field name="Longitude" type="Real" />
    <Field name="City" type="String" width="60" />
    <Field name="Province" type="String" width="2" />
  </OGRVRTLayer>
</OGRVRTDataSource>
EOF

# create shapefile
ogr2ogr PostalCodes.shp Canada2.vrt

# clean up
rm -f Canada2.csv	Canada2.vrt

Though the script is a bit Unix-centric, it’s just a simple list of instructions which could be run on any command line. What it does is add some headers to the geocoder.ca file, then sets up an OGR Virtual Format to convert the text into a fairly well-defined shapefile. When you use this shapefile, you should credit geocoder.ca as the ODbL requires.

Eek! geocoder.ca has been sued by Canada Post! (News responses: Michael Geist, Boing Boing, CBC) I’ve donated to defend this useful service.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Some of the design packages I rely on use very crude GIS facilities. In fact, all they can support is a georeferenced raster as a background image, so it’s more of a rough map than GIS. It helps if these rasters are at a decent resolution, typically 1m/pixel or better.

A while back, I asked on the QGIS forum if the package could output high resolution georeferenced rasters. I received a rather terse response that no, it couldn’t (and I inferred from the tone that the poster thought that it shouldn’t, and I was wrong to want such a thing). I shelved the idea at the time.

After having to fix a lot of paths in a QGIS project file I’d moved to a new system, I noticed that all the map composer attributes are rather neatly defined in the XML file structure. Some messing around with Perl, XML::Simple and Data::Dumper::Simple, and I had a little script that would spit out an ESRI World File for the map composer raster defined in the project.

To run this, you have to create a project with just one Print Composer page, save the composed map as an image, save the project, then run the script like this:

./geoprint.pl project.qgs > image.pngw

There are some caveats:

• This probably won’t work for projects with multiple print composers
• It doesn’t quite get the scale right, but it’s within a pixel or so. I may not have corrected for image borders.

Though there’s some fairly hideous XML-mungeing in the code, what the script does is entirely trivial. If you feel you can use it, good; if you feel you can improve it, be my guest.

#!/usr/bin/perl -w
# geoprint - georef a QGIS output image by creating a world file
# one arg: qgis project file (xml)
# $Id: geoprint.pl,v 1.3 2012/04/06 03:32:01 scruss Exp $

use strict;
use XML::Simple;
use constant MM_PER_INCH => 25.4;

my $qgis = $ARGV[0];
die "$qgis must exist\n" unless ( -f $qgis );

my $q = XMLin($qgis) or die "$!\n";
my %composer = %{ $q->{Composer} };
my $image_width =
  int( $composer{Composition}->{paperWidth} *
    $composer{Composition}->{printResolution} /
    MM_PER_INCH );
my $image_height =
  int( $composer{Composition}->{paperHeight} *
    $composer{Composition}->{printResolution} /
    MM_PER_INCH );

# we need xpixelsize, ypixelsize, ulx and uly
my $xpixelsize =
  ( $composer{ComposerMap}->{Extent}->{xmax} -
    $composer{ComposerMap}->{Extent}->{xmin} ) /
  int( $composer{ComposerMap}->{ComposerItem}->{width} *
    $composer{Composition}->{printResolution} /
    MM_PER_INCH );
my $ypixelsize =
  -1.0 *
  ( $composer{ComposerMap}->{Extent}->{ymax} -
    $composer{ComposerMap}->{Extent}->{ymin} ) /
  int( $composer{ComposerMap}->{ComposerItem}->{height} *
    $composer{Composition}->{printResolution} /
    MM_PER_INCH );
my $ulx =
  $composer{ComposerMap}->{Extent}->{xmin} -
  $xpixelsize *
  int( $composer{ComposerMap}->{ComposerItem}->{x} *
    $composer{Composition}->{printResolution} /
    MM_PER_INCH ) - $xpixelsize;
my $uly =
  $composer{ComposerMap}->{Extent}->{ymax} -
  $ypixelsize *
  int( $composer{ComposerMap}->{ComposerItem}->{y} *
    $composer{Composition}->{printResolution} /
    MM_PER_INCH ) - $ypixelsize;

printf( "%.12f\n%.12f\n%.12f\n%.12f\n%.12f\n%.12f\n",
  $xpixelsize, 0.0, 0.0, $ypixelsize, $ulx, $uly );

# FIXME? pixel scale seems a tiny bit off - allow for border?
exit;

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

At the Toronto Hack Weekend, Henk Hoff presented Richard with a very nice Soft Cities – Midnight Commander Blanket. Soft! Mappy!!

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

I caved, and bought a Garmin nüvi 1490. Not that my GPSMap 60CSx wasn’t great at routing, but remembering to update maps before travel and its fiddly mounting requirements were a pain.

So, two hours of downloading map updates, and I fire it up … to find that the three year old hotel in Dartmouth, NS we were staying in was in the wrong place on the map:

Compare with OpenStreetMap:

So now, back near home, I ask it to find a post office. Here’s me parked outside one; do you see it on the screenshots?

Again, compare with OpenStreetMap:

So I thought I’d help Garmin out, but their Report a Map Error page needs me to know the type of my GPS, its serial number, the type and revision of my map, my name and e-mail address, and the coordinates of the error. OSM has me spoiled: at best, I can go in and edit; at second best, I can drop markers on OpenStreetBugs to flag errors for others to fix.

Garmin already has my name, e-mail address, GPS type, serial number and map revision through myGarmin™. The company could just as easily have a web-based map correction system that would be point-and-click. Follow the Leader is one of Garmin’s mottoes. In terms of user correctability of maps, however, they’re only the leader because they don’t know they’ve been lapped.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Looks like the data sets at toronto.ca/open might finally actually be open; that is, usable in a way that doesn’t bind subsequent users to impossible terms. The new licence (which unfortunately is behind a squirrelly link) basically just requires you to put a reference to this paragraph somewhere near your data/application/whatever:

Contains public sector Datasets made available under the City of Toronto’s Open Data Licence v2.0.

and a link to the licence, where possible.

Gone are the revocation clauses, which really prevented any open use before, because they would require you to track down all the subsequent users of the data and get them to stop. Good. I think we can now use the data in OpenStreetMap.

While commenting on the licence’s squirrelly URL — I mean, could you remember http://www1.toronto.ca/wps/portal/open_data/open_data_fact_sheet_details?vgnextoid=59986aa8cc819210VgnVCM10000067d60f89RCRD? — I stumbled upon the comedy gold that is the City of Toronto Comments Wall log. There goes my planned reading for the day.

Related posts:

1. quite a bit of work to do in Toronto Looking at the OSM Inspector for Toronto, there are still...

Related posts brought to you by Yet Another Related Posts Plugin.

Pierre Béland reminded the Canadian OSM group of the GeoBase WMS, which has all the NRN road names in it. Here’s my neighbourhood’s OpenStreetMap data with the road names overlaid.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Looking at the OSM Inspector for Toronto, there are still a lot of nodes that need to be remapped (or OpenStreetMap users that need to be advised) when the big licence change comes.

(and yes, I blogged this just so I’d remember the link for OSM Inspector.)

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

All I really wanted to do is make a map like this:

This is an Azimuthal Equidistant projection, with me at the centre (of course) and the rest of the world spread out in a fan by distance and bearing. It’s somewhat surprising to find that South Africa is almost directly east of Toronto, and New Zealand to the southwest.

If I had a directional antenna and a rotor, this map would show me where I would have to point the antenna to contact that part of the world. I can’t rotate my dipole (unless I commit some unauthorized local plate tectonics) so I’m stuck with where my antenna transmits and receives best.

The above map was made with AZ_PROJ, a PostScript program of some complexity for plotting world maps for radio use. The instructions for installing and running AZ_PROJ are complex and slightly dated. I got the above output running it through Ghostscript like this:

gs -sDEVICE=pdfwrite -sOutputFile=va3pid_world.pdf az_ini.ps -- az_proj.ps world.wdb

The format of the az_ini.ps file is complex, and I’m glad I’m an old PS hacker to be able to make head or tail of it.

For all its user-hostility, AZ_PROJ is powerful. Here’s a version of the map I wanted all along:

This shows my furthest QSO in each of the 16 compass directions. (You might note that North is empty: my furthest contact in that direction is some 13km away, whether by lack of folks in that sector or dodginess of my antenna.) Contrast that with my Mercator QSO map, and you’ll see that Azimuthal Equidistant is a much better projection for this application.

To show how radically different the world looks to different people, here’s the world according to my mate Rob in Hamilton, NZ:

I’d been trying to use OGR to transform arbitrary shapefiles into this projection. For maps entirely contained within the same hemisphere (so having extent less than ±90° in any cardinal direction), this works:

ogr2ogr -t_srs "+proj=aeqd  +lat_0=43.7308 +lon_0=-79.2647" out.shp in.shp

The lat_0 and lon_0 parameters are just where you want the centre of the map to be. Things get a bit odd if you try to plot the whole world:

The antipodes get plotted underneath, and everything looks messed up. I may have to take my question to GIS – Stack Exchange to see if I can find an answer. Still, for all its wrongness, you can make something pretty, like my whole world Maidenhead locator grid projected this way turns into a rose:

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

I was trying to sort a series of polar coordinates by compass direction: everything in the N sector, then NNE, NE, ENE, E, … This is trickier than it looks, since the North sector includes angles between 348.75° and 11.25°.

It’s easier to think of the problem instead of being one of N bins centred on 0°, but rather one of 2N bins starting at 0°. Then, all one needs to do is shift everything round by one of these half bins (so +1, modulo 2N), then divide the number of bins back to the number we want. Done!

Here’s some code to do it. It may look like C-like pseudocode, but it’s real live AWK:

function bin(angle,nbins){
  return int(((int(angle/(360.0/(nbins*2)))+1)%(nbins*2))/2);
}

You probably want the angles to be between 0..360°. But you knew that.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Automatic Packet Reporting System — APRS — is rather clever. It’s a way of reporting position, status or messages via the amateur radio 2m band. Data is relayed via digipeaters, and routed to/from the internet APRS-IS system to any user worldwide.

It’s a little fiddly to set up, even with a very polished (read: $$) handheld radio like the Kenwood TH-D72A. I’m a bit disappointed that the purported SiRFstar III GPS in this radio takes forever to get a lock, but it’s a nice radio despite this.

The screenshot above shows aprs.fi‘s tracking of my handheld (VA3PID-7) last night as I walked to Toronto Mappy Hour.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.