Drawing Eggs

When designing wind farms, you need to keep the turbines a certain distance apart from one another. If you don’t, the wakes from the turbines reduce efficiency, and the turbulence can reduce the warranted life of the machine. Typically, a manufacturer might specify a minimum downwind separation of 5 diameters, and a crosswind separation of 3 diameters. It’s an easy check with a buffer overlap, but these buffers are elliptical, which not all GIS packages can draw.

Take, for example, the following three points designated as (completely made-up)  wind turbine locations:

name    XCOORD           YCOORD
1    557186.675000    4757125.590000
2    557447.931000    4756968.690000
3    557664.999000    4756817.810000

These look quite far apart, even if you were using large, 100+m diameter wind turbines:

But if we have a wind direction of 210°, downwind/crosswind separation of 5D & 3D respectively, and a 101m diameter rotor, it’s not so good:

Turbines 2 & 3 are too close together; the ellipses shouldn’t touch.

As awk is the only scripting language I have on my work computer, I wrote the script that generates the buffer shapefile in awk. The script calls Frank’s Shapefile C Library utilities to actually make the shapefile. Here’s the code:

#!/bin/awk -f
# draw an ellipse based on turbine location to generate
#  for WTG separation buffer
# scruss - 2011-09-27

# assumes that stdin has three columns:
#  1 - label
#  2 - x coordinate
#  3 - y coordinate

# variables:
#  diameter = rotor diameter
#  cross = crosswind separation, diameters
#  down = downwind separation, diameters
#  wind = prevailing wind direction
#  base = base for shape file name

BEGIN {
	OFMT="%.1f";
	CONVFMT="%.1f";
	OFS=" ";

	if (diameter < 0) {
		print "diameter must be set";
		exit;
	}

	if (cross < 0) {
		print "cross must be set";
		exit;
	}

	if (down < 0) {
		print "down must be set";
		exit;
	}

	if (down < cross) {
		print "down must be greater than cross";
		exit;
	}

	if (wind < 0) {
		print "wind must be set";
		exit;
	}

	if (base ~ /^$/) {
		print "base must be a string";
		exit;
	}

	pi = 3.141592654; # I know, I know ...
	# calc cartesian angle from wind bearing, in radians
	beta = ((450 - wind)%360) * pi/180;

	# output shapelib tools init commands
	print "dbfcreate " base " -s name 40";
	print "shpcreate " base " polygon";
}

# for every line
{
	name=$1;
	x=$2;
	y=$3;

	major = diameter * down/2;
	minor = diameter * cross/2;
	first="";
	points="";
	maxn=36;
	for (i=0; i<maxn; i++) {
	    alpha = (i * (360/maxn)) * pi/180;
	    x1 = x + major * cos(alpha) * cos(beta) - minor * sin(alpha) * sin(beta);
	    y1 = y +  major * cos(alpha) * sin(beta) + minor * sin(alpha) * cos(beta);
	    if (i == 0) { # store the first point
		first= x1 " " y1;
	    }
	    points = points  " " x1 " " y1;
	}
	points = points  " " first;
	print "dbfadd " base ".dbf " name;
	print "shpadd " base, points;
}

awk is charmingly odd in that you can specify variable on the command line. Here’s how I called it, with the above coordinates as input:

awk -v diameter=101 -v cross=3 -v down=5 -v wind=210 -v base="fakewtg-ellipse" -f separation.awk

Pipe the output through a shell, and there are your ellipses.

Posted in GIS | Tagged , , , , | Leave a comment

#mapfail

… or “トロント動物園“, as we locals purportedly call it. The This place has unverified edits legend is a bit of a giveaway. Looks like there’s been some messing about with Google Map Maker, which isn’t always the best tool for the job.

Posted in Uncategorized | Tagged , , , , , | Leave a comment

Ham Radio log to interactive OpenStreetMap

You might notice that there’s now a Ham Radio QSO Map lurking on the front page. Thanks to the WordPress OpenStreetMap plugin (which I’ve slightly abused before). Here’s a small piece of Perl which will take your ADIF log and convert it to a WP-OSM marker file.

Note that this program assumes you’ve downloaded your log from QRZ.com, as it requires the locator field for both inbound and outbound stations.

#!/usr/bin/perl -w
# adif2osm - convert ADIF log to OSM map file
# scruss.com / VA3PID - 2011/06/19

use strict;
use constant MARKERDIR =>
  'http://glaikit.org/wp-content/plugins/osm/icons/';
use constant QRZURL => 'http://qrz.com/db/';
sub maidenhead2latlong;

my ( $temp, @results ) = '';

### Fast forward past header
while (<>) {
  last if m/<eoh>\s+$/i;
}

### While there are records remaining...
while (<>) {
  $temp .= $_;

  ### Process if end of record tag reached
  if (m/<eor>\s+$/i) {
    my %hash;
    $temp =~ s/\n//g;
    $temp =~ s/<eoh>.*//i;
    $temp =~ s/<eor>.*//i;
    my @arr = split( '<', $temp );
    foreach (@arr) {
      next if (/^$/);
      my ( $key, $val ) = split( '>', $_ );
      $key =~ s/:.*$//;
      $hash{ lc($key) } = $val unless ( $key eq '' );
    }
    push @results, \%hash;
    $temp = '';
  }
}

# generate OSM plugin file
my @data = ();
my ( $mygrid, $station_callsign ) = '';

# output header
print
  join( "\t", qw/lat lon title description icon iconSize iconOffset/ ),
  "\n";
foreach (@results) {
  next unless ( exists( $_->{gridsquare} ) && exists( $_->{call} ) );
  $mygrid = $_->{my_gridsquare}
    if ( exists( $_->{my_gridsquare} ) );
  $station_callsign = $_->{station_callsign}
    if ( exists( $_->{station_callsign} ) );

  push @data, $_->{freq} . ' MHz' if ( exists( $_->{freq} ) );
  $data[$#data] .= ' (' . $_->{band} . ')' if ( exists( $_->{band} ) );
  push @data, $_->{mode} if ( exists( $_->{mode} ) );
  push @data, $_->{qso_date} . ' ' . $_->{time_on} . 'Z'
    if ( exists( $_->{qso_date} ) && exists( $_->{time_on} ) );
  my ( $lat, $long ) = maidenhead2latlong( $_->{gridsquare} );
  print join( "\t",
    $lat,
    $long,
    '<a href="' . QRZURL . $_->{call} . '">' . $_->{call} . '</a>',
    join( ' - ', @data ),
    MARKERDIR . 'wpttemp-green.png',
    '0,-24' ),
    "\n";

  @data = ();
}

# show home station last, so it's on top
my ( $lat, $long ) = maidenhead2latlong($mygrid);
print join( "\t",
  $lat,
  $long,
  '<a href="'
    . QRZURL
    . $station_callsign . '">'
    . $station_callsign . '</a>',
  'Home Station',
  MARKERDIR . 'wpttemp-red.png',
  '0,-24' ),
  "\n";

exit;

sub maidenhead2latlong {

  # convert a Maidenhead Grid location (eg FN03ir)
  #  to decimal degrees
  # this code could be cleaner/shorter/clearer
  my @locator =
    split( //, uc(shift) );    # convert arg to upper case array
  my $lat      = 0;
  my $long     = 0;
  my $latdiv   = 0;
  my $longdiv  = 0;
  my @divisors = ( 72000, 36000, 7200, 3600, 300, 150 )
    ;                          # long,lat field size in seconds
  my $max = ( $#locator > $#divisors ) ? $#divisors : $#locator;

  for ( my $i = 0 ; $i <= $max ; $i++ ) {
    if ( int( $i / 2 ) % 2 ) {    # numeric
      if ( $i % 2 ) {             # lat
        $latdiv = $divisors[$i];    # save for later
        $lat += $locator[$i] * $latdiv;
      }
      else {                        # long
        $longdiv = $divisors[$i];
        $long += $locator[$i] * $longdiv;
      }
    }
    else {                          # alpha
      my $val = ord( $locator[$i] ) - ord('A');
      if ( $i % 2 ) {               # lat
        $latdiv = $divisors[$i];    # save for later
        $lat += $val * $latdiv;
      }
      else {                        # long
        $longdiv = $divisors[$i];
        $long += $val * $longdiv;
      }
    }
  }
  $lat  += ( $latdiv / 2 );         # location of centre of square
  $long += ( $longdiv / 2 );
  return ( ( $lat / 3600 ) - 90, ( $long / 3600 ) - 180 );
}

You’ll need to update MARKERDIR to reflect your own WP-OSM installation. Mine might move, so if you don’t change it, and you don’t get markers, please don’t blame me.

The basic code to include a map is like this:

You’ll need to change the marker_file URL, too.

Note that, while this script generates links into the QRZ callsign database, it doesn’t hit that site unless you click a link.

Posted in GIS | Tagged , , , , , , , , , | 1 Comment

Bixi comes to Toronto!

BIXI Toronto is (nearly) here!

Here are the proposed station locations: Bixi_Toronto_shp.zip (Shapefile) or Bixi_Toronto-kml_csv.zip (KML and CSV).

Posted in GIS | Tagged , , , , | Leave a comment

More radio amateur grid squares

Toronto, as understood by the Maidenhead Locator system

After yesterday’s post, I went a bit nuts with working out the whole amateur radio grid locator thing (not that I’m currently likely to use it, though). I’d hoped to provide a shapefile of the entire world, but that would be too big for the format’s 2GB file size limit.

What I can give you, though, is:

  • A Perl program that will generate a shapefile of an entire Maidenhead grid field, down to the subsquare level: make_grid.pl. You’ll need Geo::Shapelib to make this work. 324 (= 182) of these files would cover the whole world, and at 8MB or so a pop, things get unwieldy quickly.
  • A Google Earth KML file covering the whole world in 20° by 10° grid fields: Maidenhead_Locator_World_Grid. (If you’re feeling nerdy, here it is in Shapefile format: Maidenhead_Locator_World_Grid-shp).

If anyone would like their grid square in Google Earth format, let me know.

Posted in GIS | Tagged , , , , , , | Leave a comment

Maidenhead Grid Locator, in Perl

Amateur radio operators sometimes use the Maidenhead Grid Locator Square system to identify location. Here’s my attempt at a translator, written in slightly squirrelly Perl:

#!/usr/bin/perl -w
# maidenhead - 6 character locator
# created by scruss/VA3PID on 02011/04/01
# RCS/CVS: $Id: maidenhead,v 1.3 2011/04/03 11:04:38 scruss Exp $

use strict;
sub latlong2maidenhead;
sub maidenhead2latlong;

if ( $#ARGV >= 1 ) {    # two or more args
                        # convert lat/long to grid
  print latlong2maidenhead( $ARGV[0], $ARGV[1] ), "\n";
}
elsif ( $#ARGV == 0 ) {    # one arg
                           # convert grid to lat/long
  print join( ", ", maidenhead2latlong( $ARGV[0] ) ), "\n";
}
else {                     # no args

  # print usage
  print 'Usage: ', $0, ' dd.dddddd ddd.dddddd', "\n",
    ' or', "\n",
    $0, ' grid_location', "\n";
}
exit;

sub maidenhead2latlong {

  # convert a Maidenhead Grid location (eg FN03ir)
  #  to decimal degrees
  # this code could be cleaner/shorter/clearer
  my @locator =
    split( //, uc(shift) );    # convert arg to upper case array
  my $lat      = 0;
  my $long     = 0;
  my $latdiv   = 0;
  my $longdiv  = 0;
  my @divisors = ( 72000, 36000, 7200, 3600, 300, 150 )
    ;                          # long,lat field size in seconds
  my $max = ( $#locator > $#divisors ) ? $#divisors : $#locator;

  for ( my $i = 0 ; $i <= $max ; $i++ ) {
    if ( int( $i / 2 ) % 2 ) {    # numeric
      if ( $i % 2 ) {             # lat
        $latdiv = $divisors[$i];    # save for later
        $lat += $locator[$i] * $latdiv;
      }
      else {                        # long
        $longdiv = $divisors[$i];
        $long += $locator[$i] * $longdiv;
      }
    }
    else {                          # alpha
      my $val = ord( $locator[$i] ) - ord('A');
      if ( $i % 2 ) {               # lat
        $latdiv = $divisors[$i];    # save for later
        $lat += $val * $latdiv;
      }
      else {                        # long
        $longdiv = $divisors[$i];
        $long += $val * $longdiv;
      }
    }
  }
  $lat  += ( $latdiv / 2 );         # location of centre of square
  $long += ( $longdiv / 2 );
  return ( ( $lat / 3600 ) - 90, ( $long / 3600 ) - 180 );
}

sub latlong2maidenhead {

  # convert a WGS84 coordinate in decimal degrees
  #  to a Maidenhead grid location
  my ( $lat, $long ) = @_;
  my @divisors =
    ( 72000, 36000, 7200, 3600, 300, 150 );    # field size in seconds
  my @locator = ();

  # add false easting and northing, convert to seconds
  $lat  = ( $lat + 90 ) * 3600;
  $long = ( $long + 180 ) * 3600;
  for ( my $i = 0 ; $i < 3 ; $i++ ) {
    foreach ( $long, $lat ) {
      my $div  = shift(@divisors);
      my $part = int( $_ / $div );
      if ( $i == 1 ) {    # do the numeric thing for 2nd pair
        push @locator, $part;
      }
      else {              # character thing for 1st and 3rd pair
        push @locator,
          chr( ( ( $i < 1 ) ? ord('A') : ord('a') ) + $part );
      }
      $_ -= ( $part * $div );    # leaves remainder in $long or $lat
    }
  }
  return join( '', @locator );
}

Grid square to lat/long and grid square shapefiles to follow.

Posted in GIS | Tagged , , , | 4 Comments

image georeferencing with QGIS

Quantum GIS (QGIS) has a very powerful image georeferencing module. What that allows you to do is convert screen pixels to a map of an area. The pixels could be from a scanned map or from a screen image. Scanned maps can sometimes be a little distorted, but QGIS’s georeferencer can handle that, within limits.

For an example, say I live in Redickville, ON (I don’t, but some folks do). I’ve heard from North Dufferin Agricultural & Community Taskforce (NDACT) that a large quarry is planned in my area. How close am I going to be from it?

NDACT has a helpful map (2.4 MB PNG image) had a helpful map (which I’ve kept here so you can work through this: MelancthonAerial), but it has no scale. It’s clearly derived from something like Google Maps, so it’s not exactly a free image I can throw about. One thing about georeferencing is that both the source image and the map from which you take you control points affect the licensing of the final map. You’re ending up with a derived work.

There are lots of sources of coordinates for your control points. You could always use Google Maps, but then you’re well down the derived work sinkhole. GeoGratis has a bunch of good data sources, and they are free to use. I’m going to use Toporama‘s digital image maps, as they’re clear and fairly accurate.

Redickville is on Toporama sheet 041A01. I’ve downloaded it as UTM, as it’s easier to measure distances that way. You want to set up your project so it uses the coordinate reference system (CRS) that the control point map uses. Toporama uses EPSG 26917 in the area (easily checked with gdalinfo; it’ll come up with something like AUTHORITY["EPSG","26917"]]), so you should set the project to use that CRS:

And here’s the raster map loaded into QGIS;

Opening up the georeferencer plugin gives you a whole lot of blank:

If we open the raster, you can zoom into the area into which you want to put control points. You want to have the highest zoom that the map’s still clear, as the accuracy of your final map depends on how well you placed control points. I’ve chosen road intersections as my control points. Here are the ones I’m going to use:

Point   E-W Road        N-S Road        Note
1       County Rd 21    2nd Line W      Honeywood
2       County Rd 21    4th Line       
3       20th Side Rd    5th Line       
4       15th Side Rd    County Rd 124   just S of where 124 straightens
5       15th Side Rd    Mulmur Townline
6       4th Line        5th Line        4th Line really runs SE-NW

These intersections are fairly well spaced apart, and are clear on both maps. So I choose a pixel on the map in the georeferencer, and a dialogue comes up:

(If you’ve downloaded the MelancthonAerial archive from here, the georeferencing points should load automatically from the “Melancthon Aerial July 22 09.png.points” file. If you want to go through the exercise of manually adding reference points, delete the points file.)

Here I’ve already clicked on the corresponding point on the Toporama map, and the coordinates have been filled in. If you know the coordinates, you can enter them in the boxes. Once you click OK, you have your first control point:

We can add the rest one by one. QGIS’s “Zoom Previous” is really useful for flipping between scales on both the plugin window and the main map. It’s probably a good idea to “Save GCP Points As …” every now and again so you don’t lose your work. Here are all of the points in the georeferencer plugin:

Now you want to modify the Transformation Settings; it’s the little wrench/spanner in the toolbar:

There are lots of options here:

  • Transformation Type: Linear is useful if you’re just adding georeference data to an already computer generated map. Helmert is a simple shear/scale/rotate transformation. The various Polynomial types will correct more gross distortions, but need many control points and can be processor intensive. Thin Plate Spline will distort your map locally to match GCPs; this can work well if your map’s a bit “vernacular”, but if your control points are wrong, your map will end up hilariously melty.
  • Resampling Method: This controls how the output pixels are calculated. Nearest Neighbour is quick and blocky, but useful if you’re mapping an image that has sharp transitions. Cubic maintains more detail, while applying some smoothing at the cost of some detail loss and a fair bit of processing power. The other options can look nice, but eat CPU. This is worth experimenting with many options, as there’s no one solution for all maps.
  • Compression: This controls the file size of the output GeoTiff file. JPEG and Deflate can result in small files, but there’s a chance that other GIS systems can’t read the data. Note that JPEG is lossy, and will lose some detail.

Don’t forget to set the output raster file, and make sure that the target SRS matches the CRS (EPSG 26917) you chose earlier.

Helmert is a useful transformation type to check your work. The plugin plots the residual difference between the two sets of map control points as red lines. Here, I’ve clearly made an error in point ID 2:

If you unselect the bad point, the plugin quickly calculates the residuals again. When this one bad point is removed, the residuals drop down to less than 1.0 for each point. I’ve got enough points for a Helmert transformation with 5 GCPs, but I’d probably want some more points for more complex transformations.

Once you hit “Start Transformation”, QGIS will create your referenced image. If you’ve chosen a simple linear transformation, it won’t create a GeoTiff file, but just a world file for your image.

So here’s the georeferenced image overlaid on the Toporama map, with a bit of transparency. It’s quite a good match:

(the above’s an image saved straight from QGIS. It helpfully creates a world file too, so here it is: redickville.jpgw)

So to answer the hypothetical question, Redickville’s pretty well surrounded by this proposed quarry.

Posted in GIS | Tagged , , , , , , | 2 Comments

Garmin vs Arduino GPS dance-off

So I built a GPS logger. It’s an Arduino Uno with the SparkFun microSD and GPS shields. No display, massively unwieldy, but it does spit out a track point every second and logs it fairly reliably to microSD. It works as well as my expensive old Garmin GPSMap 60Csx, as a walk around the block shows:

Excuse the noise at the top of the trace; both units were brought inside. While they keep a tenuous lock, they certainly don’t give much accuracy. I think the Arduino did rather well; certainly better than my BlackBerry

Now all I need is a case and a more useful power supply. I was disappointed that a USB MintyBoost charger didn’t seem to work for long, causing the GPS to lock up.

Posted in GIS | Tagged , , , | Leave a comment

fixing garmin file dates

While the Garmin GPSMap 60csx is a lovely unit, it saves its tracks on the card with a date just slightly younger than I am. The following Unix one-liner will correct the file dates to the actual dates the data were collected:

for f in 20??????.gpx; do touch -t `basename $f .gpx`2359.59 $f; done

I remember having a really awesome reason for making the time for each file 23:59:59, but I’ve completely forgotten what it was. Since all I remember was the awesomeness, I see no reason to change it …

Posted in GIS | Tagged , , , , , | Leave a comment

The SpatiaLite Cookbook

Alessandro‘s new SpatiaLite Cookbook is rather good.

Posted in GIS | Tagged , | Leave a comment