Geeking out on Maps

It has been a pretty productive 10 days or so - I started on Friday February 23rd manually entering data into a google map from the La Verendrye Park maps. I've long bemoaned the fact that there was next to no good data out there for this gem of a park for paddling enthusiasts, so I decided to do something about it. After 4 days and probably about 20 hours or so you can see the results linked at the top of this site. It is not perfect and there are still a few maps missing, but it is in my opinion a pretty useful tool for planning trips.


Then this weekend I started digging into some of the Algonquin Park data that you can download from the Paddle Planner website. I'm not sure where they got the data but it is pretty useful if a bit too much. Initially I tried to straight import it into google maps and that's when I discovered that google static maps have a 2000 object limit. The file from paddle planner contains about 75,000 objects including 2135 campsites, about 750 portages, and the rest are a bunch of lines which draw the outline of all the lakes and bodies of water both within the park as well as within a certain radius around it.
I examined the data for a long time to see if there was a way to programmatically separate it out. I guess now that I have a keen interest in GPS data I should finally learn how to parse XML files because both GPX and KML files are basically a type of XML. It was relatively easy to manually separate out the campsites into their own file since they were marked differently in the file than the portages and other data. But I still had the 2000 object limit and there are just over 2000 campsites in Algonquin Park! This forced me to do some googling upon which I discovered Google Fusion Tables which offer a way to store GPS data in a data file in your google account, and then access it programmatically to draw maps. I've managed to put together a few things with that and you can see the results in the links above.
Separating out the portage routes from the Paddle Planner data was a lot more challenging since inside the lines denoting the portages and the lines outlining the lakes were all basically identically within the file. There was no way to tell one from the other. But I did eventually have a bit of an epiphany. I was playing around some more with the full dataset and trying different things, and noticed that it seemed to be the case that portage routes appeared first in the file, and then some time later came the outlines of lakes and such. And also noticed that stuff inside the park came before stuff outside the park.
So the next part was not easy ... a lot of manual editing.
I started by going into my editor (vi of course) and counting out the first 500 lines drawn and saving that to its own file. Then I loaded it into google maps to see what it looked like. Yup, I was on to something.
Knowing the line number in the file for the 500th line, I went back to the original and counted out 100 more lines. Then saved that and loaded it into google maps.
I repeated that process manually adding 100 lines each time until I got to a point where it looked like it was adding lakes and points outside the park. I think it was either 800 or 900 I forget now. But once I got there I started doing something similar but starting with the full 800 or 900 and taking away about 7 or 8 at time.
I repeated that process 4 or 5 times until I finally think I identified the last portage. So a simple binary search from there and I finally had a map of Algonquin portages! To give you an idea of what I was up against here is a map showing all of the original data for the portages (minus the campsites).
I do not yet have a 100% guarantee this is complete, but based on what I saw I think it is.
I'll play around some more with the remainder of the data to be sure.
Then I saw someone on the MyCCR Forums posting about the Woodland Caribou Provincial Park, Atikaki Provincial Park, and Nopiming Provincial Parks, which traverse the Manitoba-Ontario border. I recalled seeing some GPS data for these guys on the Paddle Planner website as well so I went to get it. When I started to examine it I noticed that it had the portages listed in rods FFS! WTF was that all about! Not being very happy about this and thinking that I could improve it by converting it to meters, I set about figuring out a way to do that. In the end I issued a challenge to my geekier friends on facebook, and within 30 minutes of that challenge I had 2 different programs that would parse a text file and convert all instances of "xxx rods" to "yyy meters". The results can be seen above. It is a small improvement on the original GPS dataset, but an improvement just the same.
Anyway, I've been geeking out on maps and map data. My knowledge is still in its infancy, but as I learn I hope to provide better-and-better tools for people planning backwoods paddling trips.
Please stay tuned ...