Lachlan.Rogers.name » Hacks and Tricks

Tau really does make more sense

Lachlan — Mon, 04 Jul 2011 12:54:46 +0000

Tau Day is celebrated with two pi(e)s, and we couldn't let that expectation go unfulfilled.

Pi is wrong. This startling assertion became so abundantly clear to me last week that I was surprised it has taken this long for me to encounter the arguments. Of course, as the number of diameters in a circle’s circumference then pi = 3.141592 653589 793238 462643 383279 50288… is technically correct. But it is wrong conceptually! Mathematically it is the radius, not the diameter, which is the defining dimension of a circle.

And so a better circle constant is τ (tau) =2π. Happily I learnt of this just in time to celebrate Tau Day.

There are many excellent reasons why tau is better than pi, and I won’t bother presenting them all here. Michael Hartl makes the argument convincingly in The Tau Manifesto if you are looking for some reading, but this video presents the salient points in a thoroughly entertaining way.

If you’re strugging to catch on to tau as the new circle constant, then maybe this musical representation of the number will help.

We dutifully and cheerfully celebrated Tau Day with two pies, and it is a shame that next June 28 is a year away. I guess that is a full year in which to extol the virtues of Tau.

And there’s another job to do. The 36 digits of pi listed above were written from memory. I have them firmly lodged in my brain as a result of a friendly competition in grade 5. Now I’m going to have to memorise at least 37 digits of tau!

-----

(This post was published on my Lachlan.Rogers.name website. If you are reading this on Facebook Notes or Google Buzz or another RSS import, please consider commenting on the original post to keep the conversation accessible to everyone.)

Revisiting categories, and generally improving this site

Lachlan — Mon, 17 May 2010 11:56:38 +0000

When I first started using WordPress to construct this website, it had “categories” and not “tags”. Since then, I have tried to improve my use of categories and tags have come along to add another complication to the fray. I have now finally reached a system that I really like.

One of the difficulties of a personal website (as opposed to a blog on a specific topic) is that there is such a variety of post types. Sometimes the site becomes a little soap-box where essays can be delivered. Other times, it is a news feed to keep family and friends updated about exciting travel activities. It is also a valuable way to store and share helpful tricks and insights.

So I have now culled my category list down to six distinct topics, and a seventh “uncategorised” overflow pigeonhole. Within these broad categories, posts are further grouped by tags. It is possible to browse posts by category, or by tag. While I’m not claiming that this sort of organising strategy is anything new or unique, I think it is finally a functional way to hold this site together.

I have also revamped this site visually and under the bonnet. Here are some of the more technical details.

There are times when I need more space for a wide post (such as for this embedded interactive map with geotagged photos). To do this, I created a “Without sidebar” category and then told my wordpress to leave out the sidebar for any post in this category. After reorganising, I didn’t want this category to feature in my condensed list.

I was able to exclude it by modifying the category widget section of the file wp-includes/default-widgets.php to replace the line

$cat_args = array('orderby' => 'name', 'show_count' => $c, 'hierarchical' => $h);

with

$cat_args = array('orderby' => 'name', 'show_count' => $c, 'hierarchical' => $h, 'exclude' => '27');

(where 27 is the number of my “Without sidebar” category). I found the details for this here, but had to work out how to apply them to my situation.

Another thing that has changed since I started this site is the incorporation of my RSS feeds into other online social environments. My posts here get imported to my Facebook profile as notes, and they also get pulled in as Buzzes with Google. Both these locations allow people to comment on my writing there rather than here, but I would prefer people to participate in the conversation here. This website is visible and accessible to people without requiring them to create a profile and log on.

I have found a wordpress plugin that will add a “signature” to my RSS feeds. If it works properly, my posts here that get displayed elsewhere will have a little comment encouraging readers to join the conversation on the original post.

-----

Renaming photos to give attribution

Lachlan — Tue, 02 Feb 2010 10:44:43 +0000

At ASA Convention, a number of us took many photographs. Late on the last night, I helped compile the highlights into a souvenir data DVD. Using the nifty little exiv2 utility, I was able to adjust the image timestamps and retrospectively synchronise the clocks in every camera. This makes for very enjoyable photo browsing, as you can view chronologically regardless of which camera was used.

Unfortunately I was too rushed to place attribution in the filenames, and so it is not obvious who took each photo. Luckily we all used different camera models, and that information is still present in the exif metadata. I used this to split the photos into directories and embed the photographer in both metadata and filename.

I was shooting on my Nikon D200, and so was able to separate my photos by:

for ii in *.jpg; do if grep -q "NIKON D200" $ii ; then mv $ii lachlan/ ; fi ; done

This says: for each jpg file, check if it contains “NIKON D200″ in the exif metadata and if it does then move it to the subdirectory lachlan/

All you need to do is find the camera model string to search for (my friends with Canons had things like “Canon EOS 30D”). Now I’m going to append photographer names to the filenames, and then return them to their original chronological directories.

[For Facebook Notes readers: this post is redirected from my personal website lachlan.rogers.name]

-----

Producing handout with LaTeX Beamer

Lachlan — Tue, 10 Nov 2009 12:23:52 +0000

I jotted this down a year ago when I needed to produce a set of handout notes for a 3rd year physics lecture I took. Just last week, after taking a similar set of lectures, I wanted to find it but couldn’t. Murphy’s Law has come into effect, and my jotted note has turned up now that my need for it has passed.

The Beamer class for LaTeX is a great way to produce very nice presentation slides with useful features such as automatic progress markers and internal hyperlinks. Being LaTeX, it is also possible to completely change the output formatting by simply altering certain document settings. This allows me to produce slides that have black backgrounds for better projection onto a screen, and then change a single line (specifying the colour theme) to get a white-background version optimised for printing on paper.

To make it even more efficient to print, I used the following command to fit 3 slides to an A4 page:

pdfnup --frame false --nup 1x3 --paper a4paper --orient auto --pages all --trim "0 0 0 0" --delta "1cm 1cm" --offset "0 0" --scale 0.91 --turn true --noautoscale false --openright false --column false --columnstrict false --tidy true --outfile main3up.pdf main.pdf

-----

Converting video files for playback on Android Google Phone

Lachlan — Thu, 24 Sep 2009 05:53:50 +0000

A few months ago I purchased a “Google Phone”: a HTC Magic running the Linux-based Android operating system. I am incredibly pleased with this device.

Today I wanted to convert some video that I recorded on a Nikon Coolpix camera into a format that would play nicely on my Android phone. The resolution needed to be scaled down for the small screen, and I knew that h264 was the best video codec. After some searching online, and a bit of experimenting, I found that this worked wonderfully:

ffmpeg -i input.avi -aspect 3:2 -s 400x300 -vcodec libx264 -b 480k -r 30 -acodec libfaac -ac 1 -ab 32k -padtop 10 -padbottom 10 -padleft 40 -padright 40 -sameq -pass 1 output.mp4

This is a two-pass process, and so this command should be repeated a second time with the last option changed to “-pass 2″.

The “pad” options are to make the geometry work. The screen on the HTC Magic has a resolution of 480×320 pixels, which is a 3:2 aspect ratio. The video I was converting had an aspect ratio of 4:3, and so it conveniently scaled to 400×300. The paddings fill the video out to the full 480×320, ensuring that the aspect ratio is right during playback.

-----

Photo file-management with raw and jpg

Lachlan — Tue, 16 Jun 2009 11:58:12 +0000

Even when shooting images in raw format, it is typically easier to do simple sorting and sharing with jpg files. With Nikon *.nef raw files, it is possible to extract a full-resolution jpg image using the nefextract script. However, exif metadata is not automatically copied to this extracted jpg.

The tool that I use to manipulate my images according to their exif metadata is exiv2, and I can quite simply copy metadata from raw files to their corresponding jpgs (matching filenames, and all in the one working directory) with:

exiv2 insert -l./ -S.nef *.jpg

After sorting through the jpgs and deleting all but those worth keeping, I wanted to automatically remove the raw files of those deleted images. Sure enough, this is easily done with some bash shell magic:

for file in *??.jpg; do mv raw/${file%%.jpg}.nef 2> /dev/null rawKeep/ ; done

This command says “for each jpg file you find here, move the matching nef file from the subdirectory raw/ to the subdirectory rawKeep/”. I can then delete any files left in the raw/ subdirectory, as they mustn’t have a matching jpg.

People often ask my why I persevere with the “command line” (more technically the “shell”). It seems that they assume tools with graphical interfaces are more powerful and faster. These two routine tasks demonstrate yet again that the shell really is the most efficient way to do many common jobs.

-----

Pretty display fonts from Latex

Lachlan — Wed, 18 Feb 2009 06:35:00 +0000

The fonts that LaTeX uses by default (called “Computer Modern”) are very elegant, and are one of the great reasons to use LaTeX for typesetting documents. Frustratingly, I’ve been having difficulty lately with my pdfviewer (epdfview and sometimes xpdf) not displaying these fonts well in pdfs I’ve created using LaTeX.

Adobe acroread has been able to do a better job, but who wants to use such a slow and bloated piece of software just to quickly view a pdf? At least the pdfs have printed properly, so up till now I haven’t been too worried.

The solution, which I found at a page encouragingly called “High quality PDF output from LaTeX and TeX”, is profoundly simple. It simply involves telling LaTeX to use the scalable font rather than the older bitmap version. Add these two lines to the preamble:
\usepackage[T1]{fontenc} \usepackage{ae,aecompl}
and you’re done.

-----

Foray into geotagging photos

Lachlan — Mon, 02 Feb 2009 05:18:03 +0000

One of the really great things about storing photographs digitally is the ability to embed all sorts of information about the image in “metadata” within the file. When I shot film, I carried a notebook around with me that I tried to record exposure settings in (I was usually too lazy); I wanted to be able to learn from experience by looking at the settings I used to take a particular image. Digital cameras have trivialised this process, as they embed the exposure settings within each photograph.

But digital metadata is able to store much more than just the camera settings. In particular, it is becomming increasingly common to store location information (collected automatically by GPS). This is not so much to remind the photographer where they took the photo (most of us are pretty good at remembering that with reasonable precision), but it allows entirely new methods of displaying and browsing sets of photos.

Over the last few months I have experimented with various data acquisition and image display techniques for geo-tagging. This is a growing field, and so I’m sure there’s a lot more for me to learn. However, here are my findings and a quick demonstration of the results.

Data Acquisition

Some newer cameras connect directly to GPS devices (or have them built in!) so that the geographic position information is saved directly to each image as it is recorded. I do not have this luxury.

Instead, I collect the GPS information separately. If I carry my little Garmin Geko 201 GPS receiver with me, I can know my geographic coordinates when I take a photo. Also, it can record my trail (as I’ve done on previously for certain trips) in a tracklog which can be uploaded to my computer.

Geotagging

The simplest case is when I know the geographic coordinates that a photo was taken from. I could write down the coordinates from the GPS, or simply take a photo of its display. To embed these coordinates in a photo once its on my computer, I have found the geoexif script from here to work well. For example, I run

./geoexif.py --latitude -35.6279 --longitude 150.3213 *.jpg

to tag all the jpg files in that directory with a certain latitude and longitude.

This works well if a lot of photos were taken from the same point, but there is an even easier way!

GPS tracklogs are simply a list of positions with a timestamp. Digital cameras also record (within each image) the time that a photo was taken. Conceptually, it is straightforward to find the time an image was taken, then search along the points in the GPS tracklog for that time and find the position information.

Many software tools exist to perform this interpolation task, but I have found gpsPhoto.pl from here to work really well. For a GPS tracklog on my computer in gpx format, I run

~/scripts/gpsPhoto.pl --gpsfile .gpx --overwrite-geotagged --timeoffset -39600 --maxtimediff 7200 --dir ./

to tag all jpg files in the current directory. This even embeds altitude data!

The “timeoffset” option gives my timezone (I’m in GMT+11 so the gps is 11 hours behind the jpg timestamps, converted to seconds). The “maxtimediff” here is 2 hours, and this option means that if I set up camp and turned my GPS receiver off but still took photos for 2 hours they would get approximately correct geotags.

Geo-display and Geo-browsing

Flickr, the popular photo sharing site, has a very good “Map display” for browsing geotagged images. Even better, with jpg files geotagged in the metadata as I’ve described, Flickr can automatically import the geo-information when you upload photos.

The really fun bit is browsing other people’s photos from similar locations to your own, to see what they noticed. Here is an example of exploring my Flickr Map – and you’ll notice that in Canberra Flickr uses OpenStreetMap!

The Flickr map is heaps of fun for browsing your own and other people’s photos geographically, but I wanted to find a good way to embed a map-browsable photo album here in my own website. Here is a demonstration of the idea from this article (I had to use gpsVisualiser to convert the gpx to a kmz file). The images and tracklog are from a daywalk up to Blue Lake that I did during the SNSW Big Camp at Jindabyne a few weeks ago.

(View Larger Map)

-----

Duplicating DVDs

Lachlan — Wed, 14 Jan 2009 01:32:48 +0000

Last week I had reason to duplicate some video DVDs. Yes, it was legitimate; I wanted copies of personal video footage to share with family. After fiddling around a bit I found a very straighforward way to do this task using simple GNU/Linux tools.

The first step is to copy the DVD to the hard drive (I only have one DVD drive).
cat /dev/sr0 > dvd.iso
where /dev/sr0 is my DVD device.

Then, after changing to a blank DVD, simply
growisofs -dvd-compat -Z /dev/dvd=dvd.iso
and it worked.

-----

Un-deleting files on my ext3 “backup” partition

Lachlan — Fri, 15 Aug 2008 04:31:19 +0000

Last week I learnt a very important lesson: a “backup” is not actually a backup if it is the only copy you have, it is at most an archive. In the process of tidying up the files on my external “backup” hard disk I deleted a few directories of photos from the beginning of this year. As I pressed the Enter key I was sure that I had a copy of those photos still on my laptop; but fractions of a second later I experienced a piercing wave of doubt. It was already too late.

After checking my laptop and finding that the doubt was justified, I remembered with relief that before heading over to Europe I had copied all my photos onto DVDs and left them in my office at uni (just in case something our house burnt down or something). I went to sleep mostly certain that I my accidentally deleted files were safe on discs at uni.

As you probably suspect, I did not have a copy of the photos on DVD. My DVD backups only went to the end of 2007, and I had deleted files from the first 2 months of 2008.

It is not a tragedy, as the main photos of consequence were from ASA Convention and I do have the best of my photos on the official DVD. However, it provided me with significant incentive to learn about data recovery on ext3 formatted partitions. I’ve included some of my discoveries below.

The first thing that I came across was not encouraging; the official ext3 FAQ says that it is impossible to un-delete files. However, a number of people have defied this statement and I quickly found a tool called ext3grep.

I have my external hard disk divided into 2 partitions and so I was able to use the empty half as a playground to store any recovered files. This turned out to be very useful, because running the command ext3grep /dev/sdb1 --restore-all --after=1218027000 did not “only process entries deleted on or after” Wed Aug 6 22:50:00 EST 2008, but seemed to “recover” every file on the partition. It took all night.

Happily, the full directory structure of my deleted “2008″ folder was recovered. Unfortunately, not all of the jpeg files were found, but it was nice to have some of them back again.

Looking a little bit further I found an article on Linux.com about a tool called ext3undel, which apparently uses Foremost and PhotoRec to do the work. This article also explained fairly clearly why data recovery on ext3 partitions is difficult. I also stumbled across a guide to using foremost, and so decided to skip straight to this command-line utility.

I installed Foremost and gave it a try, and the results seemed impressive.
52661: 388052408.jpg 1 MB 198682832896 Foremost version 1.5.3 by Jesse Kornblum, Kris Kendall, and Nick Mikus Audit File

Foremost started at Fri Aug 8 18:23:52 2008 Invocation: foremost -t jpeg -i /dev/sdb1 Output directory: /mnt/tmp/output Configuration file: /etc/foremost.conf ------------------------------------------------------------------ File: /dev/sdb1 Start: Fri Aug 8 18:23:52 2008 Length: 186 GB (200005876224 bytes)

Num Name (bs=512) Size File Offset Comment

0: 00690176.jpg 1 MB 353370112 1: 00692792.jpg 1 MB 354709504 2: 00695328.jpg 1 MB 356007936 . . . 52662: 388055280.jpg 1 MB 198684303360 52663: 388058248.jpg 1 MB 198685822976 52664: 388061320.jpg 536 KB 198687395840 Finish: Fri Aug 8 20:36:15 2008

52665 FILES EXTRACTED

jpg:= 52665 ------------------------------------------------------------------

Foremost finished at Fri Aug 8 20:36:16 2008

I’ve left out a lot of lines in the middle, but you can see the important details: about 2 and a quarter hours with a total of 52665 jpg files extracted (a total of 23 GB)! Sadly, almost all of these files were not complete recoveries of the images. The data corruption and partial recovery, however, did give rise to some very intriguing artistic effects, and I’ve included some samples here. I couldn’t help thinking it was rather analogous to accidental light-damage back in the film era of photography.

I also gave PhotoRec a go, and it can also be told to only look for jpg files. It found a lot of images, but most of them were merely thumbnails from what appeared to be web-caches hidden in my backed up directories.

So of the three I found ext3grep to give me the best results as far as un-deleting my jpg files, however it also took the longest and produced the most storage-consuming recovery output. Its a bit disappointing that I have not yet been able to recover all the images I foolishly deleted, but I’ve certainly learnt about the value of true backups – storing files in at least 2 different places.

-----