Weekened Project: irFlip2
A Full Spectrum Image False Coloring Tool
I took a trip for a few days, visited a couple state parks near Fort Smith, Arkansas, and ended up with some interesting pictures. I recently finished setting up a full spectrum converted camera, and wanted to explore some postprocessing techniques that I had read about. Full spectrum cameras are created by removing a "hot mirror" filter which is typically placed in front of the silicon sensor - silicon doesn't distinguish between visible light and the rest of the EM spectrum the same way the human eye does. When photons hit the material, it creates electrical signals. The material has a wider band of sensitivity and requires this filter in order to cut out the UV and IR components of the incoming light.
These images can look a little strange, since the eye only observes visible light. In reading about some of what people have done with this type of camera, I came across references to channel swapping. This is an established technique for working with these types of images, and typically this is done with the use of some software like Photoshop, using color curves. These curve type tools provide some nonlinearity that I don't really have a solution for, but that might be a future direction to explore. I wrote a couple of command line utilities for this operation which I will describe below.
irFlip - 10 Fixed Combinations
This was a quick implementation that I did in about an hour, the first evening I was in Fort Smith. Using LodePNG, it loads an image in RGBA format and constructs several output images, assuming a 255 alpha value for every pixel. The output is a set of fixed combinations - RRR, GGG, and BBB grayscale, as well as RGG, RBB, RBG, BRG, BGR, GRB, and GBR.
Given the above input image, this produces following 10 outputs. Alt text and filename tell which is which. Each of the images link to the downsampled original, click to view directly - the originals are 6000x4000, 10-15 meg jpgs, which end up being impractical for hosting here.
The RBB ( second row, middle ) is my personal favorite, I think, for this input image. Seems like it maps the input colors to a familiar range of tones, more like what your eye expects, but at the same time the result looks totally alien. I've seen several infrared captures taken by other people colored this way, and I think it's interesting to be able to derive a tool which is able to produce the effect.
irFlip2 - Generalized Tool Using Swizzle Syntax
While I was out Saturday morning, I realized that this is relatively simple to generalize into a more general purpose tool. I wanted to be able to explore more combinations of the input image channel data, without having to rewrite and hardcode those combinations. In addition, I thought it might be interesting to remove the assumption of a 255 alpha value, and allow for value inversion of any of the input channel data. This was achieved with a very simple swizzle syntax, which should be relatively familiar if you've done any shader programming. In most shading languages, there are utilities to rearrange the values kept in a vector type, e.g. setting a three element vector value = value.bgr, where RGB takes BGR, swapping the red and blue channels.
I got rained out around noontime, so I decided to come back to the hotel I was staying at and get this implemented. It took about 3 hours to rewrite everything for irFlip2, on and off between working on this and taking some pictures in the room. The call to the program is structured thusly:
./irFlip2 input.png RbGA
This specifies the input image as the first argument, and the swizzle as the second argument. The swizzle is slightly extended from the idea of a swizzle described above, in that it can draw from the set { R, r, G, g, B, b, A, a, 0, 1 }, where uppercase elements indicate the unmanipulated value taken directly from the input, and the lower case elements indicate the color channel, value inverted by taking 255 minus the input value. 0 and 1 are special values to saturate the value in the output to either 0 or 255, respectively, fully ignoring the input for that swizzle element. In the call above, the swizzle RbGA is used - this means that the red channel is used directly, the green channel takes the inverted blue channel, the blue channel takes the input green value, and the alpha carries through untouched.
For the pictures taken with the camera, you have a constant 255 alpha channel, which can be used to emulate the 0/1 behavior, but it's a nice-to-have when you are dealing with more general images. I have added both of those options, for input images which might not have a uniform value for any given channel.
Given the above input image, the following output images are possible. Picking 4 output channels from a set of 10 choices, with repeats allowed, the combinatorics work out to a total of 10000 possible outputs for any given input image. Quite a few of these are degenerate cases, for example anything that saturates the alpha channel to 0. Again alt text and filename to identify which is is which, images link to downsampled originals. A lot of the possible swizzles are very ugly, but it is a superset of the outputs of irFlip and there are certainly some interesting ones. It's also interesting to see how much more pronounced the dirt on the screen is, in some of the different channels.
This same operation is possible to do with ImageMagick ( I used convert to downsample these ), albeit with slightly less friendly syntax. As a small utility for a very specific purpose, I think it may have quite a few applications. Source for both versions is available at the links above, along with makefiles if you have an application for it.
Edit: 11/29/2022 - I went ahead and added this to my image wrapper, since it's a useful general utility to have around. It solves a fairly wide class of problems, with a very small amount of code. The addition of special values 0 and 1 since my initial writeup here, has proved very positive, and extends the capabilities a bit beyond what's possible with shader-style swizzling.