below is the demo fft-paint applet from marc
Instructions for use The text-field at the top of the applet takes a URL of an image (GIF or JPEG) - click on 'go' to load. The top-most greyscale is the palette (the box to the right indicates the current pen shade). Below is the scaling tool which controls how the image data is mapped onto grey (or colour) pixels. The large button marked 'transform' preforms a fourier transform on the image. The three choice boxes control the three stages in the brush pipeline.

Images that are stored locally are :

  1. mondrian1.jpg (Blue Crysanthemum)
  2. mondrian2.jpg (Composition in Blue)
  3. klee2.jpg (Ancient Sound, Abstract on Black)

Scaling tool Not only does the FFT algorithm often result in complex 'images' with magnitudes outside the 0 to 255 range of the input images but it is very easy to edit such a spectrum to prevent it from back transforming into that original range. Indiviualy the real and imaginary parts of the image have little meaning. This tool lets the user set the black (left mouse button) and white (right mouse button) points of the mapping of the complex image back onto the displayable range. Other options include : 'mag' to use magnitude rather than real component (recommended), 'phase' to display the phase information as hue (red = 0 phase, saturation is fixed at 20%) and 'auto' to rescale the size of the scaling tool to match the range in the image. The phase option makes appears, in practice, to make intuitive sense rather than just being another piece of maths. Even if its presentation is arbitrary, presenting it at all means that the data is there for further processing by photoshop.

The paint tools are adjusted automatically to paint with the actual 'colour' specified by the palette not a value between 0 and 255, and usually work with the magnitudes of pixels rather than altering the phases.

The three stage brush pipeline What actually happens when a mouse movement is made is decided by the brush pipeline. Mouse movements enter to the left and 'emerge' at the right onto the palette. In practice this means that the leftmost choice provides the operation (think: photoshop's 'normal','add' etc. modes); the centre, the brush style and shape and the rightmost give some large scale remapping of that brush (such as mirroring it). The last two stages are homogeneous and new 'mappings' can be added to the code in a modular fashion.

Particular to these complex images are the 'rotate +' and 'rotate -' operations. These rotate the phase of a complex pixel while leaving its magnitude intact.

Further work This applet just begins to open up the issues associated with trying to work artistically in alternate spaces. Why are we going to such lengths to put such distance between the user and the image? Although the space between exact control and random effect is creatively rich (e.g. improvisation, repetition of mistakes), the frequency domain is arguably too far away from the spacial domain to be of every day use to the artist. Additional tools for FFT space therefore should concentrate (like many computer tools) on the gentle adjustment of existing images -- e.g. filtering and convolution are particularily easy with fft-paint, creation from stratch is much harder. An exciting possibility is to replace the FFT transform with a wavelet tranform (I might yet do this). The wavelet basis set consists of spacially localised functions, and are a good compromise between the pure frequecy domain painting shown here and traditional editing. Other options may include avoiding the complex numbers by using real transforms like the discrete cosine transform.

Keeping with the Fourier transform we might consider warping the space that it acts in and gets display in --- the user is more interested in the low frequency information close to the 'center' of the transform than in the high frequency information far away --- in particular, the 'd.c.' component of the image is particularily important. Other coordinate systems my also be appropriate. We might also consider pre and post windowing the FFT code so the transform is not dominated by the edge of the images. Finally, the phase information in the transfrom is an exciting horizon in itself, offering visually interesting ways of subverting the image. Some animation work with the phase information may prove interesting.

Minor issues: I broke the 'log' and 'zero' options last night, and the save option never worked (important, I envisage this being used in addition with photoshop). More tools are required and the brush pipeline is made from open ended objects. Maximum size is 1024x512 (hence the strange layout) which is going to consume a lot of memory. Credits to Numerical Recipes for the fft code. My double-buffering trick doesn't under IE4.

The source For the (rather hard to understand and somewhat rushed) source, look here. The main applet (and a good place to start) is called . Questions to marcd.