A great tutorial on OffScreen graphics rendering on http://blog.blprnt.com/ made me want to release some code today. I have been working on some offscreen graphics rendering to create masking effects for an upcoming performance. Processing offers PImage and PGraphics as objects that hold raw pixels or vector graphics. These are kept somewhere in memory and not drawn to the screen unless you put them there using image(). This is much faster than drawing to the screen in almost every instance.

Using the blend() command with the MULTIPLY option you can apply a masking effect to your sketch. For the example below the mask is a PGraphics object that contains a line which scrolls down the screen. Since the background is set to black (or 0) when it is multiplied against the image drawn on the screen it becomes black (because anything times 0 is 0). If this example is working properly you should see a red line move down the screen. This is because the image drawn to the screen is just a red background.

PGraphics g;
int x;

void setup()
{
  size( 720, 480);
  // create the mask
  g = createGraphics(width,height, P2D);
}  

void draw()
{
  background(244,90,10);
  // draw the mask
  g.beginDraw();
  g.background(0);
  g.stroke(255);
  g.line(0, x%height, g.width, x++%height);
  g.endDraw();

  // apply the mask to the screen
  blend(g,0,0, width,height, 0,0,width,height,MULTIPLY);

}

If you'd like to get fancy you can draw your image and mask offscreen and then send the result to the screen. Here is a little code snippet to get you started on your journey.

PImage img;
Pgraphics msk;
//.... do some drawing in both

// apply the mask to the image, both offscreen
img.blend(msk, 0,0,img.height, img.width, 0,0,img.width,img.height,MULTIPLY);
// draw the masked image to the screen
image(img,0,0,width,height);

Last one!

Today I played with spectral flux to change the size of particles across the screen. For each frame of audio a particle is moved from the middle screen. I was surprised how easy spectral flux was and how great it could look.

This link was invaluable.

Today I played a bit more with the peakiness function but I went back a simple particle system that will evolve over longer periods of time. I borrowed some of the code from yesterday and tried to make it a bit more gradual and I got some interesting results.

I also spent some time today digging into the Java API and found some nice features of Arrays. Sorting is actually quite a nice algorithm and also fill is a great way to just fill up an array with a value. Hooray for not having to write simple things over and over!

I am getting quite interested in developing some analysis tools for sound in Java. I am wanting to find the ratio of centre frequency to noise. Today I made a crude algorithm that looks for the peak frequency bin and then uses that in a ratio between spectral centroid. If the peak is off from the centroid then you can assume a distribution that is noisier (aka less pitched). The problems with this algorithm is that it is not normalized to volume and that a centroid of fftsize/2 is noise so if there is a peak at this place then you will get erroneous data. That will take me a little while to find a good solution around this but if was a fun thought experiment.

Today's patch uses my peakiness measure to draw big bubbles across the screen. More noisy means more crazy drawing. It is actually responding in generally the right way and is kind of interesting to watch. I will definitely be expanding on this idea.

Also today I left in some comments in the code for things that I tried and abandoned. You can see a bit more into my process with these.

Another day, another experiment. I tried to resurrect my Spectral Flatness code and I think it is a precision error in the geometric mean. I thought about scaling the FFT values but I will have to go back on the math for that to make sure I don't skew the values the wrong way. For now I have used the arithmetic mean of the spectrum to give me some numbers to play with.

This sketch changes colour at different volumes, currently white for quiet and black for loud, and becomes more animated with noisier spectra (not really but close enough for now). I think I am getting close to something interesting. Maybe once I sleep on it tomorrow will show me the answer.