Generating Music Out of a Film

This project has been realized for Clarence Barlow, a contemporary composer and computer music pioneer.

Clarence was asked by the Bonner Kunsthalle to compose a new music track for a 2 minute film from 1930 by Oskar Fischinger "Study No.6". The film was originally created (charcoal drawing on paper) for the existing piece "Los Verderones Fandango" by Jacinto Guerrero. The film consists of abstract objects moving and changing in shape. Clarence's idea was to measure the bounding boxes of these objects and use the measured parameters for the control of musical parameters, using his own algorithmic composition program AUTOBUSK. Imaging specialist Dr. Martin Bichsel provided his blob recognition software for measuring a single frame. We developed the software to iterate over the whole movie and apply the blob recognition to each frame. The measured data were sent to Clarence by email, where he started the music generation process. In the meantime we extended our PerEdit software by a special converter for the measurement data format. With this converter, it is possible to visually check the correctness of the blob recognition process and in case of error adapt the parameters provided by the blob recognition program. The work was presented at EX MACHINA 1996.


A frame taken from the original movie...


Same frame as in first image, but overlayed the detected bounding boxes.

Procedure

The Blob Recognition program takes a single frame in .pgm format. So the whole movie has to be converted into single .pgm-images. Due to limited disk space the movie had to be converted into .gif-images. For processing, each frame is converted to .pgm-format, then processed by the Blob Recognition program. The Blob Recognition program provides the following parameters: Our wrapper-program iterates over all frames of the movie and generates an ascii-file containing a description of each frame. This file was then sent to Clarence who used it to control his algorithmic composition program AUTOBUSK.

Reference

Some Links to Clarence Barlow

May 1997


© Bernhard Wagner 1997.