visual liszt: process documentation

As part of my research about visualization as a tool to engage the public into a discussion of literature, music, theatre and media, I created a generative animation for the 2011 fulldome festival in Jena.

Situated somewhere between data art, generative graphics and information visualization, my goal was to design a visual representation for classical music that could potentially work across media, such as film, print and objects. As part of this exploration I also made it my goal to design the visualization in a readable way, so that music experts and non-experts alike understand the potential of visualization for the experience and discussion of music.

The main part of the project is an immersive fulldome film in which the audience can experience the creation of a visualisation of Franz Liszt’s »Liebestraum Nr. 3« in combination with the music. This film is based on a computationally created object printed in full-colored sandstone with Shapeways that could work in an exhibtion scenario. Ideally, this exhibition scenario would include several objects made from various compositions and composers for comparison and discussion.

For the visualization itself, I chose a fairly straight forward principle of representation by showing every played note by a sphere, which is slightly varied in size according to it’s attack velocity. The resulting object maps the entire note range from left (low pitch) to right (high pitch), with a special emphasis (larger spacing) on the center notes. The height of the object represents the time as seen from top to bottom. This rather simple visualization principle was chosen to ensure that large parts of the audience in the fulldome can understand the process and idea.

The basis for this computational visualization is the data found inside MIDI-files from the web, which I analyzed and translated to a custom XML file including the exact timing data using Python and music21. This tool will hopefully be a starting point for further investigation into music (information) visualization.

The overall project and tools were mainly realized with Processing, toxiclibs, GLGraphics. The fulldome film was rendered with Sunflow – an open-source global illumination renderer written in Java – on Amazon EC2 with the help of Karsten Schmidt’s incredibly useful tutorial.

Thanks as well to Holger Patz and Prof. Klaus Dufke of the design department at the University of Applied Sciences Potsdam for providing me with the dome mock-up scene after filming in a 360° projection environment did not succeed.

built with