Resleton (v0.8)
One of the major challenges I have faced in integrating interactive audio technology with video, lighting and other tools is getting everything to play nice without creating an incomprehensible mass of MIDI mappings. At Juilliard’s Center for Innovation in the Arts, we frequently have multiple computers running video playback (and interactivity) in Resolume alongside other machines running a combination of Ableton, Qlab, Max/MSP and other systems as needed. Ableton is often the central hub of audio, which means events (both audible and structural) often need to be communicated to the rest of the network. Of course, I could map all sorts of commands to MIDI, but the complexity and resolution required often far outstrips both the protocol as well as any operator’s ability to remember what is mapped where.
To that end, I developed Resleton, a suite of Max for Live devices that allow Ableton to control a variety of functions in Resolume, via OSC over a network. This suite has several functions that set it apart from other systems that can allow similar functionality. Crucially, I’ve configured the devices to be able to direct commands to four different Resolume machines. (If you are a user and would like more, let me know, but so far nobody has asked for that). Most other tools assume that one will use a single computer to run Resolume, but with immerse multi-screen setups with heavy visual signal processing, oftentimes it’s impossible to use just one. Resleton provides four OSC “buses” that are routed to IP addresses set at a central hub.
Resleton includes the following devices:
RESOLUME DEVICES:
Miscellaneous Layer Control: For sending OSC to a layer in Resolume. This devices allows you to enter an arbitrary OSC string and and use Ableton’s automation to automate the layer number in Resolume that is targeted, as well as the OSC argument.
Miscellaneous Clip Control: For sending OSC to a clip in Resolume. This devices allows you to enter an arbitrary OSC string and and use Ableton’s automation to automate the layer number and clip number in Resolume that is targeted, as well as the OSC argument.
MIDI-> Column Trigger: This MIDI effect ingests MIDI notes and converts them to column triggers. This allows for the easy triggering of Resolume columns using either live input or preprogrammed midi notes.
MIDI-> Clip Trigger: This MIDI effect ingests MIDI notes and converts them to clip triggers. It also allows a user to use MIDI notes to target different layers. This allows for the easy triggering of Resolume clips using either live input or preprogrammed midi notes.
Clip Trigger: This device allows one to use Ableton automation to trigger individual clips on a particular layer. It has a configurable delay time for synchronizing audio and video.
Column Trigger: This device allows one to use Ableton automation to trigger columns in Resolume. It has a configurable delay time for synchronizing audio and video.
Deck Switcher: This device allows one to use Ableton automation to switch decks in Resolume.
Volume->Layer Dashboard: This device measures signal amplitude and maps that value to dashboard controls. Includes smoothing and scaling. Since Resolume only has one FFT input, it’s impossible to natively process many different parameters with amplitude. This solves that problem, all while allowing one to condition the signal using Ableton’s native processes.
Pitch-> Layer Dashboard: This device measures pitch using Max/MSP’s fzero object and maps that frequency to dashboard controls. Includes smoothing and scaling. This device only works for monophonic input.
Resolume Dispatch: This devices routes OSC from one of the four busses to a particular IP address and port. It can be defeated or easily rerouted for testing.
STANDALONE/MISC DEVICES:
Standalone MIDI Note->OSC: converts the MIDI notes of your choosing to an arbitrary OSC string. This device bypasses the central hub device and allows you to enter in a single IP address and Port. Useful for testing, or connecting to one-off devices.
Standalone OSC Automation: This devices allows you to use Ableton’s automation to send OSC arguments with an arbitrary OSC string. This device bypasses the central hub device and allows you to enter in a single IP address and Port. Useful for testing, or connecting to one-off devices.
An example of the GUI.
In use at Juilliard’s Peter Jay Sharp Theater for the Vox Celli program in 2023. Video design by Willie Fastenow. Ableton and Resolume programming by Nathan Prillaman and Collin Chudwick. The Ableton/Resolume connection allowed for a lighting-fast load in and load out, as well as mid-rehearsal reprogramming of a level of complexity that would have been very challenging with other workflows. In this program, Ableton generated no sound except a click track. Ableton did however measure incoming amplitude and apply it to FX in Resolume, as well as provide a GUI for programming a variety of parameter changes and triggers in Resolume, alongside a host for real-time mid-show interaction. This image is from backstage, where our video machines were located. The projection was a three-projector blend run off a single MacBook Pro.
An image of the projection design from the audience perspective.