¡Otro!

next generation Sound design for film, video + live performances

Introducing... ¡Otro!, custom-made software designed exclusively by Drew Trujillo to revolutionize sound design for live performances and videos. ¡Otro! captures and sonifies intricate performer movements and locations, seamlessly blending technology and artistry. It empowers directors, choreographers, cinematographers, visual designers, and sound designers to push creative boundaries. Crafted specifically for Drew’s collaborative projects, ¡Otro! is not available for public use, allowing Drew to work closely with top professionals in the arts and media industries. This bespoke tool exemplifies innovation and collaboration at its finest.

¡Otro! creates an immersive environment in live performances by seamlessly integrating sound and visual effects that respond dynamically to performers' movements, surrounding the audience with a rich sensory experience. This engagement transforms the performance space into an interactive canvas where every action influences the auditory and visual output, creating a deeply engaging experience for the audience.

For video and film, ¡Otro! enhances the complexity and richness of sound design by leveraging motion and location-based data. This advanced integration ensures that every nuance of performance is captured and translated into a detailed auditory landscape, elevating the overall impact of the production. For online videos, ¡Otro! preserves the immersive experience by utilizing spatialized audio techniques, providing viewers with a three-dimensional sound environment that simulates natural hearing. This approach not only enhances the narrative and emotional impact but also makes viewers feel as though they are part of the performance, even when watching remotely.

Key features

¡Otro! offers a suite of features tailored to the next generation of video and sound editing, as well as advanced sound design. Specifically crafted to meet these unique needs, ¡Otro! ensures seamless collaboration with directors, choreographers, cinematographers, and sound and visual designers.

Drew Trujillo, Bosque in Albuquerque, NM, 2022

Maewyn, Tiguex Park in Albuquerque, NM, 2023

  • Utilizes 9-axis sensors (acceleration, gyroscope, magnetometer) to precisely capture and translate movements into both sound and visual effects (not shown in this video), creating a dynamic interplay between physical motion, auditory, and visual experiences.

  • Converts actions into real-time soundscapes and visual effects, allowing users to generate, modulate, and interact with music and visuals through their movements.

  • The custom editor allows us to record performance data and edit it later, providing keyframe precise control over both sound and visual elements. A standout feature is the ability to scrub and loop through frames and in and out points of the video and audio, with the movement and location data output to a DAW for real-time sound design. Fine-tune every aspect of your performance to achieve the perfect result.

  • ¡Otro! integrates state-of-the-art waterproof motion sensors to enhance its capabilities in dynamic environments. These sensors feature a 9-axis motion measurement system, combining acceleration, gyroscope, and magnetometer data to deliver precise and comprehensive movement tracking. The robust, water-resistant design ensures reliability and accuracy even in challenging conditions, making it ideal for capturing complex performer movements and translating them into rich, detailed soundscapes. This integration allows ¡Otro! to push the boundaries of creative expression by enabling real-time, responsive audio-visual experiences.

  • In this video, the acceleration and rotation of the wrists are sonified.

  • Excited to share this video featuring the incredibly talented multi-disciplined artist Maewyn! Watch as she creates a stunning chalk drawing while wearing motion sensors on her wrists. Using ¡Otro!, I captured the motion data on-site. In post-production, I utilized a custom video, audio, and data editor that I developed to generate and modulate the audio effects based on Maewyn’s wrist movements.

    Each wrist is mapped to a different type of sound, creating a unique auditory experience that complements her movement and visual art.

  • Utilizes machine learning to enhance the expressiveness and accuracy of the performance, learning from previous data to improve future outputs and create more nuanced and responsive sound and visual effects.

  • Utilizes advanced sensors to track the distance of a performer(s) within the performance space to each other or a specific location, enabling the creation of responsive sonic and visual environments that enhance narrative and artistic expression through spatial relationships.

  • The video Phase Shifting is inspired by minimalists like Steve Reich and Terry Riley, and attempts to capture the societal dissonance from persistent inequalities, where the sound generated by each performer is out of phase when they are different distances from the column. The distance is measured using location-based sensors.

Elyse Fahey and Madrone Matysiak, Trzy Siostry (Three Sisters), 2024

  • Graphically defines specific zones within the performance space that activate or modulate unique sound and visual effects when entered or exited by performers. This feature tracks the proximity and distance between dancers and these Zone Areas, allowing for dynamic interactions and enhanced spatial storytelling.

  • Tracks the proximity and distance between multiple performers, enabling real-time interactions and synchronized effects that reflect their spatial relationships. This capability enhances the collaborative and dynamic nature of the performance.

  • The video is about being in the heart of an enchanted forest, “Trzy Siostry (Three Sisters)” captures the essence of folklore and forgotten voices, crafting a performance of joy, loss, and the enduring spirit of unity.

    As the two sisters dance, they bring to life layers of sound through advanced sensor technology. ¡Otro!, the sonification system developed by Drew Trujillo, uses sensors to create a sonified landscape based on their location, movements, and interactions, reflecting the emotional depth of the tale and a critique of displacement, bridging folklore with historical reality through innovative acoustemology.

Uncle Al, 2023

  • This video performance features a macabre clown, Uncle Al, where facial-tracking technology monitors over 100 data points to synchronize movements with sound. As the clown closes his eyes, sounds of children's laughter and chimes fill the air, while an ominous tone emerges when his mouth opens. This innovative use of technology illustrates a dynamic interplay between facial expressions, movement, and sound, highlighting facial-tracking's potential to enhance expressive dimensions in performance art.

    Within the showcased video, a noteworthy correlation between the character’s actions and auditory responses is demonstrated. For instance, upon the character closing their eyes, the auditory landscape is imbued with sounds associated with escapism—children’s laughter, and playful chimes, among others. Conversely, the character’s mouth opening prompts the emission of a sonically distinct, ominous composition.

    This exploration encapsulates the nuanced interplay between facial expressions, movement, and audio elements, shedding light on the potential of machine learning in enhancing the expressive dimensions of performative arts.

  • In this 2022 project, I delved into the artistic potential of facial expressions by transforming them into sound. Using an iPhone app, I tracked my facial movements and linked them to different sounds. Collaborating with Alanna Herrera, who portrayed a scary clown, we recorded her expressive performance and translated her facial movements into a rich audio experience. For example, when her eyes closed, the sounds of a playground played, and when her mouth opened, intense sounds were triggered. This approach allowed us to create a deeply immersive experience, enhancing the emotional impact of the performance with three-dimensional audio techniques. This project showcases how facial expressions can drive creative storytelling through sound, perfect for sound designers, choreographers, and directors of photography looking to push the boundaries of their craft.

Drew Trujillo, The Sonification of..., 2024

Combinatorics and Sensor Fusion: Millions of Unique Creative Expressions

Sensor Fusion: ¡Otro! harnesses the power of combinatorics to fuse together different sensors and incoming data. By combining their data points in a multitude of ways, it creates a new “instrument” with each unique combination. The diagram to the left illustrates various input categories and parameters, showcasing different combinations and their connections to various output sound parameters.

What happens when the motion from each hand connects with invisible, flexible cords that generate and modulate soundscapes?

In this video experiment, the body becomes the instrument. The invisible cords connected to the hands are the strings of this instrument, and the sounds are like wet clay, sculpted by the hands in real time. The material is the custom code I wrote, acting as the connective tissue between the limbs, demonstrating some of the innovative features of ¡Otro!.

Through ¡Otro!, Drew Trujillo engages in dynamic collaborations with top-tier professionals in various creative fields. This software is a personal and exclusive asset, designed to enhance the collaborative process and elevate the creative output.

collaboration

  • ¡Otro! provides directors with precise control over the auditory elements of a performance or film, allowing them to craft a cohesive narrative that seamlessly integrates sound with visual storytelling. The software's advanced sound design capabilities enable directors to experiment with new auditory dimensions, enhancing the overall impact of their projects.

  • For choreographers, ¡Otro! translates intricate movements into corresponding soundscapes, creating a symbiotic relationship between dance and music. This feature allows choreographers to explore new realms of creativity, synchronizing movement with sound in innovative ways that enhance the emotional and sensory experience of the audience.

  • ¡Otro! empowers cinematographers by providing tools to spatialize audio based on camera movements and angles, ensuring that the sound design complements the visual composition. This integration results in a more immersive and engaging viewing experience, as the auditory elements dynamically interact with the visual narrative.

  • ¡Otro! empowers Directors of Photography by providing tools to spatialize audio based on camera movements and angles, ensuring that the sound design complements the visual composition. This integration allows DPs to enhance the narrative depth and emotional resonance of their work, as the auditory elements dynamically interact with the visual storytelling. The software aids in creating a cohesive audiovisual experience where lighting, camera angles, and movement are intricately linked with the soundscape, resulting in a more immersive and engaging viewing experience.

  • With its advanced capabilities, ¡Otro! offers sound designers a platform to craft intricate and layered soundscapes that are responsive to the performers' actions and locations. This flexibility allows sound designers to push the boundaries of traditional sound design, creating unique auditory experiences that elevate the overall production quality.

  • ¡Otro! enhances the role of Creative and Art Directors by providing a platform that seamlessly integrates visual aesthetics with innovative sound design. This allows for the creation of unique, multi-sensory experiences that captivate and engage audiences. The software enables Creative and Art Directors to experiment with new artistic expressions, blending visual elements with dynamic soundscapes to push the boundaries of traditional media and craft truly immersive environments.

  • Visual designers benefit from ¡Otro!'s ability to synchronize visual effects with sound, creating a cohesive multimedia experience. The software enables visual designers to explore the interplay between visual and auditory elements, resulting in innovative and immersive environments that captivate audiences.

Integration with existing systems

  • Integrates seamlessly with various open-source microcomputers, OSC, and MIDI devices, enhancing creative possibilities by parsing, filtering, analyzing, remixing, and outputting data to different platforms.

  • Works seamlessly with VST3 plugins and all major Digital Audio Workstations (DAWs), allowing for enhanced creative workflows and easy integration into existing setups.

  • Supports DMX lighting control and integration with various DMX devices such as fog machines, lasers, and moving lights. This enables synchronization with sound and visual elements to create a fully immersive performance environment.

  • Enables high-quality video transmission over IP networks, allowing for seamless video sharing between ¡Otro! and other NDI-compatible devices and software.

  • Provides video sharing capabilities between applications on macOS, facilitating the integration of ¡Otro! with other Syphon-compatible software for enhanced multimedia experiences.

  • Enables seamless interaction with Processing, openFrameworks, TouchDesigner, and Unity, allowing for the creation of interactive visuals, as well as robust option for creating immersive virtual reality (VR) and augmented reality (AR) experiences.

  • Enables integration with various hardware devices, such as motion capture suits and VR equipment, enhancing interactive performance capabilities.

  • Trujillo's workflow integrates smoothly into existing workflows, enabling creatives to use their preferred software and tools without disruption.

benefits

  • Explore new realms of sound and visual design by merging physical movement with auditory and visual experiences, allowing for unprecedented artistic expression.

  • Empowers users to become both the music and the visuals, challenging traditional boundaries and celebrating the performer as a multidimensional artist.

  • Creates a multisensory narrative that captivates audiences, ensuring every gesture and step is not only seen and heard but felt.

  • Ensures precise alignment between performers' movements and audio-visual effects, enhancing the overall coherence and impact of the performance.

  • Captures extensive motion and location data, allowing for detailed post-production editing to fine-tune performances.

  • Enables the creation of unique, customized soundscapes that respond dynamically to performers’ actions.

  • Utilizes binaural and ambisonic audio techniques to create an immersive, three-dimensional sound experience.

  • Facilitates interactive, real-time manipulation of audio and visuals, providing a dynamic and engaging performance environment.

  • Inspired by the idea of turning performers' movements into sound, Trujillo's practice transforms the motion and location of various performers into rich, expressive audio. This approach enhances the emotional impact of sound in creative works. Beyond live performances, the latest advancements allow for recording motion, video, and audio data during shoots. Using a custom editor, this data is integrated in post-production to create intricate and resonant sound designs.

    To achieve this, Trujillo developed ¡Otro!, a custom application that uses sensors worn by performers to capture their movements and positions. In post-production, we use this data to craft soundscapes that align with the performers' actions, adding depth and emotion to video and film projects. By recording and sonifying these performances, we merge art, culture, and technology, creating a rich, multi-sensory experience.

  • Elyse is a dancer, choreographer, producer, studio owner, dance teacher, and movement designer. She has performed locally and internationally for the last 15 years, with professional companies such as J. Garcia Dance Company and Lawine Torrèn, and with choreographers such as Donna Jewell and Ja’Moon Jones. Elyse’s choreographic work has appeared in contemporary dance festivals and showcases in New Mexico, Texas, California, and Germany. In 2022, Elyse received a Resiliency Residency grant through the City of Albuquerque’s Urban Enhancement Trust Fund, and she was a 2023 Keshet Makers Space Experience artist-in-residence. Elyse is the executive director of Studio Sway, a community hub for independent dance artists in Albuquerque.


    As a dance artist, Elyse values versatility and technical proficiency, which she pursues through rigorous class taking, improvisational movement research, rehearsing, dance making, and performing. Her work in choreography and movement design expresses its thematic content through the many stylistic spokes on her wheel of dance training and movement influences, centering musicality, world-building, and rigor.

LETS WORK TOGETHER.

drew@drwoohoo.com