Robot swarms are typically associated with industrial automation, environmental monitoring, or search operations, but a recent project at the University of Waterloo is showing how the same engineering principles can be applied in a very different context. Led by Dr. Gennaro Notomista, a professor of electrical and computer engineering, researchers have developed a system that allows groups of robots to interpret music and convert it into coordinated movement and light patterns. The work demonstrates how advances in swarm robotics, signal processing, and human–robot interaction can be integrated into a single platform while also highlighting broader engineering implications beyond artistic applications.
Yang, H., Li, H., Zhang, J., Liu, T., Qi, H. J., & Sun, H. (2025). Halftone-encoded 4D printing of stimulus-reconfigurable binary domains for cephalopod-inspired synthetic smart skins. Nature Communications, 16(1), 9931. https://doi.org/10.1038/s41467-025-65378-8
The system consists of multiple wheeled robots roughly the size of soccer balls, each equipped with programmable lighting and motion control. Operating within a defined floor space, the robots respond to real-time analysis of musical features such as tempo, rhythm, and harmonic structure. As they move, they generate colored light trails that are recorded by an overhead camera, forming a visual representation of the music. Rather than following pre-programmed paths, the robots continuously adjust their behavior in response to changing audio inputs, which requires a tight coupling between sensing, processing, and distributed control.
Dr. Gennaro Notomista from University of Waterloo stated,
“Basically, we programmed a swarm of robots to paint based on musical input. The result is a cohesive system that not only processes musical input, but also coordinates multiple painting robots to create adaptive, expressive art that reflects the emotional essence of the music being played.”
A central engineering challenge in the project involved converting audio signals into coordinated swarm behavior. The research team designed algorithms capable of extracting meaningful musical features and translating them into motion parameters such as speed, direction, spacing, and light intensity. Each robot receives global information derived from the shared music analysis while simultaneously managing local constraints, including collision avoidance and spatial positioning. This balance between centralized data processing and decentralized decision-making is a defining characteristic of swarm robotics systems.
The platform was tested using groups of up to twelve robots, although the control architecture was designed with scalability in mind. Maintaining stable and coordinated behavior as swarm size increases is a persistent engineering challenge, and the researchers focused on ensuring that communication and control mechanisms could support larger deployments without requiring significant redesign. Achieving this scalability required careful attention to distributed algorithms and real-time system responsiveness.
Human interaction was also incorporated as a key component of the system. Users can influence the ongoing visual output by adjusting parameters such as the thickness, location, and intensity of light trails. Rather than directly controlling individual robots, these inputs modify higher-level system variables that guide overall swarm behavior. This layered control structure allows the robots to retain autonomy while enabling collaborative interaction, reflecting broader trends in human–robot interface design.
Although the project is presented in an artistic context, its technical contributions align with ongoing research in coordinated multi-robot systems. The same engineering principles underlying the music visualization platform are relevant to applications such as environmental data collection, agricultural automation, disaster response operations, and planetary exploration. In each of these scenarios, robots must interpret shared data inputs and act collectively while adapting to local conditions.
The research also highlights the potential for robotic swarms to serve as physical visualization tools. By translating complex data streams into coordinated motion patterns, such systems could provide intuitive ways to represent information in fields ranging from environmental science to network monitoring. This capability suggests that swarm robotics may play a role not only in performing tasks but also in communicating system states and data insights.
The work was presented at an international robotics conference focused on the social impacts of advanced robotic systems, reflecting its interdisciplinary nature. Future development will include user studies involving artists and musicians to better understand how people interact with autonomous robotic groups and how intuitive control mechanisms can be refined.
While the visual output of the system may appear to belong primarily in a gallery setting, its engineering significance lies in advancing methods for real-time coordination among autonomous agents. By combining signal processing, distributed control, and collaborative interfaces, the project demonstrates how swarm robotics research continues to expand into new domains while addressing core technical challenges that remain central to the field.
Adrian graduated with a Masters Degree (1st Class Honours) in Chemical Engineering from Chester University along with Harris. His master’s research aimed to develop a standardadised clean water oxygenation transfer procedure to test bubble diffusers that are currently used in the wastewater industry commercial market. He has also undergone placments in both US and China primarely focused within the R&D department and is an associate member of the Institute of Chemical Engineers (IChemE).
