Tiny Box Bot Tim: Miscommunication and Machine Perception in Interactive Systems

Exploring perception, failure, and human-machine interaction through a multi-system interactive installation

Laila Mariam Imran

Abstract

This paper explores a robot named Tiny Box Bot Tim and an interactive system that attempts to communicate with humans, but consistently fails because of differences in perception and interpretation. The project is primarily built in TouchDesigner, combining real-time visual processing, glitch-based distortion, and a misidentification system that assigns incorrect labels to objects.

Alongside this, an Adafruit Circuit Playground Express (CPX) provides reactive light and sound responses. Rather than correcting miscommunication, the system embraces it across visual, textual, and audio outputs. The project frames machine perception as different rather than simply incorrect, showing how humans often try to assign meaning to systems that do not understand the world in the same way we do.

Keywords

machine perception, human-computer interaction, interactive systems, media art, miscommunication, generative visuals, TouchDesigner

Introduction

As computational systems become more common in everyday life, interactions with machines are often designed to feel seamless and intuitive. These systems are usually judged by how accurately they interpret and respond to human input. However, this assumes that machine perception should work the same way human perception does.

Tiny Box Bot Tim challenges this idea by presenting a system that tries to communicate, but constantly gets things wrong. The project combines real-time visual processing, misidentification, and reactive outputs to create interactions that feel meaningful, but never fully resolve into something clear or stable.

The work builds on earlier versions of Robot Lens, which explored how machines interpret visual information through abstraction and distortion. Tiny Box Bot Tim expands this into a larger interaction where vision, language, and response operate separately and often contradict one another.

Related Work and Context

This project connects to interactive robotics and media art that explore emotional projection, failure, and human-machine relationships. Norman White’s The Helpless Robot presents a system that appears dependent on human help, encouraging users to project emotion and intention onto it.

Other robotic installations, such as Sisyphus, explore repetition, frustration, and unresolved interaction. Patrick Tresset’s computational drawing systems also show how machines can reinterpret human subjects instead of simply reproducing them accurately.

The project is also influenced by Jakob von Uexküll’s idea of Umwelt, which suggests that every organism experiences the world through its own perceptual system. From this perspective, difference in perception is not always an error. It may simply be part of how that system experiences the world.

James Bridle’s writing on non-human intelligence also supports this idea by questioning the assumption that intelligence has to resemble human thought. Tiny Box Bot Tim uses this framework to treat miscommunication not as a failure, but as part of interacting with a system that perceives differently.

Project Description

Tiny Box Bot Tim is an interactive system made from multiple connected parts. Its main functionality is built in TouchDesigner, combining real-time camera input, glitch-based visual processing, misidentification, and reactive hardware outputs. Together, these elements simulate a robot trying to perceive and respond to the world.

The first component uses an integrated camera to capture live interaction with the user. This feed is processed through a TouchDesigner glitch system, which distorts the image through fragmentation, colour shifts, and visual noise. The output is shown back to the user as an unstable and abstracted version of themselves.

The second component uses an external webcam as the robot’s main “eye.” Users can present objects to the camera, which are then processed through a misidentification system. Instead of correctly identifying the objects, the system assigns incorrect labels, such as identifying a pineapple as “a pen.” This adds a layer of linguistic miscommunication alongside the distorted visual feed.

The third component is the Adafruit Circuit Playground Express, or CPX. The CPX works as a reactive layer rather than the core system. It responds to environmental sound using its onboard microphone, triggering LED colour changes and simple audio responses. These reactions suggest emotion, but are not clearly mapped to specific meanings.

These systems run at the same time, but they do not fully align with one another. The visual output, incorrect labels, and audio/light responses each create different interpretations of the same interaction. This lack of coherence is intentional. The project uses miscommunication across visual, textual, and auditory outputs as the main experience.

Discussion

The multi-system structure of Tiny Box Bot Tim challenges the expectation that machine perception should produce one clear and accurate interpretation of the world. Instead, the project presents perception as fragmented and distributed across different processes.

Each part of the system can be understood as operating through its own logic. The glitch system interprets visual data through distortion, the misidentification system assigns incorrect textual meaning, and the CPX reacts to sound without understanding context. These systems do not come together into one unified understanding.

This supports the idea that machine perception is not necessarily broken just because it does not match human perception. The robot’s incorrect labels and confusing responses become part of its own way of processing the world.

The project also shows how users respond to ambiguity. Even when the outputs are incorrect or inconsistent, people still try to interpret the system as emotional, intentional, or meaningful. This reveals how easily humans project meaning onto systems that do not share our way of thinking.

Reflection

Tiny Box Bot Tim successfully created an interactive system that shows how users interpret and respond to machine behaviour across visual, textual, and auditory outputs. The combination of glitch visuals, incorrect labels, and reactive responses produced moments of humour, confusion, and engagement.

The project also revealed technical and conceptual challenges, especially when trying to coordinate multiple systems that operate independently. While full cohesion between components was difficult, the lack of consistency actually strengthened the concept.

Some planned features, such as user feedback buttons and more structured emotional responses, were not fully included in this version. Future versions could return to these ideas and allow user input to influence the system over time, while still keeping miscommunication at the centre of the experience.

Overall, the project shows that interaction does not need to be clear or correct to be meaningful. By embracing inconsistency and misunderstanding, Tiny Box Bot Tim offers another way of thinking about human-machine interaction, where confusion and ambiguity become part of the experience.

References