Synesthesia

The project outcome, titled ´SYNESTHESIA´ features a video interface which creates a digital reflection of the viewer, which is either screen-based (utilizing the webcam of a viewer in a domestic environment) or projected from the image of an external camera if viewed as an installation. As the viewer regards their presence within the piece, a program analyses the RGB colour values of their reflection and begins to produce a line of text in three columns. This text was originally intended to be responsive to the amount of colour within the video, with the left column producing adjectives relating to the colour ´red´, the middle column with another adjective (this time relating to feelings evoked by the word ´green´), and the final right column with a noun, relative to the colour blue.

After preparing the program, it became evident that the possibilities offered, and the capabilities of the program extended further than first theorized, and the program was a tool able to produce poetic lines of text that directly referenced and responded to the viewer in a multitude of ways. The positioning of the words within their columns moves higher and lower in relation to the separate values of red, green and blue within the overall image. The higher the value of red within the image, the higher the word relating to red would you sit within the frame. If the camera were to capture an image that is perfectly white, words evocative of whiteness and purity would appear to float along the top of the frame. If the image was made up mostly with dark colours, the words would sink to the bottom and change to words that are darker and more menacing.

The way we have structured and programmed the algorithm allow it the ability to create text that alludes to the impression that the machine is capable of being empathetic, even though it is unlikely that the machine will ever be able to comprehend such complex human emotions as love and loss, or sorrow and hatred. The emotions conjured by the piece are projected upon it by the viewer, and whilst the computer is unaware of the effect that the words might have, the viewer is most certainly reacting to the text as it summons feelings and memories. ‘Synesthesia’is completely reliant on the performance of the viewer, the way in which they behave in front of the piece is instrumental in its outcome. When faced with the work, the reader finds they are involved within an immersive environment which they can choose to approach playfully or with contemplation. Due to the way in which the text is reactive in its engagement with the viewer, it seems to trace and follow the reader almost like an aura.