Process
Our process can be split up into 3 main parts: ideation, interactive audio and visual construction, and final artefact setup. Details of these processes are elaborated below, along with images and videos to explain each checkpoint.
We also included our project’s initial proposal, consisting of the project’s background, objective, and approach.
Proposal, background
A large part of Singapore’s diversity is made up of foreigners and international students. For most of the international students, it is the first major move in their lives, both physically and mentally. Being far away from one’s home country, especially at a young age, could invoke feelings of homesickness.
Each coming from a different cultural background, the idea behind this project was the result of trying to find common ground. The string connecting us was our priceless memories of ‘home.’ Home, for us, is not a particular physical space but rather is woven deep within words and memories. We believe that it's possible to recreate the essence of home and bring the feeling of familiarity even in an unfamiliar environment. This unity fueled our collective endeavour and pushed us to craft an immersive experience that resonated both personally and universally.
Proposed objective
Our project aims to design a simulation where users can express their words and emotions, reconnecting the audience with their emotions and memories of "home." This will be achieved using interactive features such as audio-reactive visuals, creating a raindrop when sound is detected. The experience will also be enhanced by a visual narrative within the simulation. These elements will allow viewers to define and share their concepts of “home”. As the concept of "home” is very personal and can mean very different things to different individuals, we wanted to focus on a common ground, which is the “sense/feeling of home.”
Proposed approach
We each delve into distinct TouchDesigner tutorials centred around immersive experiences, sharing our insights on what we've learned and seeing how this newfound knowledge can be applied to our project.
Selecting a single TouchDesigner file as a starting point, we collaboratively work on four different visual landscapes, infusing each landscape with our unique perspectives and weaving them into a cohesive narrative. A key element of our collaboration will involve the development of different sets of OBJ files that will bloom from the raindrops, symbolising blooming “memories” as well as the diverse facets of what "home" and “memories of home” mean to each of us.
Finally, we work together to construct a "booth" housing the visuals and interactive elements. This curated space serves to cultivate a meditative and personal environment, delivering an immersive experience that strengthens the viewer's connection with the project.
Ideation
Our ideation process started off with touchdesigner and point clouds explorations. We began developing and discussing potential project ideas out of the visuals that we’re producing during this exploration stage. We went through several project ideas before ultimately landing on our final concept. Stemming from our personal experiences as international students and being far away from home, we began to look towards visualising the abstract concept of "home" through this project.
We realised early on that “home” is not only a place, but the people, experiences, and most importantly, the memories created. We wanted this project to bring the feeling of home to the audience, especially to those whose “home” is thousands of miles away.
Constructing the Visuals
Our journey of constructing the visuals started with individual experimentations of different Touchdesigner tutorials. One of the tutorials that we experimented with was of a barren landscape with glowing raindrops. We felt that this landscape could work well as the base of our project, symbolising how every place we call “home” always starts with a blank slate.
Inspired by the imagery of await, birth, hope and home depicted in the poem “Gwang-ya” by Lee Yuksa, we created a short poetic writing. This writing contributed to the depth and direction of our project’s overall visual narrative, creating a story of how “memories” are created by “droplets of words” which slowly nourish a barren landscape - filling it with recollections and creating the mental landscape which we know and love as “home.”
First, there was barren land
Seeds of Memory scattered across
Droplets of Words perched into the soil
Home blooms from the Surface
We made 4 different versions of the landscape, each representing a different time of the day. The idea was that as the day came to an end, the landscape would slowly be filled with blooming objects, forming a serene and lush landscape. The different visuals were created by changing the colour of the lighting and “sky dome,” as well as experimenting with the texture and amplitude of the land and mountains. Constant, Level and Composite nodes were also used to make the visuals more dramatic. The poetic writing we created was also presented with these visuals, acting as a narration.
Interactive Elements
The interactive element was a crucial part of our project. In the original TouchDesigner tutorial, there were no interactive elements, as the raindrops fell randomly. However, with our project’s narrative of “droplets of words” slowly nourishing a barren land, it was necessary to make sure that the raindrops reacted to and appeared based on the words spoken. For the other scenes, the interactive elements were connected to a MIDI keyboard, which changed the visuals in a way that matched the overall narrative. The interactive elements added another layer to our project, creating a playful and immersive experience.
Immersive Audio
The main thing we kept in mind was that the immersive audio was supposed to help enhance the “sense of home,” creating a meditative warmth that guided the viewer away from negative emotions. This direction helped make the decision to choose string instruments like the cello and harp, as well as sampling the sound of stepping on leaves into low-frequency percussion. These sounds were then heavily saturated with audio effects and plugins such as EQ, Compression, MRhythmizer, RC-20, and Reverb. Specifically, we used ValhallaSupermassive, which made the string sounds almost unrecognisable, yet didn’t take away the distinct warmth that string instruments possess.
Data collected
As the definition of ‘home’ vastly differs from person to person, we wanted to include some of these personal stories into our project, specifically in the fourth and last visual landscape. We conducted a simple survey, asking people to describe one object that reminds them of home, using as many words as they wish. This could be a physical description of the object, a description of how this object makes them feel, the story behind the object or a combination of the three. Given the nature of the question, the range of answers we got was quite wide and personalised.
From this survey, we got a combination of vague imagery and very detailed physical and emotional descriptions. Even one-word answers such as “Nostalgia” and “Scent” which didn’t fit into the category of an ‘object’ were used as inspiration to craft the 3D objects in Blender.
In addition to the collected “object data", our project primarily relied on real-time data collection. Our third "Sunset" visual landscape was created with the audience's voice, making the "raindrops" fall with every word. The remaining visual landscapes responded to the viewer's touch on the MIDI keyboard, adjusting to knob turns, button presses, and joystick rotations.
Artefact
Our final artefact consisted of four visuals housed in an intimate booth where the audience could immerse themselves in the interactive audio and visual elements. Opting for this intimate setting was deliberate; "home" being a deeply personal subject, the booth provided a private, tranquil space, fostering a meditative ambience and allowing ample room for personal reflection.
Initially, our idea was to build a simple cuboid booth using panels/room dividers. However, as our visuals and project evolved, we began to think of better ways to house and present the elements. We ended up on the concept of making a blanket fortress. Not only are these blanket forts a relic of our childhood, filled with meaningful memories, but the fort brings forward childhood memories and expectations even before entering the fort. This was important because we wanted the viewers to feel like they were in a familiar space.
Our final concept centred around crafting a blanket fortress. These forts, imbued with meaningful childhood memories, evoke a sense of nostalgia and anticipation even before one steps inside. This aspect was crucial for us, as we wanted the audience to feel like they were walking into a familiar and inviting space.
Three devices were located in the fort. The MIDI keyboard and microphone were used as interactive input devices, while headphones provided immersive audio. The visuals were rear-projected onto a screen, allowing them to be seen by both the participant and the spectator. This allowed for an immersive experience while also serving as a “campfire,” inviting the people walking by to come and participate in the event.
Our final project was exhibited for a single night, and it was very surreal seeing the whole project come together as a whole. It was rewarding to see everyone’s positive reactions as they participated and experienced the interactive elements of our project.
Conclusion
Our project consisted of so many parts and elements that were each crafted with the utmost attention. From the production of the visuals, crafting the audio, collecting the data, constructing the 3D objects, and making the final artefact, we each did our part in making every element be of the best quality.
One thing that we could have done better was to not lose the incorporation of data in the project. As the project progressed, our focus shifted to creating a more immersive and personal experience, and how we could present the themes of home and memories effectively. We felt that we could’ve found a way to inject more of the interpretation of data into our project.
All in all, completing a project of this scale with the amount of time that we had was a big achievement that we’re proud of. It was a long process of ideation, reflection, planning, and revision. With over 110 versions, it has definitely been a long and demanding process, but all the hard work definitely paid off in the end.