top of page
Visual Harmony
A voice-activated game which teaches players how to sing in harmony.
Solo Project
My Role: Project Lead, UX/UI Lead, Animator, Illustrator
Timeline
: 6 months
Background
Visual Harmony was a personal project inspired by the challenges I faced while learning how to sing in harmony. In this project, I explored the ways in which people can learn how to sing in harmony using visual cues.
As a result, I wanted to explore the ways in which...
A set of visuals can help people identify the note they’re singing in, eventually helping them to sing in harmony to a particular note.
Struggle with Learning How to Harmonize
Growing up, I played piano and sung at church. One day, I was introduced to the idea of vocal harmonies. However, singing in harmony became a guessing game for me; I could not do it consistently.
Singing for a Christmas service, 2008
Playing piano at Thanksgiving retreat, 2013
Design Intervention: Visual Harmony
Visual Harmony is a voice-activated game that aims to help people learn how to sing in harmony using visual cues. In the Visual Harmony Hotel, players would progress to the next floor when they successfully harmonized to the note assigned to their current floor.
Desk Research on Vocal Harmony
First, I had to learn how to produce a vocal harmony. There are two parts to a song, chords and a melody. The notes that make up the melody usually come from the notes within the accompanied chord.
To sing in vocal harmony, one must choose from the remaining notes in the chord that are not already taken by the melody. A commonly used interval is called the third interval, which denotes a note that is two whole steps/four half steps higher than the note of the melody. The C Major scale consists of C, D, E, F, G, A, B, C. If the melodic note is a C, then its third interval is the E.
C & E played separately, then together on the piano.
C & E sung separately, then together.
Desk Research on Music in Art
Next, I explored the connection between music and art. At first, I delved into Vassily Kandinsky’s work because I was fascinated by his art style. Upon further research, I learned that he had synesthesia, a condition where a sensory input could stimulate multiple senses instead of just one. To Kandinsky, the sense of sight, sounds and color were closely related in his mind.
His unique condition made me even more drawn to his paintings because I, like him, was trying to visually represent music. I took particular interest in the Dominant Curve and Composition IX because of the organic shapes and bold colors he used in those paintings.
Another artist I studied was Georgia Lupi. I admired Lupi’s ability to represent data in clear yet thought evoking ways.
Dominant Curve, Wassily Kandinsky, 1936. Oil on Canvas.
Composition IX, Wassily Kandinsky, 1936. Oil on Canvas.
Above is a part of Lupi’s greater work called Dear Data, a year-long project between her and Stefanie Posavec. Each week, for a year, they gathered and measured specific data about their lives, transformed it into a postcard drawing, and exchanged these postcards.
Determining Users’ Knowledge of Vocal Harmony
Before I dove into artistic representations of music, I needed to determine how much one could grasp the concept of harmony from a visual piece. The purpose of this test is for the viewers to understand what harmony is and sing along to an example. Due to limited resources, I was only able to conduct my test with my classmates (19 people).
After watching the video, participants were invited to complete a survey. Below were the results:
From the survey above, I learned that...
Quite a number of people already know what vocal harmony is, and many of them felt confident in finding harmony on their own after watching the video.
Sketching the Visual Cues
The first test I conducted only measured people’s knowledge of harmony. The next thing I wanted to test was if people could use visual cues to sing the third interval harmony of a certain note.
I began sketching ideas that I called Harmony Tools. These tools were meant to detect if the users are singing in the third interval harmony to the note assigned to each tool.
Above are preliminary sketches of the Harmony Tools and they were focused on different ways of visually representing the progression of reaching the right note.
Designing and Animating the Visual Cues
The first two tools I created, Moon Sailing & Watering Plant, display the successful result of a user singing in the right harmony. My intention here was to show progression and give a reward when something is done right.
However, I realized that the two visual cues do not have any indicators that would help users to know whether they are reaching the right harmony or not.
Moon Sailing
Watering Plant
Visual Harmony Hotel: Putting the Tools Together
Due to time constraints, I was only able to finish seven harmony tools, 4 of which contain indicators that aid users in understanding if they are singing the right harmony.
I placed all seven tools inside a prototype game called Visual Harmony Hotel. The goal of this game is to clear all 12 floors by harmonizing to the pitch given on each floor.
1. First, the players land on a floor and is asked to harmonize to a particular note.
2. Once the doors open, players are given 10 seconds to harmonize correctly to the given note. The visual cue in the room shows players how close they are to getting the harmony correct.
3. If successful, a new floor is unlocked and the player is allowed to advance.
A prototype of this game was built as a website and presented to a design class as part of a thesis project.
Pitch tuners have clear indicators of when a pitch is too high, too low, or just right. These three states helped me to shape the next iteration of my Harmony Tools.
The Snail and Jumping Bird tools work similar to pitch tuners that have extremes at the top (pitch too high) and bottom (pitch too low) of the targeted pitch.
Traditional pitch tuners
Traditional pitch tuners
Drawing Inspiration from Pitch Tuners
Snail: Succeeding to harmonize
Snail: Failing to harmonize
The right pitch for the snail is reached when a player is able to hold the note to stop the platform at the opening. If the player fails to do so under the time limit, the door is closed and snail will not be able to eat the flower.
Jumping Bird: Succeeding to harmonize
Jumping Bird: Failing to harmonize
The player’s pitch for the Jumping Bird is indicated by the bird. Players hit the right pitch when they can hold the note to keep the bird inside the jumping rope.
Water Flowers 1: Succeeding to harmonize
Water Flowers 1: Failing to harmonize
In Watering Flowers 1 more water droplets are added to the line when players are get closer to the desired pitch. If the pitch is too low, no droplets get added to the line. If the pitch is too high, the droplets fall to the right and disappear. If the pitch is right, the water droplet starts building up in the middle and will fall to water the plant.
bottom of page