This Shanghai exhibition explores the emotional side of data

How do you feel about travelling? Excited? Nervous? A mixture of the two? In a new exhibition at The Middle House hotel in Shanghai, artist and music producer Vicky Fung and media artist Keith Lam set out to explore these questions, creating four multisensory data sculptures and musical compositions, inspired by biometric data captured during journeys across four cities. At a moment when we have all experienced restrictions on our travel – in China these restrictions have been especially extreme, and especially long-lasting – this subject matter feels particularly pertinent.

The exhibition, which is titled Encounters Across Cultures, and is staged in the foyer of the hotel, also speaks to another relevant topic: data, and how it relates to our lives. Seen as the new oil – the world’s most valuable resource – we often view data with a level of suspicion, mindful of how our own data is being mined and to what end. But Fung and Lam cast data in a new light, using it to record emotions and then translate these emotions into sculpture and music. In doing so, they make something that many of us see as cold, warm instead – maybe even human.

What was it like working with each other?

Vicky Fung: The idea of working with Keith, who is a very respected media artist, was very exciting to me. This was my first time working with him and I found it very comfortable; his ideas were very innovative yet grounded with human touch. He was also very open to how we shape the works together and I felt that I could offer my best to the collaboration.

Keith Lam: I have known Vicky for a long time. When the House Collective came to me with this project, I said I would like to invite a composer to collaborate with me. I bumped into Vicky the same week and the following day the House Collective called to ask how I’d feel about collaborating with Vicky. It was such a coincidence!

How did you approach the project then?

Keith Lam: We invited four travellers (including Vicky and myself) to travel to a city that we have an emotional connection with. We recorded their emotions throughout the journey, using watches called Upmood, which are made by a Hong Kong science company that makes watches for medical use, predominantly for psychological counselling. We then transcoded this ‘emotion data’ into sculptures and music; each traveller has a sculpture and a piece of music ‘created’ using their emotion data. If could summarise the concept into one sentence it would be “using our own emotion like brush and violin to paint and compose music”.

Vicky, how did you go about transcoding this emotional data into music?

Vicky Fung: After we got back from the trips, I got the emotional data from Keith in the form of Excel spreadsheets. When I got the first round of data, I was stunned by the unlimited stretch of cells in the spreadsheet.

Firstly, I studied the heart rate of each of the travellers throughout their journeys and decided to use the average heart rate to set the tempo of each track, which represents the travellers as well as the city to which they travelled. BPM is a measure of tempo in music.

Then I studied the graphical notations and how the travellers’ data was presented to inspire the tone, dynamics and timbre of the relevant music track of the city and traveller.

Thirdly, I selected the most memorable recorded sounds that the relevant traveller was cognitively aware of during their journeys based on a short list of questions which I asked to each of us personally. For example, Keith demonstrated a lot of calmness during his trip to Beijing; he clearly recalled the sound of cicadas as he was walking, but was shown to be agitated by certain street and traffic sounds he temporarily experienced – all these elements contributed to the elements of the composition ‘Keith in Beijing’.

“It is not something non-human or a so-called ‘cold’ thing, it is the most honest reflection of our planet” – Keith Lam

The project brings together technology and emotion – two things we often think of as being in opposition to each other. What are your reflections on this?

Keith Lam: Data for me is always generated by nature and humans, as my academic background, it always contains cultural value or interpretation of a certain kind of phenomena. It is not something non-human or a so-called “cold” thing, it is the most honest reflection of our planet.

Vicky Fung: I think data is just a quantitative description of certain things, conditions or reactions, quantification of lived experiences. To me, it is not separated from life. Rather, it is a way of capturing and recording. The question is how you absorb, interpret or use it. So long as we remember to use our human instincts, data is a way of advancing humanity. Data is a friend, created by humans. It reflects human nature.

Keith, how would you describe the sculptures?

Keith Lam: Technically, they are composed of spheres, which are like our cells, and move around because of biometric reactions. I was thinking about a nude body sculpture when I was thinking about the form of the sculptures. Those sphere-like cells, at the same time it is kind of like a joint in our body.

The overall effect is very multisensory. Why was this important to you? What do you hope people take away from these works?

Vicky Fung: We would like the audience to interact with the works in their own ways. We often think that we listen to music and we see visual works. That’s fine but what if we could feel the music and touch the emotions? I hope the audience will reflect on their own rhythm of life and be reminded to find their own pace, whatever journeys they are on.

Find out more about Encounters Across Cultures 2023 by the House Collective here.