Butterfly – A 3D Visualization of Facebook Messenger Conversations

Interactive Art Installation


Optimizing Uptake and Use of Closed Loop Automated Insulin Delivery Systems Through Virtual Reality

We are continuously communicating with different people and listening to different voices. Conversations are such a big part of our lives. They excite us and bring us joy, but at times can also lead us to confusion and depression. It is fascinating to look back on past conversations and see how they have come to shape who and where we are today.

I exported all my conversations from Facebook Messenger in the past three years and apply sentiment analysis to identify trends in the emotional content. A single sentence is represented as a 3D object with a unique shape and color based on the content, animated into the scene according to the time the message was originally sent. The audience can use the Leap Motion sensor to control the butterfly (representing my past self) and navigate it through my different emotional stages.

If we could turn back time, we could recraft what we wanted to say, avoid what we didn’t want to hear, and foresee future misunderstandings… what would be the result today?

The Department of Pediatric Endocrinology & Diabetes in the Stanford School of Medicine is looking to work with Upload to develop an innovative VR intervention that will optimize the uptake of closed loop (CL) automated insulin delivery systems as well as allow for testing around barriers to CL system use.

CL systems comprise of two parts attached to the body: a CGM and Insulin pump and an interface that is connected to a phone to control the system. Through addressing the targeted interventions given in the following scenarios, The Department of Pediatric Endocrinology & Diabetes hopes to create environments that allow patients to have virtual exposures to stressful situations that may occur in daily life while wearing and using a CL system.

The desired outcome is for patients to have realistic expectations of CL, increased situational confidence, improved problem-solving skills and decision making, and reduced worries about the use and trust in their CL system.

After finished a three-month VR Development course at Upload VR, I teamed up with two classmates and worked on this project as an internship project, to practice our unity skills. The project was well received by Stanford medical center and will be used in their diabetes research in January 2018.


Personal Experiment

WebGL, Three.js, Python, Leap Motion, Sentiment analysis