While I was at RISD, I realized that there were a lack of opportunities to explore HCI, emerging technologies, and User Experience design. So during my Sophomore year, I recruited design and engineering students across RISD, Case Western, and Brown to create accessible learning spaces to explore Human Computer Interaction and design for emerging technology. I led the team to compete in the NASA SUITS AR Interface Challenge where we created augmented reality user interfaces to help astronauts accomplish mission critical tasks on the moon. 🚀
* Team Members: Bowne Zhou (Co-lead), Viola Tan, Kienan Ahner-McHaffie, Taylor Bjork, Benjamin Mucha, Ivery Chen, Felix Arwin, Jessica Young, Deelia Wang, Echo Yao, Gun Bolukbasi, Ilhee Park, James Lyon, Shu Tong Pan, Karem Yoo, and Santiago Alvarado
Role
Leading a Cross-Functional Team
As a founder of an engineering club at a design school, at times I needed to take on the role of a Project Manager as well as an Experience Designer. Through many obstacles, I learned to navigate the ambiguity of building a team and scoping our challenge, acquiring buy-in from faculty for resources, and motivating my team members despite tight deadlines and little resources.
From continuous dev and design iteration, we successfully built a tool that enabled our end user to navigate and collect geological samples with 100% Accuracy. We were accepted as finalists and invited to NASA Johnson Space Center to conduct usability testing and present to NASA evaluators and astronauts. We published our research finding and innovative technology into two prestigious organizations: IEEE Aerospace and MIT Space CHI, becoming RISD's second-ever published scientific paper.
* Astronaut Kathleen Rubins Pictured
Problem Statement
HMW make the experience of completing critical mission tasks frictionless despite harsh environmental and physical constraints?
Ok so... what exactly are the critical mission tasks astronauts must do? The purpose of lunar missions is to study the landscape for scientific research. Therefore, astronauts are tasked with:
Task #1
Navigating from point A to point B
Safely navigating from lunar base to geological sampling site or assigned destination.
Task #2
Sampling Geology
Documenting samples and field notes to record important scientific findings.
Task #3
Staying Alive
Avoid hazards and checking vitals status.
The Solution
Augmented Reality tools that reduce cognitive load to efficiently assist and automate critical mission tasks.
I created these Figma prototypes to demonstrate functionality and to assist hand off to our developers.
Feature #1:
Intuitive Navigation
Smoothly navigate on the moon with immersive, real time, navigation path optimized to avoid obstacles.
Feature #2:
Hands free documentation
Document geological samples using voice / video recordings and follow along geological sample station specific instructions.
Feature #3:
Quickly check health status
Check vitals at a glance with intuitive graphs and metrics.
Emerging Tech
What were the technological innovations?
To build our product, there were some technical innovations our developers created.
Innovation #1
GPS data on the Moon
The team developed custom low latency Bluetooth device real-time GPS data to position the user on the map mesh.
Innovation #2
Height Map to Assist Navigation
We then hid the map mesh and used line renderer to display the data so one can see the navigational path in 3D space.
Innovation #3
Tab-Transversal System Hardware
Hot key’s wired to our custom device allows for our integration of physical buttons and switches.
The Process
Double Diamond Design + Agile Planning
I drove our team with curiosity and close collaboration between engineering, design and research. By including cross-disciplinary perspectives early on to our research and brainstorming our team was able to stand out with an MVP that was not only functional and fluid, but also technologically innovative.
Discover & Define
What are the needs of our target user?
I lead my team to organize interviews with three major subgroups: Astronauts, Field Geologists, Augmented Reality Specialists. We watched previous lunar missions and read NASA documentation. From our responses we grouped common themes and identified the 4 following pain points:
Pain Point #1
Astronauts are at high risk for cognitive overload.
Due to multitasking under high pressure and the tight schedule of lunar missions, astronauts may feel overwhelmed and struggle with processing information.
Pain Point #2
Navigation is disorienting and risky on the lunar surface.
Because the unfamiliar lunar surface appears homogeneous under the harsh lighting conditions, identifying hazardous craters and sharp rocks is laborious and intensive for astronauts.
Pain Point #3
Locomotion is restricted by their highly pressurized suit.
The highly pressurized astronaut suit makes mission tasks tougher to complete because it's more difficult to grip, walk, and move.
Pain Point #4
Limited field of view.
Current Augmented Reality headsets have small view ports that require users to use precise movements to interact with the interface, making goals such as documenting geological samples for difficult for astronauts.
Ideate & Wireframes
What were all the potential ideas?
Based on Astronaut's challenges & tasks, the teams went through lighting rounds of brainstorming. For each pain point we generated sketches for our initial concepts. View our initial concepts by clicking through the Pain Points.
The team was having difficulty designing for usability, to create alignment I created this journey map as a tool to build empathy and help the team get in the mindset of what it's like to be an astronaut during a spacewalk.
User Flow
To narrow down our collective brainstorming we mapped out the information architecture and separated the designs into user flows.
Iterate & Testing
How was our design backed by data and evidence?
To test the usability and viability of the design we conducted multiple rounds of testing at all stages of the design: from desktop testing to a coded prototype on the Hololens.
Prioritized Vital Features
Due to students schedules and NASA timeline, we had a short dev cycle. This meant I had to prioritize design to maximize utility and feasibility. Identify the priorities and defining the bare bone MVP Experience enabled my team to stay on same page and tackle challenge with confidence.
Changes to the Navigation Mode Based on Human Factors Research
Design Decision #1
Attention is a limited resource
Participants indicated confusion when there are multiple free-floating interfaces in the physical environment.
This is because the human attention span is limited to one plane of distance at a time. (e.g. when users are focusing on task selection on a menu in front of them they are less likely to notice interface changes in the background environment.)
I simplified the navigation user flow to match natural behavior and reduce cognitive load by
Creating separate use cases for each plane of interaction within the HoloLens.
Eliminating visual clutter by aggregating actionable interfaces for different features into one expanding switch menu
Desktop Prototype
My Redesign
This redesign reduces the amount of free-floating interfaces and mimics natural human behavior which increases workflow efficiency.
Design Decision #2
The time it takes to make a decision increases with the number and complexity of choices.
In the older version of the design, when users need to navigate to a specific beacon location
They need to select the map to show possible options
Then select the location
Click start to start navigation
Click the map to open the map again to see where the destination is
I minimized the flow by
Showing all locational selection once the map is open
Having map automatically open following the start of a navigation
This redesign automated the navigation process that initially involved 4 action triggers to needing only 2 triggers.
Final Design
Multiple Modes of Interaction
Navigation System
User selects a destination.
Line populates in the environment.
Geological Sampling Tools
Select a station to view cheet sheet.
Sampling procedures for the 3rd station.
Photograph a sample.
Tested by NASA Evaluator
Learnings
LEARNING #1
Finding Common Ground
I learned that when working collaboratively with different knowledge areas its normal to butt heads. It’s when both parties understand each other’s logic, then they are able to find a common ground resolve to a solution.
LEARNING #2
Forming Interpersonal Relationships
I found that to be a good leader of my team it was important for me be personable. Delegating work takes not only an understanding of people’s strengths but an ability to personalize tasks based on their interests.
Publications
RECOGNITION
2 Published Papers
Developing an Augmented Reality Lunar Surface Navigation System
Institute of Electrical and Electronics Engineers
IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
Designing Mixed Reality Interfaces for Spacesuits
MIT Space CHI
MIT Space CHI is a workshop at MIT that invites speakers to discuss advancing Human-Computer Interaction for Space Exploration.
Next Steps
NEXT STEPS #1
Design Emergency Cases
Our developing team was not able to connect to the telemetry stream and we did not get the chance to test our messaging system. If I were to continue this project, I would design for emergency scenarios.
NEXT STEPS #2
Conduct Usability Testing
I would take usability testing learnings from my Sonos internship to redo the usability testing. For example, I would have participants use a Likert scale and ask follow up questions to fully measure and understand their experience.
-1.webp)
