Grocery Shopping AR app for Hololens
A case study on conceptualizing, designing and prototyping a mixed-reality grocery shopping application for Hololens-II
Last year in 2020, in my final semester of graduating from University of Michigan, I pursued my capstone project in Mixed Reality space. I, with my team members Hannah Wang(Designer) and XYZ (designer) identified problems with cross-cultural grocery shopping that people face. Especially in a multi-cultural country like US that comprises of special grocery stores from various countries.
Sneak peak into experience
Final video experience
Grocery list curation
Navigation & List viewing
Motivation & defining scope of problem
Our team proposed a Hololens-based application to help people with grocery shopping, especially for people who are open to exploring international food and recipes, and people who hope to optimize their shopping experience.
People who like to explore new international items in the grocery store
People who cook recipes in different cuisines and do recipe based grocery shopping
People who like to save time while shopping in large stores
Survey and results
We conducted a survey with these users to understand their behavior towards shopping for international food items in the store and recipe based navigation in the stores. The survey was also to understand the barriers people face while shopping in stores not from their native country. We had 82 participants for the survey!
Initial sketching and storyboarding
Prototyping in 3D - translation & rotation
We used Adobe Aero to prototype the flows and use the figma UI components we made as the first iteration of our UI.
1. Hand interaction to see the list - design explorations
We used resources provided by facebook(Link) to prototype how the hand interactions will look like. We also took motivation from Hololens interaction for Windows Start button.
2. Distance estimation for items in grocery store
Adobe Aero was an accessible tool to create AR prototypes, so we imported our UI elements to Adobe Aero to create some video clips that mimic the AR user experiences.
We understood the various levels at which we need to design our UI based on the distances of the UI elements from the user. So we designed and prototyped based on those distances and came up with three guides: ~.5 cm away, 1 m away and 2 m away. These distances covered the functionalities of our prototype.
We referred to the Microsoft Mixed Reality documentation for designing components in Figma. We cam up with our own set of colors, typography hierarchy and design of components used across the application.
Link to Figma for design system
Testing prototypes in Hololens-2
I am working on testing out the designs in hololens 2 using Microsoft Mesh.
I will be updating this space with my analysis and iterations on the design very soon. Stay tuned!