Key Insights

🎯 People get the idea, they want to use it. There are so many possible use cases, that we need to prioritize.

🙀 Onboarding needs to improve significantly. Users are automatically located in one corner within the app once they join. So it might happen, that users find themselves outside the audible sound radius of others. This was perceived as a tech defect and users started to reboot, re-connect headphones, etc. instead of moving around and exploring the spatial sound.

🐣 For the next test, we need to recruit a new group of test users, who do not know the project yet.

What did we want to learn?

• Do people understand by themselves that they can move around on the canvas?
• Do users understand that they can drag and drop their video to move?
• Do users understand that the volume of sound is changing according to their distance to other participants?
• Do users explore by themselves how they can interact with single other users or groups?

Besides these guiding questions for our test of course we still wanted to get answers on a meta-level. As we did not really find a way to explicitly get answers, we wrote them down for us, but did not ask them directly:

• Does our app make it more fun to do informal video calls?
• Does our app change the dynamics of informal group video calls?

How did we test?

The informal test of the test
As described, this was the first time we actually did show our app to others. 1 week before we started to use our app for our own meetings, but never with more than 5 people. Knowing, that more users equal potential bandwidth issues in jitsi, we decided to host a regular informal remote after-work-beer in our app. This session turned out to be a lot of fun and helped us to spot some issues to take into account during the formally planned user-test session.

The group situation
Six users were invited to join our app one after another. This set-up was designed to enable the observation of how users arrive, how they explore, and if they discover the interaction pattern Drag and Drop to move around.

This was obstructed by a technical issue, that caused some new users to NOT see or hear other users. The research design and assumption that new users could explore the room, driven by curiosity and the other participants they see, did therefore not work out as planned.

After a (supported) arrival, all users together did explore the app by themselves and quickly started conversations amongst each other or with our team members who were there to moderate and observe.

Individual Interviews

All test users were invited to join a debrief interview individually in which we wanted to understand the following topics in more detail:

• What did they expect to happen/see when entering the app?
• How do users move around?
• How is the change of volume according to positioning perceived?
• The mental model of the test users.
• Use-Cases. When would they use the app?

Overall Findings

1. Our test users seem to be used to tech not working when entering video calls. That results in users questioning their tech periphery before assuming a bug while entering the app. → The onboarding needs an extra smooth experience and feedback if the tech setup of the user (camera, network)is working.
   
2. We saw explorative users who tend to click on things (basically everything) to see if they can interact. And observing users who would tend to wait and see if something is going to happen. → Unless there is a decision to target purely explorative users, we need to give users enough hints so that they can learn the design pattern of our app.
   
3. We saw two opposing user types when it came to anticipated features e like "muting others", "kicking others", "form private circle": One type would prefer to have these tools, to reduce interpersonal friction and awkward face-to-face-interactions. The other type would appreciate that social interactions need to happen in a similar manner to a real live gathering: Moving away from those who talk too much, or are annoying, "Can I join your conversation?" or telling others that you would prefer to continue in private. → A decision on group-size and usage scenario will guide interaction design decisions.
   
4. Having few features, focussing on basic functionality, and a core that works makes room for – sometimes unexpected – usage. Originally we intended to include more functionality. Not implementing it gave us more room to discover user behavior and goals. → Keep the focus of the test simple, observation, and questions after informal usage seems promising.