Email Usability Test on Mobile Devices
(Usability Testing at University of Washington)
Team Members: Amol Surve, Erika Dillman, Jenn Chan, and Kanika Bhargava
email USER RESEARCH
TARGET USERS
FullĀtime undergraduate students ages 18-25.RESEARCH GOAL
- Learn which mobile phone email apps students use the most, and what influences their decisions to select those apps.
- Learn how students currently communicate and collaborate with their peers using their phones.
- Learn how students use Outlook on their phones through the usability study.
ONLINE SURVEY
The online survey was completed prior to the usability study to provide us with insights into how students use emails, and other communication and collaboration software. It contained open and closed questions to provide qualitative and quantitative data. In order to collect data on the motivations behind mobile phone use, the survey contained questions about both mobile phone and laptop/desktop use.email STUDY PLAN
We performed our usability test on the Microsoft Outlook mobile application.WORKFLOW
We created a workflow diagram to map out functionalities of Outlook, which would help us design our usability tasks.PARTICIPANT SCREENER
These are participant criteria for the usability study:- FullĀtime undergraduate students.
- Between the ages of 18 and 25
- Smartphone owners (iOS or Android), who can bring their own phones to use in the study.
- Available to participate on the day of the study.
- Download Outlook and sign up before arriving to the study.
SCHEDULE
Each member of the team rotated roles between moderator and note taker. We had 8 target users, and ran sessions concurrently by splitting the team into two. This was done because of the limited time available in the usability lab.
TIME
Maximum 60 minutes per session:- ~10 minutes for introduction and signing consent forms
- ~25 minutes for interview on how they communicate and collaborate with their peers
- ~25 minutes for usability test and post-test survey.
PRINTOUTS FOR TEST DAY
- Consent forms
- NDA forms
- Gratuity forms
- Introductory talking points
- Usability task list
- Interview questions
- Post-test surveys
DELIVERABLES
- Presentation of the findings
- In-depth report of the user research and usability testing sessions
email USABILITY TASKS
In addition to qualitative notes from feedback and users thinking out loud, we also marked each task as complete or incomplete, and asked for the users' confidence level from 1 to 5.
Moderator Action | User Task | Success Criteria |
---|---|---|
Send an email to <team member email>. | Email is sent. | |
Send user an email. | Read this email and save the attachment. | Attachment is saved. |
Set a reminder to read this email later. | Flag or mark as unread or archive or create a new filter. | |
Find the email you set the reminder for. | Find the email by filtering or searching. | |
Mark this email as junk. | Email is in junk folder. | |
If you did that by mistake, can you reverse it? | Email is back in inbox. | |
Remove this email. | Delete email. | |
If you did that by mistake, can you reverse it? | Email is back in inbox. | |
Send user 3 emails. | Remove these emails. | Select all and delete, or go into each email and delete. |
Send user an email | Schedule a meeting with the sender. | Reply and create calendar event, or go to banner and click on calendar and invite from calendar, or reply and state the meeting time. |