BAAM Usability Test

ABOUT BAAM USABILITY TEST

BAAM is an online mobile banking service of Bank Melli Iran, with more than 3M active user. Bank Melli Iran is the first national and commercial retail bank of Iran. It is considered as the largest Iranian company in terms of annual income with a revenue of 364 657 billion Rials in 2016. It is the largest bank in the Islamic world and in the Middle East. In this project, TiaraUXD company was asked to perform a set of usability tests for BAAM’s website and application inorder to figure out why the bank’s customers don’t use this online system.

My Roles And Responsibilities

UX Researcher

As a member of a team, I was involved in the entire process of conducting the test. From data gathering to identify the test participants and expert reviews for finding usability problems in the product, to analyzing test sessions’ data and writing the test report.

Facilitator

Guiding the participants through the test, explaining the scenario and each task while giving them instructions, observing their behavior and helping them whenever they had trouble going forward in the test and answering their questions.

Challenges

  • Writing the Test Script
    Since we were dealing with an online banking system and we needed actual money transactions to test the product, it was a challenge to write the script in a way that is believable. To do so, we had to write a reasonable story and ask the participants to do the tasks using account information that we gave them. This issue was problematic specially in those tasks that participants needed to enter some information, like the username and password, card number etc.
  • Lab Usability Test Environment
    The participants in a lab usability test are always aware of being observed, which affects their normal functionality, especially if they are older. Occasionally in the test, it happened that a participant made a simple mistake and was unable to perform the tasks they had just completed. And as the facilitator, it was my job to keep them back on track again reassuring them that they’re doing fine.
  • Detecting the Main Usability Issues in a Qualitative Test
    In lab usability testing, getting an accurate answer to "why" a participant cannot do a task is another challenge. Analyzing all the data from observing the test sessions and comparing it with what was recorded, we were able to discern what causes each task to fail. It's important for UX researchers to consider all the factors affecting participants' behavior.

WHAT WE DID

  • Expert Review
    First of all, we held an expert review session among the team to determine as many usability issues as possible. Four experts took part in the session.
  • Identifying The Participants' Personas
    After analyzing four different user groups based on their age, gender, and occupation, six personas were selected for each test (Baam's application and website) and a total of 12 participants were recruited.
  • Analyzing Baam’s customer service data
    In lab usability testing, getting an accurate answer to "why" a participant cannot do a task is another challenge. Analyzing all the data from observing the test sessions and comparing it with what was recorded, we were able to discern what causes each task to fail. It's important for UX researchers to consider all the factors affecting participants' behavior
  • Test Scripts
    Once we had defined the user flows we wanted to test, the script was written. Since it was an online banking system with actual money transactions, there were a lot of bottlenecks throughout the process, and we had to cross-check the script with the client almost every step of the way.ited.

Results

  • We reviewed and analyzed the quantitative and qualitative data and prepared a final report for our client. Usability issues were classified in two main sections in this report: user interface related issues such as inconsistent primary buttons, and user experience related issues such as unclear error messages and button labels. It was interesting to see how small things like changing the keyboard's language can lead to confusing older users, resulting in them abandoning the product. The test was organized for one of Iran's largest banks, and because of the NDA I am not allowed to disclose furthure details about it.