Cincom Systems A/B Test
Tools
• Figma
• Miro
• Excel
Role
• UX Design
• UI Design
• UX Research
Objective
• Improved User Experience
• User Feedback
Timeline
• Two Weeks
Problem
Current user feedback indicated that some interface elements are unclear to users while using key features within the product. Specifically, users have challenges understanding how to configure and reconfigure products depending on which path of the journey they take. This study will help us better understand the challenges and needs of the users of this system,
Research Goals
We would like to understand between two designs which is more intuitive to the prototypical users. In addition, we would like to get an idea if there are differences in how long it takes users to complete their tasks between design A vs design B. It would also be helpful in this study to understand which of the "configure" actions do the users prefer and why the would prefer one over the other.
Method
Business users will complete an approval flow by completing a series of tasks. We will recruit 2 separate groups of 8 participants from our pool of users who match our persona for this unmoderated study.
Test Plan
Introduction:
Thank you for taking the time to complete this test. As a software product team, we are interested in how people use our software and your participation in this exercise is of great help.
Scenario:
Your duties are to navigate through the flow. By reaching the cart page. During this study you will see some questions after the prototype experience, you are expected to explain and provide your honest expectations and goals on each screen and then you will navigate through it.
Research Tasks
Tasks:
The tasks for this study provided insights to how effective the interface is in helping user achieve the tasks they were provided. In this unmoderated study, users went through the following tasks which ranged from the findability of specific interface elements as well as gathering some additional metrics.
Unmoderated Tasks:
Launch URL: Take a look at the screen and move onto the next question once it is loaded.
Take a moment to consider this interface. Based on what you see, what do you think you could accomplish here? Please talk through what you see and what your expectations may be. [Verbal response]
Launch URL: Once the new page fully loads, please proceed to the next task. If this screen is already loaded please proceed to the next task.
On this screen how would you approach customizing an N1 Lock? [Success: Yes, No] [5-point rating scale: Very difficult to Very easy]
Launch URL: Once the new page fully loads, move on to the next step.
On the N Series Locks screen, how would you approach adding an "External Lock" [Success: Yes, No] [5-point rating scale: Very difficult to Very easy]
Once you have added an external lock, where would you go to modify the finish of this door lock? [Success: Yes, No] [5-point rating scale: Very difficult to Very easy]
Launch URL: Once the new page fully loads, move on to the next step.
Assume you've configured several products, where would you to see the final pricing for all the products? [Success: Yes, No] [5-point rating scale: Very difficult to Very easy]
Launch URL: Once the new page fully loads, move on to the next step.
Your product has been added to the cart but you need to make a change to the finish, where would you go to make changes to this product? [Success: Yes, No] [5-point rating scale: Very difficult to Very easy]
How difficult or easy did you feel this interface was? [5-point Rating scale: Very difficult to Very easy]
Are there any changes that would make this interface easier to use? [Written response]
Once the new image fully loads, move on to the next step.
After going through A and B, which do you think you would easily remember how to use again? [Written response]
Which task was more visually appealing, A or B? Do you think the visual appeal influenced your willingness to interact with it? [Written response]
Which task is more intuitive, A or B? [Written response]
How difficult or easy was it to find the configure wrench icon or button? [Written response]
Between the clickable wrench icon and the clickable button with the wrench configuration button which do you prefer and why? [Written response]
