Branch eSign: Experience Benchmark

Project description:

Replace an existing system that facilitates the process of electronically signing of 'deposit account', loan, and 'account maintenance' documents in the branch.

Project impact:

Provided the initial metric that will be used to understand if the design for the new system preformed better or worse than the original system.

Define the problem

Worked with stakeholders to define the problem space and decide what aspects of the overall experience is in an out of scope for this research. Established clear timelines based on the project team's needs and the scope of this research.

Problem Statement:

Establish a baseline of the end user perceptions about their experience using the current eSign system before they start using the new system. In the future we will run the same survey with the new system. Then we will compare the Experience scores over time as large changes are made to the UI.

Create study plan

Define project background, business objectives, research objectives, timeline, and specific questions to ask participants during the study. Work with design, product, and development stakeholders to validate the information in the test plan and ensure the stakeholder's key questions are addressed.

Research objectives:

Use System Usability Scale (SUS) to get a benchmark metric for how end users feel about the usability of the system
Use CSAT to obtain a customer satisfaction score and use open ended questions to dive into the why behind the response.

Study type:

Survey

Recruit respondents and run survey

Recruited 400 potential respondents through established processes.

Contacted participants through email distribution lists, to ask for their participation in the study.

Gave respondents a 2 week period to take the survey.

Send a reminder email 1 day before closing the study.

Analyze data

200 respondents completed the survey

Exported data from Qualtrics and used Excel to create SUS score. Created CSAT based on top 2-box method. Categorized open ended responses and pulled out key themes using Mural.

Create presentation

Created slide deck and Confluence page to convey key learnings from the research to the stakeholders and decision makers. Multiple versions of the slide deck were created to convey information to different levels of stakeholders, e.g. Platform leaders vs immediate project team

Present findings

Presented findings to stakeholders while allowing room for discussion of each key finding. Utilizing this method enables stakeholders to build a shared understanding of the finding and if it is an issue commit to a specific owner and timeframe to address the issue.

Follow-up

Track progress towards rectifying issues by following up with the owner to check in about the status.

Plan for 2nd phase of benchmarking study