City of Seattle: IT Service Hub Redesign
My role
UX Researcher
Our client
City of Seattle
The project
IT Service Hub redesign
Project duration
2.5 Weeks
Project team
Peter Thompson - Project Manager
Julia Lu - Interaction Designer
Primary persona
Micheal Stayhan, 46 yr old
Staff Analyst for Seattle Public Utility
Contribution
Stakeholder interviews
Stakeholder mapping
User interviews
Competitive analysis
Comparative analysis
A/B testing
User persona
UX writing
Usability testing
Design performance metrics
Hi-fi Prototype
Style guide
Tool
Pen & Paper
Whiteboard
iMovie
Otter.ai
Word cloud
Sketch
InVision
WebAIM Contrast Checker
After IT consolidation, city employees without prior IT knowledge have a difficult time understanding the flow, content, and language of the IT Service Hub homepage and Request Offerings (RO) in the Service Catalog. This causes users to deviate from using the IT Service Hub and choose to call the IT help desk for support.
The challenge
“I am unable to find what I need!”
— Interview participant
My research included data collection through user interviews, usability testing, and the implementation of multiple design iterations. The analysis of the data revealed the importance of simplifying language across the IT Service Hub platform making it more intuitive and client-centric. I developed a consistent brand voice and tone that prioritized the users needs while the Interaction Designer focused on the implementation of new iconography, colors, and simplifying user flows. The outcome of these changes, improved the average user experience by 49.3 points and feedback from users was very positive.
“User experience improved by 49.3 points.”
The solution
The City of Seattle recently consolidated its IT services necessitating a new tool for its departments to engage with Seattle IT. With 13,000 employees, it became vital to create a user friendly language that would support a single point of contact for all IT requests.
I completed a 2.5-week collaborative project to evaluate and improve the user experience of the City of Seattle’s IT Service Hub. The IT management team provided us with the preliminary UX research that informed the hi-fi prototype they developed in-house.
Background
Our client’s goals were to improve the intuitive nature of the IT Service Hub homepage, develop a client-centric nomenclature to guide users in finding what they need, and explore new colors, icons and filters that fit the mental models of the 13,000 users.
Objective
I applied two main methods of UX research—user interviews and usability testing—to explore and uncover the underlying issues and frustrations the City of Seattle staff members were facing while using the IT Service Hub.
User interviews
I utilized my experience in ethnography to develop my research strategy on this project. I wanted to explore the personal side of the users I was working with before I started with interviews, which I found allowed users to relax and open up.
I conducted interviews to validate the previous findings from the internal UX Design team and explore the underlying pain points users are facing while using the Hub.
Usability testing
I conducted rounds of usability testing to evaluate design changes and validate that my team was moving in a direction that solved and addressed user issues. I performed A/B testing with language changes to discover the best fit. For design performance metrics, I used the System Usability Scale (SUS) to add quantitative data points to my testing process and help measure the success of our design choices. First, I establish a baseline user experience and average SUS score with five users on the IT created prototype. From the synthesized findings our Interaction Designer created a new interface and prototype for further testing and we iterated upon it.
Research
methodology
Study participants
12 City of Seattle Staff members
Seattle Police Department
Seattle Public Utility
Seattle Department of Human Resources
IT Help Desk Seattle
Office of Planning and Community Development
Finance and Administrative Services
Ten individuals with internal management system experience.
To validate the UX research provided by the City of Seattle's internal UX Design team and explore further issues faced by staff members, I conducted 12 interviews with different individuals from six different departments.
“User interviews with 12 individuals from six different departments.”
User interviews
Interview with Seattle Public Utility staff member.
From these interviews I uncovered four main questions that I needed to answer and validate through usability testing.
Where exactly do individuals struggle in using the Service Hub?
Why do people become frustrated with the IT Service Hub?
How do we or do we need to consolidate request offering?
What elements of the language are confusing?
Exploring other management systems language
Before I moved forward with rewriting the language I researched and evaluated other competitive internal management system software. As these are internally facing, I was limited to the homepage and service catalog. This exploration gave me a substantial glimpse into how other softwares have bridged the gap between the IT service language and their primary users.
Language clarification with the IT Management team
Once the pain points of the users were established, I conducted very focused interviews with the IT management team and developers to simplify the IT-centric language of the top 15 most used Request Offerings (ROs) in the Service Catalog. This was to tease out the intent of each RO and help me understand its meaning. Interestingly, often even the Managers in charge of creating the ROs didn’t fully understand what they mean or encompass. If the creators are not sure, it is impossible for the users!
Establishing language values
Users of Seattle IT’s previous platform reported feeling unheard and frustrated by the difficulty of IT jargon. To address these pain points created a comparative and competitive analysis of other internal management systems and customer facing help desks to establish trust, and develop a clear, concise, and helpful brand voice.
Developing
a voice
The unified brand tone of voice offers a foundation of three core values which will unify writing across the IT Service Hub platform. It is designed to inform the user in a conversational manner to assist them in accomplishing their tasks and goals intuitively. The intent of this voice is to bridge the gap between the formal language of the IT hub and the individual mental model of its users.
Core values
of the voice
Thoughtful
A balance between formal titles and informative descriptions.
Simple & Direct
Writing is straight-forward, clear, and easy to understand.
Deliberate & Consistent
Words are used intentionally and users can count on the language provided to inform them.
Setting a baseline
To establish a baseline user experience and explore the flow, expectations, and frustrations, I had five screened users with IT management system experience complete two task scenarios using the hi-fi prototype developed by the IT Service management team.
To test the issues users were facing with the IT-centric language, I asked users to explain each decision, as they completed set tasks.
Collecting quantitative data
I tracked whether a user succeeded or failed a task, which allowed me to set base metrics for the task completion rate. Once a user completed the task scenarios I had them fill out a SUS form. This also established a baseline SUS score, which would allow my team to determine if our language and design choices were accomplishing the the business and user goals.
By changing the language of the platform from IT-centric to client-centric, more users will be able to find what they want in the service catalog as the experience becomes more intuitive. Thus, this change will encourage users to use the tool by increasing user familiarity and learnability.
Hypothesis
After IT consolidation, city employees without prior IT knowledge have a difficult time understanding the flow, content, and language of the IT Service Hub homepage and Request Offerings (RO). This causes users to deviate from using the IT Service Hub and choose to call the IT help desk for support.
Problem statment
From my findings, I synthesized the characteristics of the users and developed a primary persona. This helped humanize our users and this persona assisted in communicating my findings to the IT management team.
Primary persona
Secondary and tertiary persona development
As I was not able to access the demographics of all 13,000 users that use the service hub, I feel it is important to develop at least two more personas to further represent the needs and pain points of the Service Hub Users.
I created a storyboard to illustrate the emotional arc users experience when using the IT Service Hub.
Storyboard
So, what does all this mean? The primary findings for the IT Service Hub Homepage and Service Catalog:
Inadequate search function
The metadata in the search index is not built out and does not match users’ mental models as they are searching for what they need.
Interview findings
“I cannot find what I am searching for because I only know the ballpark of what I need.”
— Interview participant
Language validation and learnability
I validated that the language is not user-centric or comprehensive enough for users to accomplish tasks smoothly. Often users cannot repeat tasks or remember how they accomplish something.
Defining ‘Standard’
I discovered that each user has different interpretations of the word standard causing confusion and frustration.
“People get really confused with Purchase Standard Software, Request Software, and Request a Hardware or Software quote.”
— Interview participant
Beyond the issues of language that users were facing, there were several other challenges with the IT Services that users felt diminished their trust in the system. I thought it was important to outline these pain points:
Beyond
the hub
Need more information up-front
When filling out service requests, users felt they didn’t have enough information up-front and would get deep into forms before finding they didn’t have permissions, or know what information they needed.
Need status updates
Users don’t feel that they get feedback on the status of their ticket and where it is in the process of fulfillment.
IT accuracy and oversight
Delayed, Unfulfilled, wrongly fulfilled requests. e.g. Network Jack activated at the wrong cubical number.
“Nine times out of ten a new employee can’t access their files because the IT department didn’t pay enough attention to the details of what files that person needed access to.”
— Interview participant
The accumulation of these pain points have made users lose faith in the system.
To evaluate the changes in language, I wrote up two task scenarios. The first task was designed to test the language and comprehension of how users experience finding the software they need. The second task was designed to test the confusions users were facing with requesting access to the City Network when working remotely.
After the prototype was completed I tested the language changes with nine users.
Usability testing
Usability test findings
Making the language more customer-centric greatly improved the user experience. These were the major challenges users were facing.
Too many software Request Offerings.
Users don’t understand the difference between the six different software ROs and they often guess when they select one. This results in requests being closed without fulfillment, and even work delays for weeks. Users also don’t remember which software RO they have used when requesting software.
Reporting an Incident vs. submitting a Service Request.
Users don’t know the difference between the Incident and a Service Request. They only see that they have a problem and need it taken care of in a timely manner so they can return to work.
“I find that most people I talk to don’t know the difference between an incident and Service request and they don’t really care, they just want to get things done.”
Task success
To understand how well we solved the users problems with the IT Service Hub we calculated the average success rate of our two task scenarios.
Overall for the first task three out of five users succeeded in completing the task.
For the second task four out of five users were able to complete the task.
It was important to note that we did not have an active search function in the prototype and the filters were not built out. With these elements completed the user may have an easier time finding what they need.
Measuring success
System Usability Scale
To measure our success in improving the users experience we used the System Usability Scale (SUS).
Baseline
We first established a baseline score with the five original users with the preliminary prototype. The baseline experience for users was on average 22.5.
To give context to this score, on an academic grading scale it would be an F and ultimately not an acceptable experience.
Last round of usability testing
After we implemented our changes to color, icons, and language and tested with seven users, the usability score averaged 71.8, which on an academic scale would be a high C and fall in the margin of ‘acceptable.’
We have made great strides in changing the user experience, but it also means there is a long way to go.
Original scale from https://measuringu.com/
Wireframes
After the team project was completed, I redesigned the wireframes and hi-fi prototype to implement my own design solutions based on my research and user feedback.
Style guide
Color palette
For the City of Seattle hi-fi prototype I chose to use a complementary color palette. With an intent to elicit the idea of technology and the city itself.
Typography
For typeface the goal was simplicity, clarity, and legibility. Also balance and stability to establish a sense of quality and trust.
Icons
For the icon selection, I chose simple line based icons to account for usability test findings. These icons needed to align to the users mental model and assist them finding what they need.
Prototype
Hi-fi prototype design
My strategy for creating the hi-fi prototype included implementing user research and findings. I examined and applied Material Design principles, to unify shape language and use hover and shadow to imply selection.
Icon color
I unified the icon color, because users had been distracted and overwhelmed by the color in the previous prototype.
Accessibility
To ensure I was accounting for accessibility I used WebAIM Contrast Checker to validate that all the color relationships on the prototype met the accessibility standards of 4.5:1 (most were much higher).
What we recommend
Remove the word ‘Standard.’
As we met with the client twice weekly, some of the recommendations were implemented early on in the research process. The first thing to go was the word “Standard” because it means something different to each department.
“I don't know what the distinguishing characteristics are between what is considered standard of the city and what is non standard."
— Interview participant
Consolidate Software requests.
As established by the data collected on the task failure rate, despite the language, icon, and color changes, users were still unable to distinguish and understand what software ROs they needed. We recommend creating an overarching RO encompassing all software and then guide users through the process of finding what they need.
Expand search capabilities.
To bridge the gap of the users mental models and the search index, we recommend expanding and adding targeted metadata to ROs to improve the search function.
Continue language edits and testing.
Due to the time constraints of the project, we were unable to complete language edits for all 105 ROs. We recommend using the voice branding guide and more user testing to redefine the language on the remaining 90 ROs.
Continue interviewing and iterating with users.
The most important element about improving the user experience is to continue interviewing and iterating with users that have low to mid tech empathy.
Reflections and lessons
It is important to have someone take notes for you.
I found, it was very important to have another team member in the room to take notes during interviews. It is not efficient to try and take down all the information and important points users are making. I also found that it created a distance between me and the user I was talking to.
If possible, have the Interaction Designer attend the interviews and usability tests.
This project was very research heavy and I was lucky to have my Interaction Designer attend at least half of our user interviews and usability tests. Having her present and taking her own notes streamlined the implementation of making changes to the wireframes and we were able to move into further iterations more quickly.
Have stakeholders witness usability testing.
We were not able to have our clients or stakeholders witness any of our interviews and I feel that this was a missed opportunity. I felt that even though the clients and stakeholders were receptive to all our findings, I think it still would be more powerful to have them witness the challenges faced by their users.
Create a user journey map.
It would have been very valuable to schedule in and create a journey map for this project. I think it would give a cohesive and holistic view of the whole experience of the customer and their decision path and emotional relationship to the process of using the Service Hub. It would also be a fast way of making sure the whole design and development team have an aligned mental model of the users goals and experience.
Clients feedback
“We appreciate all the excellent work your team has accomplished. Now, we can make our case to expand our budget and implement these changes!”
— Client
Our team and client: City of Seattle IT management team. Feb 2020
Thank you!
Other projects
King County Library System
Native app redesign
UX Researcher
Elysian Brewing Company
E-commerce redesign
UX Designer (end-to-end)