Usability Metrics

Measuring usability provides critical feedback on the effectiveness of a new web design, indicating whether it has truly enhanced the service or if the old design requires updates.

Measuring apparatus

What is usability and why should it be measured?

Understanding usability is essential. It's often misconceived in its definition, given that it varies based on the user's specific needs and circumstances. Hence, usability should be viewed as a situational assessment, encompassing various characteristics (qualitative indicators). These characteristics are known as the dimensions of usability.

 

Measuring usability is crucial for assessing whether a website, online store, or any other information system meets its intended objectives. By identifying key activities to monitor and knowing where to look for insights, we can gain a comprehensive understanding of usability. Importantly, this approach offers a clear indicator of improvement, demonstrating whether changes in design have indeed resulted in enhanced usability.

Nielsen 5 usability principles

Is measuring usability expensive?

There's a lot of talk around usability measurement focusing on the need for usability testing to observe how end-users actually interact with the platform. This is extremely crucial, but it can become quite expensive, especially if done frequently for a large-scale website.

On the other hand, statistical measurement occurs automatically. This allows for the convenient and cost-effective monitoring of user behavior and the early identification of potential issues.

Woman overlooking a city
Arrows drawn on tarmac and a man standing next to them

Why isn't Google Analytics alone sufficient?

Traditional web analytics, like Google Analytics or Piwik, do provide information on the volume of searches and the keywords used, but further analysis tends to be limited.

Our solution, however, offers the following:

  • A clear overview of the issues
  • Guidelines for solving these problems
  • Real-time measurement of the impact of changes

Do you truly understand how your users are experiencing your site?

There aren't many possible reasons why usability might be poor, but one thing is certain: poor usability almost always leaves its mark. Perhaps you'll recognize some of these signs immediately?

Negative feedback

 

Users are providing a lot of negative feedback

 

Incomplete processes

Users often abandon processes (e.g., form submissions) midway

Dead ends

Users frequently land on 404 error pages

Abandoned shopping carts

Users frequently abandon their purchases midway

Need for help

Users seek for help

Unconventional Journey

Users employ unusual methods to achieve their goals

How do we measure usability at Trinidad Wiseman?

Clipboard
Collect Data

Traditional web analytics (such as Google Analytics or Piwik) do provide information on the volume of searches and the keywords, but further analysis remains limited.

 

Our solution delivers a clear overview of site traffic, including specifics like the frequency of visits to a 404 error page. We delve into identifying usability issues and provide the opportunity to measure the effects of changes in real-time. This allows for a more dynamic and effective approach to enhancing user experience, focusing on targeted improvements based on concrete data.

Desktop monitor
Create a Dashboard

To facilitate an easier overview, we have compiled essential indicators into a dashboard that provides a percentage-based summary of the usability issues encountered.

 

Our specially developed solution is configured on platforms like Google Tag Manager or Piwik Tag Manager and presents the results as a Power BI dashboard. This integration allows for a more streamlined and visual approach to understanding and addressing usability challenges.

Statistic chart
Monitor

On our dashboard, we track a variety of metrics that signify a poor user experience, showing the percentage of user sessions affected by any of these issues.

 

You can view the data on the dashboard over time and filter it based on language and search keywords. This functionality allows for a nuanced understanding of the user experience across different dimensions.

 

The dashboard is a valuable tool for website administrators, helping them to better tag and improve content, making it more easily discoverable and user-friendly. It also enables the comparison of enhancements over time, providing a clear perspective on how changes are influencing user experience. This assists in making informed decisions for optimizing the site's interface and overall usability.

Figure 1: Example of a Search Usability Dashboard 1

Usability dashboard example 1

Figure 2: Example of a Search Usability Dashboard 2

Usability dashboard example 2

Examples of Possible Usability Metrics

The metrics can be further developed and configured as needed. There's always the possibility to delve into more detail and customize the dashboard to suit the client's specific requirements.

  • % of no search results
  • % need for help
  • % of broken pages
  • % of unsubmitted forms
  • % of page refreshes
  • % of 404 page visits
  • % of forgotten shopping carts
  • shopping cart abandonment %
  • % of useful content
  • search filter usability %
Silhouette of a man in front of a screen

How much does it cost to measure usability?

Currently, we offer four distinct modules for usability measurement. These modules are designed to identify user issues in several key areas: using search-related functions, navigating the website, interacting with various forms, and utilizing the shopping cart.

 

The setup for each module involves three stages: configuring tags, creating an API interface, and constructing the dashboard. Additionally, there is a monthly maintenance fee of 100 euros. As the scope of setup varies for each module, and each project has its unique requirements, we provide bespoke quotations tailored to the specific needs of each project.

THE PRICE DEPENDS ON THE COMPLEXITY OF THE PROJECT

Just as each module configuration is different in scope (e.g. the number of tags varies), each project is also different (e.g. the needs of the client, the complexity of the website) - therefore we can provide an accurate quote on a project-by-project basis. To give you an idea of how much a usability measurement project could cost, here is an example of an ongoing project.

Cost of a sample project

  • We offered 3 different modules according to the content of the client's website: search, navigation and forms. The first 2 of these were selected by the client.
  • The client's website is related to the tourism sector and visitors come there primarily to find information. Therefore, the user experience related to search and navigation is at the heart of their site.
  • The search module was priced at € 5940 and the navigation module at € 6900. The difference in price was influenced by both the number of tags and their complexity - tags often need to be created by hand and require more work.
  • Their monthly maintenance fee after setting up the modules is €100, which is a minimum, as the site does not have large amounts of data and is not a more complex site than average. There are no purchase options and no complex forms.

If you wish to discuss your project or have questions about the work being done at Trinidad Wiseman, please feel free to get in touch with us.

Get in touch!

*marked fields are mandatory

Full service (UX, analysis, development, testing)

Atlassian services

Business analysis or service design

System analysis

E-government consultation/development

UX/UI

NeuroUX

WCAG (Accessibility) consultation or audit

Programming

Software testing

Training/other consultation

Other

EUR

/

/

Never submit passwords or credit card information through this form.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Contact

Looking to measure the usability of your service? Contact our Chief Innovation Officer.

Hegle Sarapuu-Johanson

Chief Innovation Officer, Partner

Hegle is a co-founder of TWN and serves as our innovation manager, overseeing the development of our services and organization. With over 20 years of experience in user experience, service design, and information technology, she is one of the pioneers of UX in Estonia and possesses comprehensive skills and expertise across the entire project life cycle. Hegle's responsibilities include managing TWN's innovation process, exploring new service business models, adopting emerging technologies, and implementing new work methodologies.