Strategies for Inclusive User Testing with Assistive Technology Users

Written By: Makeeda Johnson and Amy Sullivan 

Conducting remote user testing sessions with participants with disabilities is a valuable and inclusive approach to gathering insights for improving digital experiences. Despite how important these tests are, they’re often missed! Designers and developers can do all the accessibility checks they want, but they won’t replace learning from people who use assistive technologies every day.

In part one of our blog series on Inclusive Design, Disability, Accessibility, and Assistive Technology in User Research, we focused on understanding the key terms associated with this work. We provided resources to those who wanted to dive deeper into these topics. If you’re new to this journey of inclusive user research, we recommend you begin there. If you just need a quick refresher, here are some important points of clarification:

• Accessibility: Refers to access. When something online is accessible, people with disabilities can equally perceive, understand, navigate, and interact with it.
• Usability: This is about designing usable digital experiences. In other words, intuitive, easy to use, efficient, and enjoyable!
• Assistive technology: Similar to keyboards and mouses, assistive technology increases, maintains, and/or improves the functional capabilities of a device such as a cellphone or a laptop. They enable people to use their devices and, in turn, participate in the digital world. Assistive technology types include screen readers, screen magnifiers, and alternative navigation.

In the second installment of our Inclusive Design blog series, we’re focusing on delivering a more applied look at organizing a successful remote user testing session with users with disabilities who use assistive technology. 

Let’s get started!

Preparing for Accessible and Inclusive UX Tests

When recruiting participants with accessibility needs, a thoughtful approach is key. Here are four key areas to consider to make your recruitment process more inclusive:

1. Recruitment: Before diving into research, reach out to a diverse group of participants with different disabilities relevant to your user base. It’s critical to ensure clear communication about the nature and purpose of the session. There are many superb organizations to lean on throughout this process, so utilize platforms like UserTesting or Fable, which offer access to a diverse participant pool, including those with accessibility needs. We’d also suggest partnering with organizations and communities that advocate for people with disabilities. They can help connect you with potential participants.
2. Start with the screeners: As designers and researchers, we know communication is everything! You’ll want to design screener surveys to identify participants with specific accessibility needs – ask about their preferred technologies and any accommodations they might require. Include specific scenarios in the screener to ensure you’re recruiting users representing your target audience. For example, you might consider asking: “Do you require any accommodations for accessibility, including but not limited to visual, auditory, motor, or cognitive support to navigate our website/application?”
3. Invitations and accessibility setup: Prioritize an accessible and inclusive online platform for the testing session. Ensure the chosen video conferencing tool, collaboration platform, and testing interfaces adhere to accessibility standards. In your invitation to the user testing session, you can include which tools you’re planning to use for your accessibility setup so your participant is aware of what to expect from the session.
4. Planning your script:
   • Incorporate helpful visual descriptions for visually impaired audiences! This tip is easy yet super impactful. Facilitators can mention their visual description during a verbal introduction, and it shouldn’t take longer than 30 seconds. These descriptions can provide helpful contextual information to the user they’re interviewing, especially if the user is blind or has low vision, to help eliminate unconscious bias and navigate social situations throughout the meeting.
     • For example: Hi, I’m Amy Johnson, and my pronouns are she/her. I’m Latina with blonde curly hair. Today, I’m wearing a cozy pink sweater. I’m a User Researcher conducting today’s session.
5. Pre-Session setup (for all sessions with people using assistive technology):

• If you’re a first-timer, familiarize yourself with the tools: 
  • Screen Readers: Gain a solid understanding of how popular screen readers work, including their navigation commands, reading modes, and interaction methods. The most commonly used screen readers are JAWS, NVDA, VoiceOver, TalkBack, or Narrator.
  • Screen Magnification Tools: Gain a basic understanding of how each screen magnification tool functions to assist participants better during the session. Familiarity with keyboard shortcuts or gestures specific to each tool can enhance your ability to support participants.
  • Alternative Navigation Tools: Familiarity with tools such as Dragon/NaturallySpeaking, Switch Systems, and Headmouse will enhance your ability to support participants effectively.
• Confirm that the participant’s preferred assistive technology is installed and configured correctly on their device: 
  • Ensure the testing environment is compatible with the selected screen reader, including adherence to accessibility standards.
  • Verify that the testing environment accommodates screen magnification, including appropriate font sizes and sufficient contrast.
  • Ensure the testing environment is optimized for compatibility with the tools, including any necessary software or hardware adjustments.
• Provide a Comfortable Remote Setup: Confirm that participants have a comfortable seating arrangement and that the screen reader is set up to their preferences, such as reading speed and verbosity settings, and offer flexibility in adjusting font sizes, contrast, or other settings based on participant preferences.
• Clear Communication: Establish clear communication with participants to understand their preferences and comfort level with their chosen navigation tool. Ask participants about their specific needs and preferences for interacting with digital content.

 

View this post on Instagram

 

A post shared by Outwitly | UX & Service Design (@outwitly)

Tips for Facilitating Inclusive User Testing Sessions

Once you’ve completed all the pre-work, you’re ready to dive into carrying out these user testing sessions. We’ve broken this section into three categories of facilitation involving various participants:

1. Those who use screen readers.
2. Those who use magnification tools.
3. Those who use alternative navigation. 

Sometimes, you’ll come across participants who use a combination of these accessibility tools. We’re confident that our tips and best practices will guide you through. 

Facilitating User Testing Sessions: Screen Readers

Facilitating sessions with participants who use screen readers, such as JAWS, NVDA, VoiceOver, TalkBack, or Narrator, requires careful consideration to ensure an inclusive and effective user testing experience. Here are nine tips to help you facilitate sessions with participants who rely on screen readers:

1. 1. Use Accessible Content: Design tasks and content compatible with screen readers. If possible, ensure semantic HTML, proper labelling of elements, and descriptive text for images and links. Avoid using content that relies solely on visual cues, and provide alternative means of conveying information. 
   2. Keyboard Accessibility: Verify that all interactive elements, including forms and buttons, are accessible using keyboard navigation, as many screen reader users rely on keyboard commands.
   3. Testing Across Browsers and Devices: Test the compatibility of your website or application with various screen readers across different browsers and devices. Consider platform-specific screen reader nuances like VoiceOver on iOS or TalkBack on Android.
   4. Assistive Technology Compatibility: Ensure that the screen reader is compatible with other assistive technologies participants may use, such as braille displays or speech recognition.
   5. Share Computer Sound: If testing remotely, you must ensure you can hear the participant and their screen reader during screen sharing. Proper sound setup allows you to understand their interaction with the interface fully.
   6. Provide Orientation: Orient participants to the interface’s structure and inform them about the presence of landmarks, headings, and other structural elements that assist with navigation.
   7. Responsive Design: Design content and interfaces that are responsive to different screen sizes and orientations, accommodating users who may rely on various devices.
   8. Check for ARIA Implementation: If using ARIA (Accessible Rich Internet Applications) elements, verify that they are implemented correctly to enhance the screen reader’s interpretation of dynamic content.
   9. Be Patient and Allow Time: Recognize that screen reader users may navigate and consume information slightly slower. Allow participants the time they need to complete tasks.

Enjoying this post? Join our weekly newsletter and be updated on our newest blog posts, job openings, and endless educational opportunities.

Facilitating User Testing Sessions: Magnification Tools

Facilitating sessions with participants who use screen magnification tools, such as Zoom Text, iOS Zoom, Android Zoom, or browser Zoom features, requires thoughtful planning to ensure a positive and inclusive user testing experience. Here are six tips to help you facilitate sessions with participants who use magnification tools:

1. Check for Compatibility: Verify that the website or application being tested is compatible with the selected screen magnification tools. Test the responsiveness of the interface to ensure that magnified content remains accessible and functional.
2. Use Responsive Design: Design tasks and content with responsive layouts that accommodate different screen sizes and magnification levels. Avoid fixed layouts that may cause usability issues when magnified.
3. Demonstrate Patience: Recognize that participants may require additional time to navigate and interact with magnified content. Be patient and allow participants to adjust magnification settings during the session.
4. Consider Platform Differences: Understand the specific features and gestures of screen magnification on different platforms, such as iOS or Android. Be prepared to assist participants using various devices and operating systems.
5. Assistive Technology Compatibility: Ensure that the screen magnification tool is compatible with other assistive technologies the participant may use, such as screen readers or voice commands.
6. Accessibility Testing Tools: Utilize accessibility testing tools that simulate different magnification levels to identify potential issues before user testing sessions.

Facilitating User Testing Sessions: Alternative Navigation

Facilitating sessions with participants who use alternative navigation tools, such as Dragon NaturallySpeaking, Switch systems, or Headmouse, requires careful consideration and preparation. Here are some tips to ensure a smooth user testing session with folks who use alternative navigation:

1. Allow Time for Familiarization: If participants are not already familiar with the testing environment or software, allow time for them to become acquainted before starting the actual tasks. 
2. Be Patient and Flexible: Recognize that participants may navigate at a different pace than users with traditional input methods. Be patient and allow them the time they need. Be flexible in adapting the testing process based on the participant’s preferences and needs.
3. Use a Variety of Tasks: Design tasks that accommodate the strengths of alternative navigation tools. For example, if a participant uses Dragon NaturallySpeaking, include tasks involving voice commands and dictation.
4. Consider Environmental Factors: Minimize background noise and distractions that could interfere with the participant’s use of their navigation tool. Ensure proper lighting to assist participants who rely on visual feedback.

How to Reflect After a User Testing Session

When your test concludes, leave time to ask your participants questions to reflect on how the session went. At the end of your testing script, build this important time in so you don’t forget. Debrief to gather insights into your user’s experience and learn about any challenges they may have encountered along the way. You should also reflect on the session to identify any opportunities for the testing process for participants who use assistive technology! Remember, these opportunities for feedback and reflection are how we improve for future experiences. Here are a few questions to include:

• Is there anything you would like to add?
• Would you like to talk about a specific topic before we finish?
• Any questions you would like to ask me about the project?
• More specifically, Is there anything I should have asked you that I didn’t ask you to understand your experience navigating this software?
• What haven’t we asked you today that you think would be valuable for us to know?
• May I contact you if we have any other questions or for possible further research for this project?
• Can you share any recommendations about how I could improve facilitating this user testing session?

Pro tip: Embrace the power of silence when you ask these questions – don’t turn the recorder off! Many people will give you interesting feedback once the interview is ‘finished.’ 

Similarly, we like to practice adapting the system usability scale to final interview questions. Fable has found a way to adapt the usability scale to be specific to use cases where users have assistive technology. We recommend incorporating Fable’s free Accessible Usability Scale (AUS) and adapting it as needed. Inspired by the System Usability Scale (SUS), the Accessible Usability Scale consists of ten questions administered at the end of a user experience to calculate a score. 

Here is the sample questionnaire:

1. I would use this website frequently if I had a reason to.
2. I found the website unnecessarily complex.
3. I thought the website was easy to use.
4. I think that I would need the support of another person to use all of the features of this website.
5. I found the various functions of the website made sense and were compatible with my technology.
6. I thought there was too much inconsistency in how this website worked.
7. I would imagine that most people with my assistive technology would learn to use this website quickly.
8. I found the website very cumbersome or awkward to use.
9. I felt very confident using the website.
10. I needed to familiarize myself with the website before I could use it effectively.

Best Practices for Facilitating Tests with Assistive Technology

We know there was a ton of information in today’s blog post – we covered preparing for user tests, shared detailed tips for facilitating, and gave our best advice for reflection and debriefing. Feel free to bookmark this blog post as a resource to return to!

We will cap things off by providing a list of best practices to always consider when diving into user testing with assistive tech. 

• Prepare for the Basics: Start strong by making sure you’ve considered the essentials. Whether it’s a modified usability test script or task instructions, prepare all documentation needed to set your participants up for success and empower them to work independently.
• Prioritize Feedback from Real Users: While tools like accessibility checkers are a great starting point, the work shouldn’t end there. Feedback from real users is invaluable. It’s a surefire way to learn how to make a website more user-friendly for those with accessibility needs.
• Foster an Environment of Patience and Understanding: Recognize the extra challenges faced by participants with visual impairments and create a testing environment that is patient, understanding, and accommodating to their needs.
• Design with Assistive Technology in Mind: It’s not just about compatibility with screen readers; it’s about considering the flow and experience of someone using these tools. A site can be technically accessible yet still not user-friendly for someone with accessibility needs.
• Work to Standardize Accessible Design Practices: Acknowledge the lack of standardization in accessible design and advocate for more consistent practices across different websites and platforms. The ultimate goal is not just to follow accessibility guidelines but to develop design practices that are truly more inclusive for all. 

Best of luck in your user testing journey. You’re doing great work, and the Outwitly team is rooting for you!