Case study - 2021
Google Auditorial

Project summary

An experiment in adaptable storytelling built for blind and low-vision readers, combining text, audio, visuals, and customisable controls to let users experience narratives in the way that suits them best.

Partnered with Google, The Guardian, and RNIB to set a new inclusive standard in web content that doesn't require a full production studio to produce.

My Role

CD - Experience design.

Led the research and ideation/strategy phase. Owned the relationship with the target audience and expert panels. Partnered with a UI design lead to create the full experience and design system. Client liason.

Challenge

Over 97% of the top million websites suffer from accessibility issues, making it difficult for ~300 million blind or low-vision readers to access or enjoy content.

Barriers such as budget and time stand in between story tellers and accessible content creation.

Objective

A content website that adapts to readers’ sensory needs, that can make a piece of storytelling accessible and enjoyable.

A platform that allows it's audience to experience the story in whichever sensory mode suits them — reading, listening or watching, and to customise the visuals and audio along the way.

Research & Insights

Methods

• Interviewed experts and target audience on the opportunity
• Watch alongs
• Existing landscape deep dive
• Established target audience and expert review panels with ongoing channels of communication
• Emphasis on quick prototypes for regular feedback and improvement loop
  

Key Findings /
Design Principles

•  Users should be able to customize visuals, audio, and reading modes per their preference (text, audio, visual) 
• Accessibility must be baked in from the start, not an afterthought
• A multi-modal experience (text + narration + visuals + spatial audio) can optimise storytelling for visually impaired users
  
• Space in design and UI placement creates visual deserts for heavily zoomed in users. All UI needs to be located in a single area if it's to be usable
• Clear and obvious interaction feedback is essential. Audio feedback should be incorporated into UI design systems

Solution

For the audience

• 22 adaptive accessibility features enabling over 100 ways to experience a story: visual filters, focus control, audio controls (speed, narration), captions, etc
• Mode switching: you can read, listen, or watch (visual + audio) versions of the same content
• Soundscapes reinforcing narrative transitions.
• Reusable sensory interaction patterns
• Audio and visual cues on all controls to provide clear and obvious interaction feedback

For the storyteller

• A v/o and a set of imagery is all that's needed to start telling stories. The platform will do the rest
  
• Automatic video subtitling
  
• Screen reader friendly storytelling as standard

The project has already reached over 100,000 readers and makers, and inclusive design lectures from Google and the RNIB are still touring the creative community.

Proposed next steps

• Expansion into more languages, platforms, and content types
• An AI led content creation suite with automated backing track ageneration from an extensive sound library; add a location or a description to get going
• AI generated emotive alt tags for imagery and video
  

96% of testers said Auditorial vastly improved their experience.

Additional learning material

With the help of the Guardian editorial team, I wrote an open sourced accessibility playbook to promote broader adoption and understanding.

"It unlocks a whole different area of the internet that we were shut off from, for so long"

"This was easier to access than anything on a traditional news website or app."

"I hope developers use this project as a benchmark"

An early prototype.
The panel feedback was that the UI was hard to find what with the UI settings and the playback controls being in different parts of the screen. If heavily zoomed in users were to be able to use this platform, we'd need to restrict ourselves to a small area in which all UI would need to reside.

Audio enabled user interface.
Every touch of a button produces and feedback sound. Success and unsuccesful actions, innactive and active buttons, controls and settings are clearly differentiated.

Emotive alt tags.
Editorially written (as opposed to purely descriptive) alt tags were very popular with our test audiences.