eric e. dolecki
 
Work System Design SoundTouch controller VideoWave Purchase Journey Listening Modes

Listening Modes

My Role
Concept Design UX/UI Design
Duration
2 months
Tools
Sketch, Trello, Slack, Jira, dscout
Team
UX Research, Marketing, Product
 

Global Research

 On headphones machine learning models

Some senior engineers at Bose found a way to run models directly on a pair of custom Bose NC700 headphones that enabled various listening experiences to be enjoyed when used in conjunction with a forked Bose Music iOS application. This was an initial and technical proof of concept project.

Marketing and Product groups were anxious to try the new modes out to help determine which, if any, would resonate with customers and warrant further development in the hopes to differentiate our product offerings from our competition. Engineering was excited as well. Design had not yet been engaged.

Initially there were twelve listening modes, some of them with names of locations, others with suggested activity matches.

What is a mode anyway?

A mode is a way of describing a shortcut to an experience that can be listened to or experienced. For example, a conversational type of mode may make it easier to hear yourself (self-voice or voice-based transparency), those around you, remove unwanted environmental noise, affect your own voice in some way to make it clearer, etc. It can use digital signal processing, machine learning, specific microphone usage, etc. in order to deliver such specific experiences.

Due to the fact that ML models can run on headphones (without requiring powerful hardware like an Android or iPhone), there is much a reduced overall latency experienced in the system, allowing for quicker processing and less-obvious lags in modified responses to sound events and situations. It allows for pleasant experiences that would not have been enjoyable before.

The initial 9 modes

The initial group of modes was reduced to nine for sake of an initial internal test. Some of the modes were similar enough to warrant consolidation.

Off
Coffee Shop
Commute
Conversation
Library
Listen
Meditate
Outdoors
Study
Workout

Issues

Informal at home testing feedback

When the Marketing and Product Managers took the units home to test, they encountered numerous issues that centered on a few key aspects of the experiences.

Example simplified journey map

  • "There are too many modes."
  • "I need to use my phone every time I want to switch experiences?"
  • "What do the modes actually do?"
  • "These should be easy to use."
  • "They are annoying and confusing."
  • "I can't remember these."
"Put a designer on this."

User-centered design...

The modes existed were a simple List of titles with single-select checkboxes beside them. You could activate one at a time, which was perfectly acceptable, but their actual function was questionable because they had no descriptions. Some of them sounded quite similar. It was quite difficult to remember which mode you were currently using. There were no settings for any of the modes. There was a lot of guess work, too many modes to consider, not enough information to make repeatable and enjoyable experiences possible, and in its current state untestable.

The modes were implemented by engineers for engineers - but even they were having a difficult time. User-centered design was needed.

Invited to the team

I was invited to join the team to tackle the design challenges, to improve the overall experience, to allow us to test not only modes themselves but also the experience design and interface direction that would lead the project to new and desired shippable products. Products that would help differentiate our line and deliver better experiences for our customers and those thinking about potentially buying their first Bose offering.

Initial tasks

  • Meet to discuss a double-diamond user experience design process.
  • Discover: the needs and preferences of end-users. Include hypotheses.
  • Define: insights discovered during the discovery phase, create a design brief and problem statement.
  • Develop: Sketch, quick wireframes, prototypes, high-fidelity assets with implementation documentation for developers. Iterate.
  • Deliver: Produce the final design as testable and repeatable and gather data analysis (qual and quant here).

In an important meeting before setting about the design work, a process timeline was agreed to by the team on how best to approach this user-centered design challenge.

Design improvements

Design study examples

Reduce the number of modes to 6 with a carousel of up to 3 modes

After looking at the current list of available modes, I thought there were too many and I found some whitepapers that backed up the findings that people can most easily retain 5-7 items. This is why automobiles have 6 presets per frequency band in them. So my hypothesis was that concentrating on 6 presets would improve usability. Having half that number active in a carousel would also be a good idea.

Using headphones and not a phone to switch modes

I checked with the project's firmware engineer about repurposing a button on the headphone's left ear cup. It was currently being used to bring up an optional assigned voice assistant (i.e. Siri), so after clearing this with the team - it could be used to toggle through modes. After each toggle, we'd use TTS (Text To Speech) to recite the mode to the user so they knew which mode they were in. A press and hold could toggle mode use on and off.

Introduce the modes through onboarding

Upon first launch of the prototype application (Bose Music), step the user through the modes - giving them more appropriate titles (based on activity), an identifiable icon, and a description. The user can skip through the onboarding - the information will be available on-demand later, but they can become familiar with what each of them does before trying them.

Provide simple settings for each individual mode

The user can choose a settings sheet per mode in order to tweak the settings for that mode. In the settings sheet, the mode will again explain its functionality to remind the user of it's intended purpose.

Manage the carousel in the application

If the user wants to change the modes available in the carousel toggle, they have a designed user interface which allows them to change their order and to set the active carousel using easy drag and drop operations.

Putting the modes list into edit mode turns each mode's settings icon into a gripper handle, allowing the freedom to drag to replace and change the order of mode items in their respective positions.

Weekly design meetings

I called weekly design meetings to go over the current design. These were sometimes attended in person, sometimes using Teams video conferencing while sharing designs in Miro (which some in Visioneering & Validation preferred). I would also call up quick impromptu critiques with User Research to get their thoughts from one of their designers.

Document and manage 3rd party developers (scrums)

We farmed out the iOS application to a team who were already involved in the integration of the iOS to Headphone communication layer - MindTree located in India. I would document designs and deliver that documentation as well as the assets to them in order to allow them to complete our agile weekly scrums.

E-Ink considered

E-Ink would provide battery-powered devices energy efficient means to display the currently selected mode (which would also be displayed in the application) as well as simple battery state. This could be useful for inspection before donning the headphones but was ruled out as it would have added significant development time before being ready to test on device.

My hopes is that it could be paper prototyped in a different set of tests in the future. The display could be on the toggle button itself. rOr beside it if we could get a membrane to match in either case: black background, white text - on midnight headphones, silver background, black text on silver headphones.

Voice assistant requests

A virtual voice assistant was considered. We already have Google and Siri integration and have tinkered with custom VPAs in the past. One could certainly request a listening mode using your voice and if there was a match, switch to that mode. It would most likely be, "Hey headphones, turn on xyz mode." Bose sounds a lot like "Bows" after all. This was deemed to be out of scope for this testing, but as a backup in regards to availability it could be a useful addition one day.

Results

Internal testing & preparation for external testing

We set up eleven units for team members to take home and use. We were able to iterate on the design, the language used, the setting details for each mode, and refine the naming conventions.

  1. A group of 11 team members took the units home for a week's testing. Suggested usage in noisy environments, video calls, etc. to push the modal processing.
  2. Participants were asked to use the headphones in a mode of their choosing for at least one hour a day each day of the test.
  3. All bugs (UI/UX/modal) were reported into the Bose Jira system to allow assignments to be made, items prioritized, and awareness shared.
  4. TestFlight for iOS was used to distribute new builds of the custom Bose Music application being handled by MindTree in India. I supplied them with assets and developer documentation to allow them to complete their weekly scrums.
  5. Firmware updates were handled at work by updating the binary with a custom installation tool on site.
  6. dscout survey questions were finalized with User Research at this time.

External testing

Daily dscout surveys were completed by eleven participants from Bentley College. They were given a Get Started booklet I printed for them, an email about the dscot survey, their reward for participation, and how there would be an hour-long interview at the conclusion when they turned their hardware back in to us.

  1. Eleven testers from Bentley College took a daily dscout survey to collect quantitative data on their experiences.
  2. Participants were asked to use the headphones with modes for at least one hour a day in noisy environments that would test their selected mode's performance.
  3. At the week's conclusion during hardware return, an completion interview was conducted to gather up qualitative feedback.
  4. Participants received 750 Bose Points with which they could use to spend on a Bose rewards website.

Conclusions after analysis

100% of the participants completed testing and came to rely on mode use during their week. They enjoyed the offered modes, the experience of being introduced to them, and liked how easy they were to activate and manage.

  1. I garnered my group in Global Research some visibility and confidence from the company by developing good design solutions, helping User Research to formulate experience questions, handling the asset deliveries and design changes, as well as the documentation and being point person for the interface development by MindTree in India.
  2. Users did not manage the carousel much beyond being interested in the management operation. Once three favorites were decided upon, they became regular features during the test.
  3. 55% of the participants modified the Mode Carousel after initial assignments after onboarding.
  4. Working with a cross-functional team is always inspiring for me and I enjoyed the brainstorming, iteration, attention to system details, and communicating with team members.
  5. 100% of the participants valued the onboarding introduction to modes and quickly associated their icon and title with their function.
  6. Aware, Focus, Speech Detect, and Voice Boost performed the best out of the six for these participants. This shows the importance of having situational awareness while wearing around-ear portable listening devices such as headphones.
  7. The hardware held up well but there were enough hiccups to warrant further technical investigation into the issues.
  8. The modes were received well enough to proceed to "above waterline" project status - meaning that it will gain resources to move it along towards production (road-mapped) - a new line of headphones along with a revised Bose Music (now called Bose) application to support them!