Hearables and Earbuds

Do you have a set of Bluetooth earbuds yet? If not, you will.

The iPhone was the first to kill the ubiquitous 3.5mm headphone jack, but many other manufacturers have quietly followed. Of course, various wired adapters are available, but the replacement of choice is increasingly one of the sets of Bluetooth-enabled earbuds. The catchy name for this market is "hearables", wearables for your ear. But cutting the wire has advantages, too. When you are exercising in the gym, your phone can be beside you without having a wire that might get caught in your weight machine. The latest Bluetooth standard allows for sharing: you can let someone else listen to the same thing as you on their own hearables.

Many of these earbuds are True Wireless Stereo (TWS). First, some terminology: "true wireless" means that there is no connection between the earbuds like there is hidden in the headband of wireless headphones. Of course, stereo means that the earbuds are receiving different channels of music. But TWS is actually a specific part of the Bluetooth standard. Ears are very sensitive to delays between left and right ear, so it is important that the timing between the two channels is perfect. You might assume that both earbuds are connected to your phone, but that is not true. One is connected to your phone, and the other is connected to that first one. A wrinkle is that you might take out the earbud that is connected and you'll expect the other earbud to carry on, so there needs to be an automatic handover sometimes.

Hearables are so popular that Cadence Tensilica licensees shipped over 200M processors inside earbuds during 2019. That number is a little bit of a cheat since there are two earbuds in each set, so a minimum of two processors. What you probably don't know is that often there are as many as three processors in each earbud. That seems a lot of processing power for something about half-an-inch across. But actually getting earbuds to work wirelessly, in stereo, without delay, suppressing the background noise, and providing access to voice commands, is challenging.

Building Earbuds

Designing a set of earbuds is very difficult at the system level because the task has a number of mutually conflicting constraints. Even without knowing anything about it, it is obvious that they have to fit in your ear, they have to be light...and thus there is not a lot of room for the battery. Yet you want long battery life. So whatever you put in there clearly has to be low power.

The challenge is to design something wireless that fits in your ear and addresses users' desires for:

• Voice control ("hey Siri", "ok Google")
• Better call quality
• Higher quality high-resolution audio streaming in stereo
• Better battery life
• More sensor features
• Sharing with other people

Those desires translate into technical requirements and challenges:

• Voice UI such as wake word and voice command 
• Neural Network-based voice processing
• Multi-microphone processing
• Beamforming
• Active noise control (ANC) noise cancellation
• Automatic echo cancelation (AEC)
• Audio codecs
• Speech codecs
• Neural Network-based voice processing
• Audio effects
• Speaker processing
• Sensor processing
• Sharing between multiple wearables

Tensilica DSPs

The Cadence Tensilica line of HiFi DSPs are widely used in hearables, providing both audio processing along with a wide range of codecs, noise and echo cancellation, and more. But also neural network processing that is required for wakeword recognition and simple voice control. For a recent post on using TensorFlow with HiFi, see HiFi DSPs - Not Just for Music Anymore.

One example is Samsung's Galaxy Buds Plus. That's the buds I used at the start of this post. Let's open them up:

That's a lot of microphones and speakers for something so small, three of each. There is a Broadcom SoC in each bud, and inside that SoC, a lot of the heavy lifting is done by Tensilica HiFi 3 DSPs. They last for 11 hours on a charge so the chips are very parsimonious with energy. They filter out ambient noise while letting through anything important, such as someone speaking to you. Like other earbuds, you can control them with taps.

Many other buds are powered by chips containing Tensilica HiFi DSPs, too. For example, the Amazon Echo Buds have three microphones and voice control. They each contain a RealTek SoC where the audio processing is powered by "enhanced Tensilica HiFi-mini compatible 24-bit DSP core providing maxima 160 MIPS computation power."

Personalized Hearing

Everyone's ears turn out to be different. So everyone's hearable needs to be different. I don't just mean size and shape for comfort. I'm talking about the way the sound is handled. We all need a personalized audio experience.

Another company that Tensilica works with is called Mimi. They will give you a hearing test to determine where you have any deficiency or weakness. Their smartphone-based hearing test apps are actually certified as a class 1 medical device. But if you don't have time for that, they can just plug in your age to get an approximation. I well remember when I was in high school and we had a signal generator attached to a loudspeaker. All us teenagers could hear 20KHz easily, but none of the teachers could. The 16KHz signal from the line refresh rates on old TVs used to annoy me (in the UK, 650 lines times 25 times per second gives 16,250; in the US, 550 lines times 30 times per second gives 16,500). I have no idea if I could still hear that, I suspect not, but nobody has a CRT-based TV anymore. Everyone's high-frequency sensitivity declines with age.

At CES, Yipeng, one of our Tensilica HiFi marketing team, tried it. They played her some music normally. Then they added their secret sauce that corrected any hearing deficiencies, boosting some frequencies, attenuating others, and even moving some frequencies to other ones where you would be able to hear them better. She said it was amazing. "I thought my hearing was OK until they did the thing," she said. 

Here's another interesting fact Yipeng told me: "if you are listening to your second language, English for me, your hearing is -6dB". That's a lot. That also explains why, for me, spending a day talking and listening in French, which I speak almost fluently, is so tiring.

Learn More

The latest member of the HiFi DSP family is the HiFi 5 DSP. I wrote about that when it was announced in my post "Alexa, What Is HiFi 5?"

More details on the whole family are on the HiFi DSP audio product page. The table below gives a summary of the HiFi DSPs and compares their capabilities:

Sign up for Sunday Brunch, the weekly Breakfast Bytes email.