Opus Is a Model for Future IETF Standards Work

By: Carolyn Duffy Marsan

Date: November 1, 2012

line break image

The Internet Architecture Board (IAB) showcased the Opus codec at its technical plenary in Berlin as a recent example of how Internet Engineering Task Force (IETF) working groups can produce a high-quality standard, avoid intellectual property minefields, and meet a market need.
Opus is a versatile audio codec that can support both interactive speech and streaming music over the Internet. It was defined in RFC 6717, which was published in September 2012. Opus applications include voice-over-IP (VOIP), video conferencing, streaming music, broadcast, and network music performances. Audio and video from the technical plenary session were streamed to the Internet in real time with the audio being encoded using Opus.
“Opus was designed to work for the vast majority of audio applications,” said Jean-Marc Valin, a Mozilla engineer who coauthored the Opus specification. Valin said the market needed an open standard for a codec that could provide high-quality audio for real-time applications.
Until now, two types of codecs existed: (1) speech codecs that provide the same level of quality as analog telephone service and (2) general-purpose codecs designed for more demanding music streaming applications.
“What we were trying to achieve here is to get the best of both types of codecs—something that works for voice and music, something that works in real-time environments, and something that can still have high quality,” said Valin.
Opus is highly flexible, supporting bit rates that range from 6 to 150 kilobits/sec. It can support narrowband to fullband as well as frame sizes from 2.5 to 60 milliseconds. Even better, Opus can dynamically adjust bitrate, bandwidth, and frame size on the fly. And it is optimized for the Internet, meaning that it has packet-loss concealment and can handle discontinuous transmission.
To achieve this level of flexibility, the Opus design team merged the SILK codec designed by Skype for speech applications and the constrained energy lapped transform (CELT) codec used in streaming music applications. “The result is better than the sum of its parts,” said Valin. “We have a hybrid mode that combines the best of SILK and CELT, and we can switch between modes in real-time without a glitch.”
Adoption of Opus appears solid. Opus is supported in Firefox and Chrome because it was accepted as one of two mandatory-to-implement codecs for the Web real-time communication (WebRTC) application programming interface (API) for browser-to-browser applications. It has been adopted by several VOIP clients and the real-time chat component of gaming applications. And it’s in use by applications like IceCast for music streaming and for standalone music players like Rockbox.
Because Opus is so flexible–it has more than 4,000 possible configurations–it was difficult to test. Greg Maxwell from the Foundation described a series of subjective and objective testing protocols that were used during the creation of Opus. Maxwell said that continuous listening tests “showed Opus doing significantly better than other codecs.”
Cisco engineer Peter Saint-Andre said the Opus working group had to get past skepticism on the part of many IETF contributors, who doubted that the standards body had the expertise to create a codec specification. The working group also had to deal with several intellectual-property-rights disclosures. But engineers who wanted to develop an innovative codec got to work figuring out how to meld SILK and CELT.
“There was a great sense of shared purpose in the work,” Saint-Andre said. “That was really key to making this effort a success.”
Saint-Andre admits that the group didn’t have a testing plan in place ahead of time, nor did it have a plan for outreach to other interested standards bodies or a plan for dealing with patent issues.
“The working group succeeded despite itself,” Saint-Andre says. “We had a core group of people who were really committed to making it happen. They were really smart and well informed and knew a lot about codecs. And that is why we were able to succeed… Opus sounds great, and it is being built into a lot of products. But it wasn’t clear that we would reach that goal right from the start.”
The working group is continuing to refine Opus, including adding new features and speeding up performance. In addition, the group is exploring the creation of a similar standard for a video codec.
But Tim Terriberry, one of the coauthors of Opus and a Mozilla engineer, warned that creating an audio codec standard may be easier than creating a video codec standard.
“Opus was produced by an open, multistakeholder standardization effort that included three of the four major browser vendors,” Terriberry said. “We also had royalty-free licensing with a clear IPR history… Opus started in 2009 and went through 2012, but all of that time was spent making forward progress and not just going around in circles.”
Terriberry said the IETF held a successful Birds-of-a-Feather session about the video codec in Atlanta a year ago. “We encourage people to contribute on the video codec mailing list,” he added.
Representatives from Mozilla, Google, Jitsi, and Meetecho described how they are integrating Opus into their product offerings.
“We are super happy about what Opus has given us,” said Justin Uberti from Google. “Opus handles the core parts so well that we can focus our attention on the new and weird cases, and that’s the way we like it. A big thank you from those of us here at the Google team.”
Emil Ivov told the audience that Jitsi loves Opus. “It all comes down to a decent default choice, and I think we have that with Opus,” he said. “I’m very grateful from the entire Jitsi community.”
During the question-and-answer period, Opus working-group members were asked about lessons learned that could be applicable to other IETF working groups.
“The big lesson for me is that when people come up and point out the shortcomings in what you’ve done, the best response is actually to go back and do the work,” Terriberry said. “People pointed out that we didn’t support switching between all the different modes… So we put in the elbow grease to solve it, and I think the result we got speaks for itself.”
Valin said that the working group helped implementers use Opus while it was still in development. “We were working very closely with them to get feedback and move it in the direction they wanted,” he said. “That proved very useful.”
In related news, the IAB’s Open Mic session prompted a discussion about the IETF’s use of the term Proposed Standard to describe a specification that is ready for deployment, rather than the term Internet Standard, which implies that it is done.
Former IAB Chair Olaf Kolkman said Proposed Standard is causing confusion for the group that sets technical standards for European government procurements. The government group is only allowed to choose formal standards, but the IETF’s own description of a Proposed Standard implies that it is immature. Kolkman said the EU group wanted to specify IPv6, but all the RFCs are at the Proposed Standard level instead of the Internet Standard level.
“This puts the IETF at a disadvantage when compared to other standards organizations,” added Thomas Narten, a former IETF liaison to ICANN and an IBM engineer. “That may not be a big deal from an engineering perspective, but it is a big deal in other circles.”
Eliot Lear, a former IETF liaison to the ITU-T and Cisco engineer, agreed. “If people in other standards bodies feel there isn’t a standard, then they get into the pool. Then we get into having market confusion as a result and multiple standards,” he said.