By: J. Trent Adams
Date: November 1, 2013
This is a story about how to prevent crime. It’s about experimentation and empirical evidence that shows real world results. It’s about industry working with the open standards community. It’s also about rough consensus and running code. Most important, though, it’s about preventing people from becoming victims of fraud.
The story begins with a loose series of conversations that started about five years ago. A few prominent email senders, service providers, and large mailbox providers started to talk about how to handle the problem of email fraud. While the email community was grappling with the broader difficulties associated with generic spam, these senders were particularly mired in the battle to defend their domains from being spoofed. Attackers were preying on their customers as part of protracted phishing campaigns—attacks designed to extract customer access credentials in order to break into their accounts and do real harm.
To be clear, phishing isn’t just unsolicited email attempting to convince unwary consumers into purchasing something they don’t need. It is outright theft and an undeniably illegal activity. Unfortunately, due to the patchwork of legal regimes in play around the world and the complexity of identifying the criminals, prosecuting those responsible isn’t a straightforward process (even when it’s possible). Further, legal recourse is a reactionary solution and only comes into play after a customer is victimized.
A much better solution is to prevent a customer from becoming a victim in the first place. To explore solutions, senders like PayPal, JP Morgan Chase, and American Greetings worked with mailbox receivers such as Yahoo, AOL, Google, and Microsoft on a series of experiments. They focused on using emerging, open email authentication standards to find the “sweet spot” in how they could be effectively deployed for the most effective protection.
By 2011 it was determined that the right mix was to use Sender Policy Framework (see http://tools.ietf.org/html/rfc4408) in combination with DomainKeys for Internet Mail (see http://tools.ietf.org/html/rfc6376), both of which are specifications being worked through the IETF. To learn how well the scheme was performing, the participants in the experiment instituted a method for the mailbox providers to send email authentication reports back to the senders. Once that was in place, the effectiveness of the model was obvious: for the first time, senders were able to see that in some cases up to 60 percent of email received was fraudulent.
Given that the number of spoofed messages was staggering, this new stream of data was rigorously analyzed to ensure it was accurate. In the end, it was clear that email spoofing a particular domain could be entirely blocked using this new scheme. Unfortunately, it wasn’t a silver bullet against phishing, but it significantly degraded the unfettered attacks. However, companies still needed to figure out how to handle look-alike or cousin-domain attacks that try to masquerade as a sender, as well as common attacks that try to play with how From addresses are displayed in various mail clients. Further, questions remained about how to handle redistributed mail sent via discussion lists and automated forwarding services.
Nevertheless, the success of the experiment was too important to ignore. The next step was to document the experiment as a specification describing the methodology and submit it to the IETF, the same standards body that nurtured the underlying technologies of SPF (Sender Policy Framework) and DKIM (DomainKeys Identified Mail). As part of that exercise, the experiment was expanded to test the existing running code at Internet scale. In January of 2012 the group published the draft specification that became known as Domain-based Message Authentication, Reporting, and Conformance (DMARC).
What happened next was unprecedented: within one year of the specification being made public, an estimated 60 percent of the world’s email boxes were protected by DMARC—that’s 1.976 billion email accounts. Breaking this down further, 50 percent of the top 20 sending domains had published a DMARC policy that resulted in 80 percent of the typical customers in the United States being protected—a phenomenal response that clearly endorsed the value seen in the more limited experiment.
The amazing show of support changed the anticipated trajectory of the standardization plan. Rather than request the formation of an IETF working group (WG) to rework the specification, the authors discussed possible options with the IETF Applications Area WG and its Area Directors. Through conversation it became clear that there were some paths worth considering that would more closely follow the accelerated adoption curve associated with the draft DMARC specification.
The end result was to ask an Applications Area Director to sponsor the specification as a candidate for the Standards Track, at the same time convening a Birds of a Feather (BoF) at IETF 87 in Berlin to discuss DMARC extensions and supporting materials. At the BoF, a charter for a proposed DMARC WG was presented, which drove a discussion about potential work items for the WG. Barry Leiba, one of the Application Area Directors, agreed to sponsor the specification with the caveat that messages sent to the IETF DMARC discussion list clearly said that community experts validated that the specification was ready to move to the Standards Track.
Within a month of the BoF in Berlin, a handful of supportive comments by industry experts were sent to the discussion list. A few items were called out for further work on the DMARC specification, but many could be disposed of during the final-call phase of the Standards Track. While the DMARC specification could be tuned up, it’s clearly functional for its intended purpose and the community supports the process for standardizing it.
It’s been nearly five years since DMARC started its journey and it is now nearing the point where it is recognized as a strong foundational technology on which the email community can rely. But, it’s incredibly important not to lose sight of the true goal of this work: protecting people from crime.
All of us who work tirelessly on these processes should feel incredibly proud that what we do matters. There’s still room for improvement, but rough consensus and running code has already made a significant difference in the lives of a majority people with mailboxes around the world. For that I am grateful for the strong support of everyone involved and for the IETF structure that helps make work like this successful.