The Internet engineering community faces many economic and political barriers to creating a more efficient routing system, according to an IETF 90 technical plenary session about how Internet topology interacts with geography.
“There are, of course, people and governments who are keen to link intranetwork traffic to geography, to geopolitical boundaries,” said panel moderator Andrew Sullivan, an IAB member and director of architecture at Dyn. “But there are also real issues of geography that affect the way the network operates.”
The first presenter was Antonio Gamba-Bari, a PhD student at the University of Toronto’s Faculty of Information and a member of its IXmaps project. IXmaps is an Internet mapping tool that enables end users to see how their personal data travels across the Internet and identify privacy risks. Under development since 2009, the IXmaps tool has gained prominence by offering transparency into Internet traffic surveillance by the US National Security Agency (NSA).
“We encourage people from disparate geographic locations and ISPs to install and run our traceroute program to feed our database with traceroutes through hostname parsing, latency comparison and topological analysis,” Gamba-Bari said. “We geolocate the intermediate routers for mapping the routes their packets take. We highlight the exchange points where these routes pass through suspected sites of NSA interception.”
IXmaps has gathered more than 30,000 traceroutes from more than 250 contributors and tracking data from more than 2,500 URLs.
“There are, of course, people and governments who are keen to link intranetwork traffic to geography, to geopolitical boundaries. But there are also real issues of geography that affect the way the network operates. ”
IXmaps wants to dispel the notion that the Internet is a “cloud,” and instead demonstrate that it consists of a few Internet exchange points that route a massive amount of traffic. In the United States, for example, almost all Internet traffic passes through switching centers in 18 cities. Further, traffic that begins and ends in Canada often travels through the United States, a phenomenon IXmaps calls boomerang routing.
What worries IXmaps researchers is the fact that the NSA has a wide-ranging surveillance system that intercepts, copies, analyzes and stores all Internet traffic on US networks.
“Our work has shown that NSA interception in just 18 US cities can capture nearly 100 percent of US domestic traffic,” Gamba-Bari said. “Foreign traffic that transits the United States is also very likely to be intercepted. From our data, we estimate that 25 percent of domestic Canadian traffic is routed via the United States and, hence, subject to NSA surveillance.”
University of Toronto researchers are transforming IXmaps from a prototype into a more widely usable Internet mapping and policy analysis tool, thanks to a grant from the Canadian Internet Registration Authority (CIRA). The goal is for IXmaps to become more reliable and flexible as well as to improve the accuracy of the geolocation component. IXmaps also hopes to expand beyond North America.
“We welcome offers of help internationalizing IXmaps and making it more sustainable,” Gamba-Bari said. “We will put it under a free Open Source software license to make it easier for others to take it in their own directions.”
Next, the Internet Society’s director of development, Jane Coffin, spoke about the group’s effort to build local infrastructure, which includes Internet exchange points (IXPs) as well as the human, technical and governance infrastructure around them.
An IXP “is a physical location where different IP networks meet to exchange traffic and to keep local traffic local,” Coffin said, adding that it is more than boxes and wires. “Ninety-five percent of this is the human engineering—how we bring the different character sets together of ISPs, network operators, and research and education networks.”
By building IXPs in far-flung locations, ISOC is creating local communities of interest. “You improve the quality of Internet services,” Coffin said. “You drive up demand. Latency comes down. Quality of service usually goes up.”
In addition, IXPs attract content development. “Content is generated by businesses that have confidence in those infrastructures,” she said. “We know this is a catalyst for overall Internet development from our experience and what we’ve seen.”
For example, a new IXP in Kenya resulted in latency reductions from 200 to 600 milliseconds down to a range of 2 to 10 milliseconds. Not only did end users in Kenya see better Internet performance from the new exchange point, but there were cost savings of $1.5 million per year on international transit for local mobile operators. Further, the new IXP facilitated e-government services, with the Kenya tax authority peering there.
Coffin said similar improvements in Internet performance and economics are occurring in Argentina and Brazil. “In Ecuador, before the IXP went in, international transit was $100 per megabit per second,” Coffin said. “It is now local traffic that is $1 per megabit per second.”
“When you improve the quality of Internet services, you drive up demand. Latency comes down, and quality of service usually goes up.”
Coffin said developing countries can use these new IXPs to deploy emerging technology, such as public key encryption, IPv6 and top-level domains. “After a content delivery network [CDN] cache was installed in Quito in 2009, traffic went up by 700 percent. This is local traffic,” she added.
Coffin said there are more than 350 IXPs around the world. ISOC is not only building new IXPs around the globe, but helping grow IXPs that already exist. ISOC provides equipment, technical assistance, and economic guidance, and cooperates with local government.
She outlined two ongoing projects in the Africa Union, called Axis I and II. So far, they’ve held 30 best practices workshops and launched four IXPs this year. In addition, they’ve sponsored five regional meetings across Africa to bring regulators, policymakers, and network operators together to discuss the importance of growing the regional Internet infrastructure.
“There are so many landlocked countries in Africa that it’s important for some of those government entities to try to work together,” Coffin said. “There was one instance in Zimbabwe where it took almost two months to string some fiber about a hundred meters, due to the fact that it was over a bridge that was historic.”
Coffin said ISOC is working with the Regional Internet Registries such as LACNIC in Latin America as well as individual country network information centers like NIC.br in Brazil. Large corporations are supporting the work by providing grants and equipment.
Coffin emphasized that IXPs are designed to keep Internet traffic local for better performance and lower costs, not to be centralized locations for government surveillance. “It is not set up to be a monitoring facility for deep packet inspection. Or at least that’s our philosophy,” she said.
Finally, Amogh Dhamdhere described the data regarding network topology and geography available from the Center for Applied Internet Data Analysis (CAIDA) at the University of California, San Diego. CAIDA operates a network measurement infrastructure called Archipelago that consists of 102 monitors, which collect data about IPv4 and IPv6 traffic in 39 countries. Archipelago collects traceroutes from the entire routed IPv4 and IPv6 space, as well as alias resolution measurements for router-level topologies and measurements of interdomain congestion.
Dhamdhere, a researcher at CAIDA, said the Archipelago infrastructure has collected 6 terabytes of compressed data since 2007, all of which is available to network research and operators. CAIDA provides the raw traceroutes in their original form, curated topology data sets and asynchronous number topologies for IPv4 and IPv6.
“One of the goals of a currently funded project that we’re working on is to make it easier for researchers and people interested in this kind of analysis to actually access this data and do interesting things with it,” Dhamdhere said. “We’re building support for rich queries on this traceroute data, and the idea is to put them together with other kinds of data such as geolocation, annotated AS-level topologies, and router-level topologies.”
Eventually, CAIDA wants to provide data that can be used for regional analysis such as measuring how many routes for Canada-to-Canada communications exited Canada and traverse through US networking hubs.
“Suppose we predicted that a certain region was going to be affected in the sense of a natural disaster like a hurricane or a storm coming up or political instability. We’d like to know all the paths from our current monitors that actually traverse that region,” Dhamdhere said. “These paths might be rerouted or might even go down when something actually happens.”
He said CAIDA is looking for volunteers to host additional Archipelago monitors, which are Raspberry Pi computers that cost only $35 each. People with Archipelago monitors can take advantage of an interactive topology-on-demand service called Vela, which visualizes traceroutes on a map. Another CAIDA service is a DNS-based geolocation service, which gives hints about the geographic location of a domain. Finally, CAIDA offers a repository of tools and data regarding autonomous systems (AS), including a map that uses geolocation data to infer where an AS has deployed infrastructure.
“We have an interface where operators can go in and enter corrections to the inferences we’ve made,” Dhamdhere added.
One work in progress at CAIDA is data about which networks peer at IXPs.
“We’re trying to expand the set of Internet exchange points from which we can actually infer reliably the set of connected networks,” Dhamdhere said. “We’ve recently done some work on mining historical peering data… to figure out colocation by different networks at IXPs, what kind of peering policies they advertise, how all of this evolves over time, and we can actually find interesting things like geographical expansion of networks just by looking at historical peering data.”
CAIDA has used its data to analyze country-level Internet blackouts and outages such as those that happened in the Arab Spring as well as the impact of natural disasters such as earthquakes and hurricanes. CAIDA is “trying to develop metrics and tools to automatically detect outages of this type,” Dhamdhere said.
He said most of CAIDA’s data, research, and tools are available online to the IETF community. “If you’d like to collaborate on anything or just get access to the data, then we’d love to hear from you,” he added.
Sullivan asked the panelists why network operators don’t spend money to build IXPs given that it is more efficient to keep traffic local.
Coffin said that in countries such as Chad, the network operator can’t afford to build new infrastructure. In other countries, such as Cote d’Ivoire, the incumbent doesn’t have an economic incentive to build a community of interest around an IXP. She said ISOC has to explain to governments the importance of IXPs and that the benefits are not always obvious. Being able to donate equipment and staff to install it and train others to run it helps get IXPs built, she added.
With regard to boomerang routing, Gamba-Bari said it may occur because network infrastructure doesn’t exist to support a more direct route. For example, traffic from Halifax to Vancouver must travel through the United States. In other cases, inefficient data paths result from networks deciding to peer with some networks and not others. That’s why traffic from one Toronto building to another might end up going through the United States.
A commenter from the audience, Jacques Latour of CIRA, pointed out that Canada’s network incumbent doesn’t want new IXPs built or to peer with local ISPs because it cuts into their revenue stream. He said all of the new IXPs in Canada are bringing in tier one carriers from other countries that are competing with the Canadian incumbent provider and driving down prices, which is good for consumers. He said CIRA is helping set up new IXPs in every province.
“The core of the Internet is the IXP,” Latour said. “That’s where you generate bandwidth. That’s where content providers go. This is where people can get high volume of data for low cost.”
Coffin added that it takes years to build a community of interest around new IXPs in developing countries, explaining the economic advantages and addressing worries about surveillance systems being installed in these locations. In Trinidad, it took seven years, while Bolivia took three years, she added.
“It is very difficult to grow those communities of interest,” Coffin said. “They are not mushrooms—you don’t just sprinkle some water and they come up. It takes a lot of time and energy.”